Rahul Kumar

I’m a Big Data Engineer with 2.1 years of experience specializing in Apache NiFi, Apache Kafka, and large-scale data pipeline engineering. I focus on building secure, real-time, highly available data integration solutions across diverse platforms, with strong ownership of monitoring, automation, and performance optimization.

I’ve designed and managed scalable data pipelines using Apache NiFi, integrating sources such as Slack, Discord, Zoho, Intercom, Google Reviews, Apple App Store, and customer support platforms. I implemented Kafka integration with NiFi for real-time streaming, delivering low latency and high throughput, and I’ve built multi-cluster Kafka replication using MirrorMaker 2.0 for active-active configurations across environments.

I’m experienced in configuring NiFi clusters with HA and load balancing to ensure fault-tolerant, secure data flows. I develop NiFi flows for MongoDB, SQL, and API ingestion to enable real-time data enrichment and transformation, and I automate NiFi administration tasks like flow deployments, RBAC configuration, and scaling using Python and Shell scripting.

To keep systems reliable, I integrate Prometheus, Grafana, and the ELK stack for end-to-end observability—performance metrics, error tracking, and log analysis. I also support secure operations by implementing Azure SSO and RBAC in NiFi, using LDAP where applicable, configuring KeyVault for credential management, and applying TLS/SSL encryption, including custom Groovy processors for advanced transformations.