Open to opportunities

Harsha B K

@harshabk

Message

Backend developer focused on scalable data systems, Kafka streaming, and high-throughput ETL.

India

Message

What I'm looking for

I seek backend/data-engineering roles where I can build scalable streaming/ETL systems, optimize performance, and collaborate on high-impact data platforms.

I am an enthusiastic backend developer with two years of professional experience building scalable ETL pipelines, stream-processing solutions, and resilient data platforms. I have designed Kafka-based monitoring and control systems, optimized Spark Structured Streaming throughput by 4x, and processed batches of ~90 million records per day while ensuring data integrity.

I thrive in collaborative environments and deliver production-ready solutions using Java, Spring Boot, RocksDB, Docker, and cloud-friendly tooling. I have upgraded Spark clusters, implemented application-wide pipeline versioning and migration scripts, authored RESTful APIs, and earned internal recognitions for consistent, innovative delivery.

Experience

Work history, roles, and key accomplishments

Current

Engineer II

Current

American Express

Feb 2025 - Present (8 months)

Built a Kafka lag monitoring solution that prevents cluster restarts until consumer lag is zero and implemented byte-level EBCDIC processing to handle multiple row identifiers, enabling processing of ~90M records/day in 15M record batches.

Java Kafka EBCDIC Processing Spark Structured Streaming Distributed Systems Docker MySQL Monitoring

Software Engineer

Subex

Aug 2023 - Feb 2025 (1 year 6 months)

Designed item-level pipeline controls, application-wide pipeline versioning, and static windowing; upgraded Spark from 3.3.0 to 3.5.0 and increased Structured Streaming throughput 4x while reducing state memory and latency.