Skip to main content
KK
Open to opportunities

Kundan Kumar

@kundankumar6

Senior Data Engineer specializing in scalable lakehouse and CDC ingestion, optimizing Spark pipelines for SLA-driven healthcare and genomics data.

India
Message

What I'm looking for

I’m looking for a Senior Data Engineering role where I can own SLA-driven batch and streaming pipelines, build lakehouse/CDC ingestion on AWS and GCP, and improve reliability, latency, and data quality for mission-critical healthcare and AI workloads.

I’m a Senior Data Engineer with 6 years architecting and owning large-scale distributed data platforms, processing up to 40 TB/day across healthcare, genomics, and semiconductor domains.

My core focus is CDC-based ingestion and SLA-driven batch + streaming pipelines, with deep hands-on expertise in lakehouse architectures using Apache Hudi and Delta Lake. I’ve improved pipeline reliability by 60% and reduced ETL latency by 50% through Spark/PySpark optimization, partitioning, and shuffle reduction.

At Guardant Health, I designed HIPAA-compliant AWS lakehouse ingestion using idempotent writes, schema evolution, checkpointing, schema validation, and automated failure recovery—cutting production pipeline failures by ~60%. I also automated infrastructure with Terraform and containerized workloads on AWS ECS to improve deployment repeatability and reduce infra drift.

Before that, at HippocraticAI and Micron, I built hybrid batch/streaming clinical analytics pipelines and real-time telemetry ingestion—enhancing observability (e.g., CloudWatch SLA dashboards), implementing data quality frameworks for AI workflows, and enabling sub-minute anomaly detection with Apache Kafka.

Experience

Work history, roles, and key accomplishments

GH
Current

Senior Data Engineer

Guardant Health

May 2025 - Present (1 year 1 month)

Architected a HIPAA-compliant AWS lakehouse on S3 using Apache Hudi to process genomics and clinical data with a <10-minute SLA, using CDC-based ingestion with idempotent writes and schema evolution. Reduced production pipeline failures by ~60% through checkpointing, schema validation, and automated failure recovery, and optimized PySpark workloads to improve throughput.

HI

Senior Data Engineer

HippocraticAI

Sep 2024 - Apr 2025 (7 months)

Designed hybrid batch and streaming pipelines for clinical analytics (~1 TB/day), reducing ETL latency by 50% via Spark execution-plan optimization and partition redesign. Built FHIR/HL7 real-time ingestion into a unified clinical data store, improving observability and reducing MTTD by ~40%, while implementing data quality frameworks for AI/LLM workflows.

MT

Data Engineer II

Micron Technology

Aug 2022 - Aug 2024 (2 years)

Engineered distributed ETL and streaming pipelines processing 30–40 TB/day of semiconductor telemetry, XML, and image data across global manufacturing systems. Reduced batch processing time by ~50% using adaptive query execution and join tuning in Apache Spark, and built Kafka-based streaming for sub-minute anomaly detection, reducing RCA time by ~40%.

MP

Software Engineer

MphRx

Mar 2018 - Aug 2020 (2 years 5 months)

Built a patient-practitioner engagement platform supporting report management, slot booking, payments, prescriptions, and discharge summaries for 50,000+ patients. Implemented HIPAA-compliant FHIR/HL7 interoperability into a centralized repository and developed RESTful APIs and data transformation pipelines, integrating clinical AI safety checks to reduce manual medication review time by ~30%.

Education

Degrees, certifications, and relevant coursework

National Institute of Technology Calicut logoNC

National Institute of Technology Calicut

Master of Technology (M.Tech), Computer Science

2020 - 2022

Grade: 8.9 / 10

Earned an M.Tech in Computer Science from NIT Calicut, graduating with a GPA of 8.9/10.

Maharshi Dayanand University logoMU

Maharshi Dayanand University

Bachelor of Technology (B.Tech), Information Technology

2014 - 2018

Grade: 8.5 / 10

Activities and societies: Gold Medallist; University Rank 1

Completed a B.Tech in Information Technology at Maharshi Dayanand University, graduating with a GPA of 8.5/10 and receiving University Rank 1.

Find your dream job

Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!

Sign up
Himalayas profile for an example user named Frankie Sullivan