Kundan Kumar
@kundankumar6
Senior Data Engineer specializing in scalable lakehouse and CDC ingestion, optimizing Spark pipelines for SLA-driven healthcare and genomics data.
What I'm looking for
I’m a Senior Data Engineer with 6 years architecting and owning large-scale distributed data platforms, processing up to 40 TB/day across healthcare, genomics, and semiconductor domains.
My core focus is CDC-based ingestion and SLA-driven batch + streaming pipelines, with deep hands-on expertise in lakehouse architectures using Apache Hudi and Delta Lake. I’ve improved pipeline reliability by 60% and reduced ETL latency by 50% through Spark/PySpark optimization, partitioning, and shuffle reduction.
At Guardant Health, I designed HIPAA-compliant AWS lakehouse ingestion using idempotent writes, schema evolution, checkpointing, schema validation, and automated failure recovery—cutting production pipeline failures by ~60%. I also automated infrastructure with Terraform and containerized workloads on AWS ECS to improve deployment repeatability and reduce infra drift.
Before that, at HippocraticAI and Micron, I built hybrid batch/streaming clinical analytics pipelines and real-time telemetry ingestion—enhancing observability (e.g., CloudWatch SLA dashboards), implementing data quality frameworks for AI workflows, and enabling sub-minute anomaly detection with Apache Kafka.
Experience
Work history, roles, and key accomplishments
Senior Data Engineer
Guardant Health
May 2025 - Present (1 year 1 month)
Architected a HIPAA-compliant AWS lakehouse on S3 using Apache Hudi to process genomics and clinical data with a <10-minute SLA, using CDC-based ingestion with idempotent writes and schema evolution. Reduced production pipeline failures by ~60% through checkpointing, schema validation, and automated failure recovery, and optimized PySpark workloads to improve throughput.
Senior Data Engineer
HippocraticAI
Sep 2024 - Apr 2025 (7 months)
Designed hybrid batch and streaming pipelines for clinical analytics (~1 TB/day), reducing ETL latency by 50% via Spark execution-plan optimization and partition redesign. Built FHIR/HL7 real-time ingestion into a unified clinical data store, improving observability and reducing MTTD by ~40%, while implementing data quality frameworks for AI/LLM workflows.
Data Engineer II
Micron Technology
Aug 2022 - Aug 2024 (2 years)
Engineered distributed ETL and streaming pipelines processing 30–40 TB/day of semiconductor telemetry, XML, and image data across global manufacturing systems. Reduced batch processing time by ~50% using adaptive query execution and join tuning in Apache Spark, and built Kafka-based streaming for sub-minute anomaly detection, reducing RCA time by ~40%.
Software Engineer
MphRx
Mar 2018 - Aug 2020 (2 years 5 months)
Built a patient-practitioner engagement platform supporting report management, slot booking, payments, prescriptions, and discharge summaries for 50,000+ patients. Implemented HIPAA-compliant FHIR/HL7 interoperability into a centralized repository and developed RESTful APIs and data transformation pipelines, integrating clinical AI safety checks to reduce manual medication review time by ~30%.
Education
Degrees, certifications, and relevant coursework
National Institute of Technology Calicut
Master of Technology (M.Tech), Computer Science
2020 - 2022
Grade: 8.9 / 10
Earned an M.Tech in Computer Science from NIT Calicut, graduating with a GPA of 8.9/10.
Maharshi Dayanand University
Bachelor of Technology (B.Tech), Information Technology
2014 - 2018
Grade: 8.5 / 10
Activities and societies: Gold Medallist; University Rank 1
Completed a B.Tech in Information Technology at Maharshi Dayanand University, graduating with a GPA of 8.5/10 and receiving University Rank 1.
Tech stack
Software and tools used professionally
Availability
Location
Authorized to work in
Social media
Job categories
Skills
Interested in hiring Kundan?
You can contact Kundan and 90k+ other talented remote workers on Himalayas.
Message KundanFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
