Prasanth Kumar
@prasanthkumar2
Senior data engineer building distributed GCP/Databricks platforms for faster, cheaper, correct pipelines at TB scale.
What I'm looking for
I’ve spent 7+ years designing distributed data platforms on GCP and Databricks for teams at UPS, Ford, Mondelez, Loblaws, and Micron. I focus on the trade-offs that actually matter—streaming vs. batch, cost vs. latency, and correctness vs. throughput—especially for entity resolution, deduplication, and distributed skew handling at TB scale.
At UPS, I architected multi-source marketing ingestion into BigQuery, unifying Google Ads and Meta Ads behind a single incremental framework that handles quotas, auth, backfill limits, and schema drift. I also built a chain-of-custody dispute resolution system that correlates customer disputes with delivery scan events and proof-of-delivery imagery, using Databricks PySpark/Spark SQL plus Python services on GCP for idempotent re-runs and full auditability.
Before that, I led migrations into BigQuery for Mondelez, collapsing data-availability latency from hours to minutes using a hybrid of parallel batch ingestion and Pub/Sub-driven streaming. I’ve also migrated large Teradata and on-prem Spark workloads into BigQuery/Dataproc, reworking joins, window functions, and orchestrations in Cloud Composer to cut processing time, runtime, and cost while keeping pipelines safely re-runnable through replays and schema drift.
Experience
Work history, roles, and key accomplishments
Led migration of 100+ enterprise tables to BigQuery for Mondelez and designed Databricks pipelines for Loblaws, implementing idempotent incremental loads and ID-resolution frameworks.
Re-architected Alteryx workflow into optimized BigQuery SQL, reducing processing time from 24 hours to ~20 minutes, and migrated on-prem Spark workloads to Dataproc.
Reduced Dataproc compute cost by ~50% through cluster tuning and built automated BigQuery ETL pipelines with anomaly detection using Isolation Forest.
Built reusable batch-ingestion frameworks loading PostgreSQL, MySQL, and Oracle data into BigQuery and Cloud SQL, and implemented near-real-time pipelines using Pub/Sub.
Education
Degrees, certifications, and relevant coursework
Usha Rama College of Engineering & Technology
Bachelor of Technology, Electronics & Communication Engineering
2015 - 2019
Bachelor of Technology in Electronics & Communication Engineering from Usha Rama College of Engineering & Technology, Vijayawada, from 2015 to 2019.
Tech stack
Software and tools used professionally
Availability
Location
Authorized to work in
Job categories
Skills
Interested in hiring Prasanth?
You can contact Prasanth and 90k+ other talented remote workers on Himalayas.
Message PrasanthGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
