Rohit Didel

I’m a results-driven Data Engineer with 2+ years building enterprise-scale ETL pipelines that improve speed, reliability, and decision-ready outputs. At American Express, I reduced data ingestion time by 35% by migrating batch workflows and optimizing end-to-end processing.

I specialize in turning raw data complexity into measurable business impact—whether that’s cutting Hive query execution time by 28% through partition pruning and schema restructuring, or improving targeting accuracy by ~22% by automating Hive & Hadoop ETL for marketing campaigns. I also built pipeline refreshes that shrank data availability from 6 hrs to under 2 hrs for 500K+ daily records, and reduced team reporting workload by 40% by consolidating multiple reports into a single Tableau HR dashboard.

I bring strong distributed and cloud engineering through Apache Spark (Scala), including Spark Streaming for 50M+ events and GraphX analysis for 10M+ node graphs for anomaly and pattern detection with sub-second latency. On Azure, I’ve architected full data workflows with ADF → ADLS Gen2 → HDInsight/Databricks and delivered Power BI dashboards that cut incident response time by 50%.