Prerna Gupta

I am a Data Engineer with 2.5+ years of hands-on experience building and operating production ETL/ELT pipelines on AWS using Python, PySpark, and Apache Airflow. I focus on improving pipeline reliability, automating data-quality checks, and delivering SLA-compliant workflows across large-scale orchestration environments.

At IBM I operated 30+ production ETL pipelines handling 50+ GB daily, maintained 99%+ SLA compliance, reduced MTTR by ~40%, and cut recurring pipeline failures by 30% through systematic root-cause analysis and targeted fixes. I implemented SQL validation checks, schema-drift automation, and Delta Lake upserts to enable idempotent runs and reduce manual effort.

I bring a practical mix of cloud, orchestration, and data-quality tooling expertise—AWS (S3, Athena, Glue, Redshift), Airflow DAG design, dbt testing, Great Expectations concepts, and robust scripting in Python, SQL, and Bash—to reliably deliver timely, cost-efficient data for downstream consumers.