At Tata Consultancy Services, I optimized a production AWS Glue ETL job by distributing its workload across child jobs, reducing runtime from ~16 hours to ~2 hours. I also engineered PySpark and Python pipelines and built an AWS data platform that improved query performance by 25%.
I migrated Informatica pipelines to AWS Glue and orchestrated workflows with Apache Airflow, Step Functions, Lambda, and EventBridge. On a Databricks project, I built a PySpark ETL pipeline to clean, transform, and load structured tables for analysis.
