Afroj Shaikh

I’m a Data Engineer with 4 years of experience designing and optimizing scalable data pipelines using PySpark, Databricks, AWS, and the Hadoop ecosystem. I focus on ETL modernization, distributed data processing, and building cloud-native solutions with strong performance, monitoring, and reliability.

In my current role at Barclays, I developed scalable PySpark pipelines for enterprise banking control frameworks and automated migration of legacy SQL workflows into standardized PySpark frameworks, reducing manual conversion effort by 70%. I’ve also built reusable ETL framework components for logging, auditing, exception handling, and control validation.

I bring practical engineering rigor from Capgemini, where I optimized PySpark ETL pipelines on AWS EMR and Hadoop, reducing resource utilization by 50% and improving pipeline performance. I design orchestration with Apache Airflow, improve Hive query performance by 35%, and strengthen production resilience using AWS Fault Injection Simulator—supporting releases with zero critical production incidents.