At Accenture, I developed and optimized PySpark data pipelines, cutting production runtime by 37% through Spark optimization techniques.
I also orchestrated pipelines with Lakeflow Jobs and built ingestion and transformation pipelines using Lakeflow Spark Declarative Pipelines on a Medallion architecture. I applied data quality expectations and Unity Catalog controls, including row filters and column masks.
Previously, at IZERTIS, I designed an event-driven ingestion pipeline using AWS Lambda, AWS Glue, EventBridge, and Amazon S3. At NTT Data, I developed ETL/ELT processes and Python and Apache Airflow data pipelines, and optimized Spark workflows.
