At ShyftLabs, I support an existing ETL application built on Databricks and develop PySpark code for new features.
At Innovation Incubator Advisory, I supported an enterprise data warehouse migration from MySQL to MBS using AWS services and structured migration workflows. I also built PySpark ingestion and transformation pipelines for analytics workloads.
At Coforge DPA (Southwest Airlines), I modernised legacy ETL workflows by converting Ab Initio graphs into PySpark processing on AWS EMR. I optimised PySpark jobs with partitioning, caching, broadcast joins, and Spark tuning, reducing processing time by 30%.
I’ve also configured Apache Hudi indexing and data-skipping features, and worked on AWS Glue optimisation that reduced reported execution cost from $100 to $1.80 per run. I hold an AWS Certified Cloud Practitioner certification and am currently upskilling in Databricks.
