At Epsilon, I owned the end-to-end migration of a production 49-billion-row email marketing ETL pipeline from on-prem Cloudera Hadoop/Hive to AWS S3 and Databricks Unity Catalog. I moved PySpark jobs to Databricks Photon and ported eight Hive tables to Delta Lake.
I also built CI/CD systems for data and web application migrations, using Jenkins, Terraform, and Ansible to automate deployments and infrastructure. In a separate project, I built GenerateZipfile, a Python CLI with a validation layer and 96 pytest unit tests.
