At The Hackett Group, I architect Azure Databricks and Delta Lake platforms that process 100K+ SAP payment lines daily and deliver agent-ready data within 90 minutes of extraction.
I build parameterized medallion pipelines with PySpark, ADLS Gen2, Delta Lake, and Unity Catalog, including schema validation, deduplication, audit logging, quarantine handling, watermark-based processing, and Delta MERGE upserts. I also automate deployments through Databricks Asset Bundles and Azure DevOps CI/CD, with readiness gates that prevent downstream processing when critical files fail validation.
For Americold, I built AWS Glue and PySpark pipelines supporting inventory, shipment, customer, and profit/loss data, reducing Spark runtime from more than two hours to approximately eight minutes.
I've also built Python and Snowflake ETL pipelines for 8.6M+ NYC Taxi records and backend services with Node.js, Express, MongoDB, JWT authentication, and role-based access control. My work has been recognized with The Hackett Group's Rising Star Award and Employee of the Quarter award.
