At ADP-Member Technical, I worked on OneData, ADP's centralized cloud data platform, and independently designed, built, and owned a Python/PySpark QA framework on Databricks. It replaced a fully manual, multi-step validation process and became the standard QA process for the project's data pipelines.
I automated CDC testing across source and downstream layers, and built row-count reconciliation and schema-comparison tools with audit logging. These brought counts, schema checks, and INSERT/UPDATE/DELETE verification into a repeatable notebook run.
For my Urban Mobility Data Platform project, I built an incremental batch pipeline for NYC TLC Yellow Taxi data using Delta Lake, dbt Core, and Databricks. It includes Bronze/Silver/Gold layers, automated data-quality checks, and current-state and SCD Type 2 taxi-zone dimensions.
