At AvidXchange, I rebuilt the payment and invoice data ingest layer in Microsoft Fabric, consolidating one-off extracts into governed pipelines for finance and operations reporting. My Python ETL and ELT jobs process roughly 4 million records nightly, with retry and backfill handling.
I designed an Azure Functions data quality service that checks row counts, null rates, and referential integrity, and alerts the team when anomalies breach thresholds. Automating daily quality assurance cut the team's manual reconciliation time by about 50%.
Previously, at LiveRamp, I designed batch and streaming pipelines for identity and audience data, handling several terabytes per day with Spark and Airflow. I also built quality gates that reduced downstream data incidents by roughly 30%, and migrated a high-volume feed to cloud orchestration, bringing its runtime to under 45 minutes.
