At Visa, I redesigned ETL pipelines with Spark to replace legacy Ab Initio workflows, reducing processing time by 80%. I also developed real-time transaction pipelines with Spark and Kafka for ingestion, transformation, and monitoring.
I maintain batch and streaming pipelines across Databricks, Snowflake, and Microsoft Fabric using Python and SQL. By tuning Spark jobs, I reduced pipeline runtime by 35%, and I implemented automated data quality checks across Lakehouse ingestion layers.
At Verizon, I supported the migration of 100+ enterprise datasets from Teradata to BigQuery and built and maintained 30+ ETL pipelines. I also maintained Airflow workflows and developed Tableau dashboards and Shell scripts for routine data operations.
