At Impetus Technologies, I led the migration of 50+ legacy Talend and MuleSoft ETL workflows to a cloud-native Azure Databricks ecosystem, reducing daily batch execution time by ~70%.
I designed Medallion pipelines with PySpark on Azure Databricks, turning raw ADLS Gen2 staging files into optimized Delta Lake tables. I also developed Auto Loader ingestion that reduced file processing latency by 4x compared with legacy runs.
At KPMG, I developed and optimized PySpark notebooks for transaction data, including datasets exceeding 12 TB. I built reconciliation and post-migration verification frameworks, including checks across 200+ tables for a 15 TB+ Finacle migration.
Earlier, at Tata Consultancy Services, I developed and maintained Talend Big Data ingestion pipelines moving transactional data from on-premise databases to cloud target storage. I also integrated Java code and custom transformations into ETL frameworks.
