At HCL, I designed and maintained Azure Databricks data pipelines that sustained 99% uptime for daily production processing. I also built ingestion workflows on Azure Blob Storage and ADF, cutting ingestion latency by 60%.
At IBM, I built Azure ingestion workflows and PySpark structured streaming modules to make data available for analytics. I also helped move legacy data from the local Hive Metastore into a centralized governance framework using Databricks Unity Catalog.
Across my data engineering work, I integrated MongoDB into ingestion pipelines and built CI/CD pipelines with Jenkins and Terraform. I also created Power BI and SSRS reports with custom DAX measures to give stakeholders faster access to KPIs.
