At Tata Consultancy Services, I designed and built parameterized Azure Data Factory pipelines for a Banking Customer Relationship Management system, processing 100 GB of daily data from multiple source systems.
I reduced execution time across multiple pipelines by 40% using Spark optimization techniques, and cut query response time by 30% through Delta table Z-ordering and file compaction.
In my personal project, E-Commerce Customer & Sales Intelligence Platform, I built an end-to-end Databricks data platform using medallion architecture, Lakeflow, and Auto Loader.
I also implemented CDC into SCD Type 2 tables and used Unity Catalog for access controls, PII masking, and lineage. During my career break, I completed Trendytech’s Advanced Azure Data Engineering program and built hands-on projects with modern Databricks tooling.
