I built an end-to-end data platform migration from on-premises SQL Server to Azure Data Lake Gen2, covering four tables across two source databases. The project used full and incremental loading, with Azure SQL control tables to process new and changed records.
I developed reusable Azure Data Factory pipelines and PySpark and Spark SQL notebooks in Azure Databricks, applying a Bronze, Silver, and Gold architecture. I also added data validation and reconciliation checks and automated orchestration with ADF Triggers and Databricks Jobs; at ADS Softek Private Limited, I contributed to an AI-based surveillance project involving vehicle detection and face detection and recognition.
