At Unizen Technologies Pvt Ltd, I develop and maintain Azure Data Factory pipelines that ingest data from multiple source systems. For the CNOFinancialServices project, I use Azure Data Lake Storage Gen2 to manage structured and semi-structured enterprise data.
I build data transformation workflows with Azure Databricks, PySpark, and SQL, and maintain Bronze, Silver, and Gold layers for reporting and analytics. I also implement incremental loads and SCD Type 1 and Type 2 logic.
On the Blue Yonder project, I built and maintained ADF pipelines and used Delta Lake for incremental processing and reliable data management. I integrated data from SQL Server, APIs, and CSV and JSON files, then supported curated data loading in Azure Synapse.
Across these projects, I monitor pipelines and Databricks jobs, troubleshoot production issues, and tune SQL queries and Spark jobs. I use Azure DevOps and Git for source control and deployments, and Azure Key Vault and Managed Identity for secure access.
