Open to opportunities

Durgesh Kate

@durgeshkate

Message

Azure Data Engineer with 3+ years building scalable ETL/ELT pipelines on Microsoft Azure and PySpark.

India

Message

What I'm looking for

I’m looking for an Azure-focused Data Engineering role where I can build end-to-end data pipelines, optimize ETL/ELT workflows with PySpark/Databricks and ADF, and deliver high-quality, cost-effective cloud solutions in an Agile/Scrum environment.

I’m an Azure Data Engineer with 3+ years of experience architecting and implementing scalable ETL/ELT pipelines and high-performance cloud data solutions.

I specialize in Big Data processing with PySpark and Azure Databricks, driving large-scale transformation, cleaning, and optimization. I build end-to-end orchestration in Azure Data Factory (ADF) using Triggers, Self-Hosted Integration Runtimes, and complex Control Flow. I’ve implemented Medallion Architecture (Bronze, Silver, Gold) and Data Lakehouse patterns to improve data quality and reliability.

In my projects, I engineered robust insurance analytics pipelines on Azure (including SparkSQL business logic for depreciation/IDV) and improved data quality by 30%. I also deliver actionable insights through Power BI dashboards and support maintainable deployments with Git and Azure DevOps CI/CD.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Smartwork Ideas Pvt. Ltd.

Apr 2023 - Present (3 years 2 months)

Built scalable Azure-based ETL/ELT pipelines using Azure Databricks and PySpark, implementing Medallion Architecture (Bronze/Silver/Gold) and Delta Lake tables for reliable analytics. Improved data quality by 30% through extensive cleaning/transformation and optimized Spark processing performance for large insurance datasets.