Looking for a job

sid shukla

@sidshukla

Message

Experienced Data Engineer specializing in Azure Databricks, Spark, and scalable ETL pipelines.

India

Message

What I'm looking for

I'm seeking a mid-level Data Engineer role leveraging Azure, Databricks, and Spark to build scalable ETL pipelines, collaborate with BI teams, improve data quality/governance, and grow in a collaborative, growth-oriented company.

I am an experienced Data Engineer with 3+ years designing, developing, and optimizing scalable data pipelines using Azure Databricks, Apache Spark, and SQL.

At Diggibyte Technologies I built automated ETL workflows in Azure Data Factory, triggered Databricks notebooks for daily and monthly processing, and integrated diverse sources including APIs, SFTP, SharePoint, delta tables, and SAP Open Hub.

I designed and maintained end-to-end ETL/ELT workflows with ADF, Databricks, and Apache Airflow, optimized batch-processing for large datasets, and delivered high-quality structured data to the Power BI team for business visualization.

I'm Databricks Certified (Associate and Professional) and Microsoft Certified: Azure Data Engineer Associate, and I use Git-based CI/CD, strong debugging, and data governance practices to improve data quality and operational efficiency.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Diggibyte Technologies Private Limited

Jul 2022 - Present (3 years 1 month)

Designed and maintained end-to-end ETL/ELT workflows using Azure Data Factory, Databricks, and Apache Airflow, automating daily and monthly pipelines and integrating data from APIs, SFTP, SharePoint, delta tables, and SAP Open Hub. Optimized batch processing for large-scale datasets, delivered high-quality structured data to Power BI teams, and implemented CI/CD with Git to improve deployment reli