Looking for a job

Yash Varshney

@yashvarshney2

Message

Data Engineer delivering scalable ETL, data warehousing, and Azure-powered analytics solutions.

India

Message

What I'm looking for

I seek senior data engineering roles building scalable ETL and data warehouse solutions on Azure/Databricks, within collaborative Agile teams focused on measurable business impact.

I am a highly adaptable Data Engineer with 6.5 years building end-to-end data solutions that drive business value. I specialize in designing scalable ETL pipelines, data warehousing, and data integration using Azure services, Databricks and PySpark.

I have led development of production Data Warehouse solutions and metadata-driven, idempotent pipelines processing hundreds of GB daily, enabling analytics for dozens of clients via PowerBI. I engineered a complex automated archival pipeline, implemented robust data validation and logging, and optimized costs through targeted platform changes.

I collaborate with stakeholders to prioritize initiatives, champion Agile practices, and provide technical leadership on best practices for performance, scalability, and maintainability. I quickly learn new technologies and focus on delivering reliable, business-focused data products.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

Mars, Incorporated

Jan 2025 - Present (8 months)

Developed a Databricks-based data warehouse for US sales analysis and championed Agile adoption to improve delivery timelines and stakeholder alignment.

Databricks Data Warehousing ETL Pyspark Python SQL Delta Lake Agile

Senior Data Engineer

Hasbro Inc

Jun 2024 - Dec 2024 (6 months)

Built and maintained complex ETL pipelines using Azure Data Factory and PySpark in Databricks and implemented monitoring to improve operational oversight of pipelines.

Azure Data Factory Pyspark Databricks ETL Data Pipelines Python SQL

Senior Data Engineer

Vision Group Retail

Dec 2021 - May 2024 (2 years 5 months)

Engineered metadata-driven, idempotent PySpark pipelines on Databricks and Synapse to process 100s of GB daily across 30+ datasets and delivered a Synapse DWH enabling PowerBI analytics for 50+ clients.

Pyspark Databricks Delta Lake Azure Functions ETL Power BI

Data Engineer

Pure Software

Sep 2020 - Nov 2021 (1 year 2 months)

Translated business needs into data solutions using Azure Data Factory and Dataflows, implemented SCD2 stored procedures in Azure SQL DB, and optimized pipelines processing over 500 GB daily to reduce compute costs.

Azure Data Factory Delta Lake Azure SQL SCD2 Stored Procedures Python ETL

Associate Data Engineer

Oxygen2 Innovation

Jan 2019 - Aug 2020 (1 year 7 months)

Developed ETL pipelines with Azure Data Factory to ingest and clean image/text data, deployed Python ingestion scripts into MongoDB, and provided DBMS technical support for client integrations.

Azure Data Factory Python MongoDB ETL Database Administration

Education

Degrees, certifications, and relevant coursework

Dr. A.P.J. Abdul Kalam Technical University

Bachelor of Technology, Electrical, Electronics And Communications Engineering

2015 - 2019

Completed Bachelor of Technology in Electrical, Electronics And Communications Engineering at Dr. A.P.J. Abdul Kalam Technical University from 2015 to 2019.