Looking for a job

vaibhav Agrahari

@vaibhavagrahari

Message

Senior Azure Data Engineer at LTIMindtree building Azure Databricks pipelines that process batch and streaming data.

India

Message

At LTIMindtree, I designed PySpark pipelines on Azure Databricks using a medallion architecture, processing batch and streaming data into Delta Lake tables for enrichment and reporting.

I developed a metadata-driven Azure Data Factory framework to ingest structured and unstructured sources into ADLS Gen2, cutting development overhead by 20%. I also built data quality checks that reduced discrepancies by 20%.

At Tata Consultancy Services, I built and maintained Databricks workloads for reporting and analytics, and introduced validation controls for market risk data. Earlier, at Mphasis, I developed SQL scripts to prepare datasets and monitored scheduled pipelines.

Experience

Work history, roles, and key accomplishments

Current

Senior Data Engineer

Current

LTIMindtree

Dec 2024 - Present (1 year 10 months)

Designed end-to-end data pipelines using PySpark on Azure Databricks with medallion architecture, processing ~100 GB of batch and 500+ events/sec of streaming data. Developed metadata-driven ADF framework and automated data quality framework, reducing discrepancies by 20%.

PySpark Azure Databricks Azure Data Factory Delta Lake Unity Catalog Spark Streaming Python SQL ADLS Gen2 Medallion Architecture Databricks Workflows Data Quality Data Modeling CI CD GitHub Actions

Systems Engineer

Tata Consultancy Services

Jan 2023 - Dec 2024 (1 year 11 months)

Built and maintained PySpark workloads on Databricks, refining raw data into curated Delta tables for daily reporting and analytics. Introduced pipeline-level validation and reconciliation controls, improving market risk data accuracy and reducing discrepancies by ~20%.

PySpark Databricks Delta Lake SQL Data Validation Data Reconciliation

Associate Software Engineer

Mphasis

Mar 2021 - Oct 2022 (1 year 7 months)

Developed SQL scripts to clean, transform, and prepare datasets based on analysis requirements. Monitored scheduled data pipelines and validated outputs, identifying source-to-target inconsistencies to maintain data accuracy and operational stability.

SQL Data Pipelines Data Validation