Looking for a job

Soundarya VK

@soundaryavk

Lead Data Engineer at Diageo India, cutting Databricks Serverless Compute operating costs by 30%.

At Diageo India, I lead data engineering delivery for enterprise analytics platforms and reusable data products. I design Azure data architecture and unified semantic layers that support self-service analytics and Databricks Genie.

I implemented Spark Declarative Pipelines integrated with DataHub and transitioned workflows to Databricks Serverless Compute, reducing operational costs by 30%. I also applied Liquid Clustering to high-cardinality Delta Lake tables to improve query execution and data accessibility.

At Mercedes-Benz Research & Development India, I led a squad of five data engineers delivering cloud-native Lakehouse solutions. I designed data models and Medallion-layer warehousing across Finance, Procurement, and Operations, and implemented Unity Catalog governance and DataHub lineage.

Earlier, I optimized Azure Databricks workloads at Mercedes-Benz Research & Development India, reducing processing duration by 40%, and built ETL/ELT pipelines at Fractal Analytics. At Tata Consultancy Services, I developed Python automation and data transformation scripts for enterprise cloud migration initiatives.

Experience

Work history, roles, and key accomplishments

Current

Lead Data Engineer

Current

Diageo India

Feb 2026 - Present (7 months)

Lead a cross-functional team of data engineers and external partners delivering enterprise-scale analytics platforms and trusted, reusable data products. Design end-to-end Azure data architecture and unified semantic layers, powering self-service analytics and conversational AI tools.

Azure Databricks PySpark SQL Delta Lake Serverless Computing Databricks Genie Spark Declarative Pipelines DataHub DataOps Unity Catalog Data Quality Data Lineage CI CD

Principal Consultant | Principal Data Engineer

Mercedes-Benz Research & Development India

Jun 2024 - Jan 2026 (1 year 7 months)

Led a global engineering squad of 5 data engineers to deliver cloud-native Lakehouse solutions using Azure Databricks, Delta Lake, and PySpark. Architected data models and unified semantic models across Finance, Procurement, and Operational domains, and enforced governance using Unity Catalog.

Azure Databricks Delta Lake PySpark Data Modeling Unity Catalog DataHub CI CD Serverless Computing Data Governance Data Lineage PII Classification Data Masking

Senior Consultant | Senior Data Engineer

Mercedes-Benz Research & Development India

Jun 2021 - Jun 2024 (3 years)

Architected Medallion Architecture data pipelines using Azure Databricks, PySpark, and Spark SQL. Optimized large-scale data processing workloads using Liquid Clustering, reducing processing duration by 40%, and implemented Unity Catalog governance controls.

Azure Databricks PySpark Spark SQL Medallion Architecture Liquid Clustering Unity Catalog Data Lineage Data Governance

Consultant | Azure Data Engineer

Fractal Analytics

Dec 2020 - Jun 2021 (6 months)

Built scalable ETL/ELT pipelines and cloud integration solutions utilizing Azure Databricks and Azure Data Factory. Contributed across all delivery stages of client cloud migration frameworks and automated data processing modules.

Azure Databricks Azure Data Factory ETL ELT Cloud Migration Automation