Ankur Patel

Senior Data Engineer, 5.8 years, building production data platforms on Azure Databricks — lakehouse, real-time streaming, CI/CD, and GenAI that ships to real users.

Currently at DCM Shriram Limited, a listed Indian chemicals manufacturer, where I own the end-to-end Data & AI platform and Azure DevOps CI/CD. Two plant-optimization platforms I built end-to-end have delivered $1.2M (₹10.8 Cr) in measured savings, and re-architecting the data platform cut Azure cloud spend 65% in under 6 months — root-caused a mis-set blob access tier driving 99.5% of storage operation cost, capped Kafka to 1 eCKU, retuned pipeline triggers, and right-sized clusters and App Service plans. I set technical direction for a cross-functional team of 4 (2 Data Engineers, 2 Data Scientists).

Before that, 5 years as a Data & Analytics Engineer on multi-year engagements with international clients across e-commerce, retail, and healthcare — Kafka + Databricks streaming, Snowflake + dbt warehouses, CDP implementations, and HIPAA-compliant healthcare pipelines.

What I build:

- Config-driven streaming frameworks on Databricks — one Lakeflow Spark Declarative Pipeline serving every real-time use case through pure configuration; 1/3 the infrastructure cost of the setup it replaced, zero repeated failures, 1-second event latency

- Production GenAI / RAG on Azure AI Search and Azure OpenAI — hybrid BM25 + vector retrieval, semantic reranking, metadata filtering, and source citations across 70 GB of documentation

- Lakehouse platforms on Delta Lake with Unity Catalog and a Databricks Asset Bundle monorepo across DEV / TEST / PROD

- Real-time Kafka pipelines and medallion architectures processing millions of daily events

- Full-stack data applications — FastAPI + PostgreSQL fronting Pyomo optimization models (MILP / nonlinear)

Stack

Daily: Python · SQL · PySpark · Azure Databricks · Delta Lake · Delta Live Tables · Unity Catalog · Databricks Asset Bundles · Kafka · FastAPI · Docker · Azure DevOps

Regular: dbt · Snowflake · Azure Data Factory · Synapse · Terraform · PostgreSQL · Azure Functions · React

GenAI: Azure AI Search · Azure OpenAI · LangChain · FAISS · Pyomo (MILP / nonlinear)