Shreya Gupta

I’m an Azure Data Engineer with nearly 4 years of experience building data pipelines, streaming solutions, and analytical data platforms on Microsoft Azure. I develop batch and streaming workflows on Azure Databricks with Spark and Kafka, processing 500K+ booking and post-booking events daily, and I build robust ETL/ELT pipelines using Delta Lake and Medallion Architecture.

I focus on reliability and performance—contributing to data modeling, schema evolution, partitioning, and data quality workflows with Python and Spark. I’ve delivered curated datasets and reporting layers powering 50+ KPIs, and improved batch processing performance by ~45% through Spark optimization and workflow enhancements, while automating Spark deployments with Databricks Workflows and CI/CD. I also apply GenerativeAI with RAG and embeddings to automate engineering workflows, including an MCP-based AI agent for code review and quality enforcement.