yash malhotra

I’m a Google Cloud Certified Professional Data Engineer with around 5 years of experience designing production-grade batch and streaming data pipelines on GCP. I specialize in advanced SQL and scalable distributed processing to deliver analytics-ready datasets under strict SLAs.

At Accenture, I designed scalable SparkSQL ETL pipelines processing 50M+ cross-border transactions daily, serving downstream reporting through BigQuery. I also built near real-time streaming microservices with Apache Kafka, enabling sub-minute data availability for monitoring and reducing case resolution time.

I lead migrations of on-prem datasets to GCP and focus on performance optimization, including dbt-based analytics engineering and data modeling. I’ve implemented Vertex AI agentic workflows for data observability—performing RCA, triggering alerts, and recommending remediations—which significantly reduced incident resolution time.

I enjoy partnering with product, analytics, and business stakeholders in agile environments, translating requirements into technical solutions. I mentor junior engineers through KT and documentation to raise delivery consistency and engineering quality.