Shubham Singh

I’m a Data Engineer with 3+ years of experience designing and optimizing scalable data pipelines and cloud-based data platforms on Google Cloud Platform (GCP). I focus on ETL/ELT development, data modeling, and BigQuery-based data warehousing to deliver reliable, high-performance analytics.

At General Mills, I built an AI-powered metadata intelligence platform that integrates dbt lineage extraction, LLM-generated metadata, and BigQuery—enabling searchable analytics documentation and conversational dataset discovery with a 40% improvement in discoverability. I also created a Vertex AI and Glean-powered conversational analytics agent that reduced query time by 30% and enabled 50+ analysts to self-serve analytics.

I modernized legacy data workflows by re-architecting 50+ ETL pipelines into modular Airflow–BigQuery workflows, optimizing data throughput by 40% and reducing pipeline failures by 25%. I led dbt implementation across 160+ data models, automated data quality testing with dbt-expectations, and increased pipeline reliability by 35%.

I’m equally driven by governance and cost efficiency: I’ve strengthened dataset documentation and compliance through data governance work, improved data quality via lineage and metadata management, and reduced GCP costs by 15%. I also migrated SAP S/4HANA datasets, improving reconciliation accuracy by 20% and reducing migration time by 35%.