Looking for a job

Ayush Upadhyay

@ayushupadhyay2

Data Engineer at Brembo building real-time vehicle IoT pipelines and production RAG and Text2SQL systems.

At Brembo, I design and maintain data pipelines processing vehicle IoT data, and built real-time streaming pipelines for sub-second data availability. I also developed a RAG solution that retrieves information from 300 vehicle manuals and deployed it globally.

I built a production Text2SQL system that translates natural-language questions into SQL, reducing ad-hoc data request turnaround from hours to seconds. I also created an LLM-based document extraction pipeline that reduced manual processing effort by 80%.

At Kaiser Permanente, I migrated legacy Oracle ETL pipelines to a Hadoop ecosystem and developed data pipelines for healthcare analytics. Earlier, at LinkedIn, I worked on marketing analytics, Hadoop workflows, and Tableau reporting.

Experience

Work history, roles, and key accomplishments

Current

Data Engineer

Current

Brembo

Jan 2023 - Present (3 years 8 months)

Designed and maintained scalable ETL data pipelines processing over 10M+ vehicle IoT data daily using AWS Lambda. Built real-time streaming pipelines and deployed production ML models, reducing manual processing effort by 80% and cutting processing time from 6+ hours to under 30 minutes.

AWS Lambda AWS IoT Core Kinesis Databricks MLFlow RAG Azure Data Factory PySpark Unity Catalog DevOps FinOps Data Governance Text2SQL NLP LLM

Data Engineer

Kaiser Permanente

Dec 2018 - Dec 2022 (4 years)

Led migration of legacy Oracle ETL pipelines to Hadoop-based ecosystem and engineered production-grade ETL pipelines for kp.org clickstream data. Built automated data quality monitoring and Python-based NLP pipelines, maintaining 99.9% pipeline uptime.

Hadoop Hive Azure Data Factory Python NLP Tableau Airflow Data Quality ETL Data Warehouse

Marketing Data Analyst

Feb 2018 - Dec 2018 (10 months)

Analyzed trends and monitored factors impacting subscription customer acquisition and retention. Engineered optimized Hive queries and developed R models to score customer responses, supporting A/B testing and GDPR compliance.

Hive HDFS Teradata Hadoop R Tableau Presto Azkaban A B Testing GDPR

IQS Analyst

Kia Motors

Aug 2017 - Dec 2017 (4 months)

Used R programming to automate data cleaning and transformation. Created executive dashboards providing product performance and key insights using Tableau.

R data transformation Tableau Data Visualization