# Prabhat Yadav

Prabhat Yadav is a remote worker on Himalayas, Senior Data Engineer at RTA, based in United Arab Emirates.

Source: https://himalayas.app/@prabhatyadav1

```text
Senior Data Engineer at RTA building real-time transport monitoring and AI-driven operational insights.
```

| Detail | Value |
| --- | --- |
| Availability | Looking for a job |
| Location | United Arab Emirates |
| Authorized to work in | United Arab Emirates |
| Salary expectations | 65k+ USD |
| Job categories | Data Engineer |
| Skills | Data Engineering, Data Platform Engineering, Data Warehouse, Dimensional Data Models, ETL, ELT, Real Time Data Ingestion, Change Data Capture, Streaming Data Architectures, Distributed Systems, Data Quality, Data Governance, Performance Optimization, Cloud Migration, Lakehouse Architecture |
| LinkedIn | https://linkedin.com/in/prabhat-yadav115 |

## About

```text
At RTA, I architect real-time pipelines and backend data services for traffic, fleet, geospatial, and external data, supporting operational monitoring and analytics. I also designed a LangChain-driven layer for operational summaries, compliance insights, and incident analysis.

At phData, I built PySpark and Databricks ELT pipelines for United Airlines’ Snowflake migration and a Streamlit self-service analytics app with ChatGPT and dbt. Earlier, I worked on streaming anomaly detection and CDC data platforms at Nagarro Pvt. Ltd, and data engineering projects at Tata Consultancy Services.
```

## Experience

### Senior Data Engineer (current)

RTA. Jul 2024 - Present

```text
Owned technical design reviews and engineering standards across data-platform initiatives. Architected real-time streaming pipelines and backend data services for traffic, fleet, geospatial and external data, enabling near real-time operational monitoring and analytics.
```

Skills: Spark, Python, Kafka, Cloudera, Scala, MongoDB, REST-APIs, OpenShift, Power-BI, Lang-Chain, GIS, Neo4j

### Senior Data Engineer

phData. Apr 2023 - Jul 2024

```text
Built scalable ELT pipelines using PySpark on Databricks (Delta Lake) integrated with Snowflake to process multi-year historical datasets for inventory forecasting. Designed a Streamlit self-service analytics app with ChatGPT and dbt, enabling business users to query and visualize data without SQL expertise.
```

Skills: Snowflake, AWS, Databricks, CloudFormation, Streamlit, PySpark, Python, Great-Expectations, Informatica, Salesforce, DBT, Power-BI, Git, GitHub

### Data Engineer

Nagarro Pvt. Ltd. Aug 2021 - Apr 2023

```text
Designed a real-time anomaly detection microservice (Spark Streaming) to analyze manufacturing logs and trigger proactive alerts. Implemented CDC architecture using Debezium and Hudi enabling incremental processing, ACID upserts and low-latency analytics.
```

Skills: AWS, Glue, Redshift, Athena, S3, PySpark, Kafka, Airflow, Python, Azure-DevOps

### Data Engineer

Tata Consultancy Services. Oct 2019 - Jul 2021

```text
Engineered secure data pipelines on Cloudera Hadoop ecosystem (Hive, Spark), managing sensitive PII data in accordance with governance standards. Developed a file compaction pipeline with custom checkpointing, reducing processing time by ~60%.
```

Skills: Spark, Scala, Cloudera, Autosys, YAML, Hive, Sbt, TeamCity, XL-Release

### Associate Data Engineer

Tata Consultancy Services. Nov 2018 - Sep 2019

```text
Implemented real-time ingestion pipelines on Cloudera using Spark Streaming and Kafka, integrating Schema Registry to stabilize schemas and reduce manual interventions.
```

Skills: Hive, Kafka, Cloudera, Spark-Streaming, Scala

## Education

### Amazon Web Services (AWS)

AWS Certified Solutions Architect - Associate, Cloud Architecture

```text
AWS Certified Solutions Architect - Associate certification is in progress.
```

### Snowflake

SnowPro Core Certification, Data Engineering

```text
Earned the SnowPro Core Certification in June 2023.
```

## Tech stack

- Amazon Redshift
- Snowflake
- Apache Spark
- AWS Glue
- Amazon S3
- GitHub
- Kubernetes
- Salesforce
- PySpark
- Debezium
- dbt
- MySQL
- PostgreSQL
- MongoDB
- Hadoop
- Databricks
- Neo4j
- EasyOCR
- OpenCV
- Azure DevOps
- Python
- Scala
- Streamlit
- Kafka
- Git
- Docker
- Amazon EMR
- Amazon Athena
- SQL
- LangChain
- Delta Lake
- Great Expectations
- Apache Hudi
- LM Studio
- Power BI
- AWS

To contact Prabhat, message them from this profile with a Himalayas employer account: [Hire on Himalayas](https://himalayas.app/hire).

## People also viewed

- [Faraz User](https://himalayas.app/@farazuser): Data Engineer, Roads and Transport Authority
- [Rajkumar Rajagopal](https://himalayas.app/@rajkumarrajagopal): Senior Data Engineer, DataBees AB
- [Syed Jawad Gilani](https://himalayas.app/@syedjawadgilani): Data Solution Architect / Unit Head Data Mgt. & Analytics, Department of Culture & Tourism
- [Ramya Mahi](https://himalayas.app/@ramyamahi): Data Engineer/ Analyst and Manager, Capgemini
- [jayant sonawane](https://himalayas.app/@jayantsonawane): Senior Manager, Data Engg & Analytics, TPConnects
- [Utkarsh Mukherjee](https://himalayas.app/@utkarshmukherjee): Data and AI Solutions Engineer, DP World
- [Shaik Waheeda User](https://himalayas.app/@shaikwaheedauser): Senior Data Engineer, EmiratesNBD
- [karthik buddhavarapu](https://himalayas.app/@karthikbuddhavarapu): Senior Data Engineer II, MBC Group
- [Addri Majee](https://himalayas.app/@addrimajee): Data Architecture Engineer, Accenture
- [Vinod User](https://himalayas.app/@vinoduser5): Solution Architect / Generative AI Engineer / Lead Data Engineer, Wipro
