Prabhat Yadav

Owned technical design reviews and engineering standards across data-platform initiatives. Architected real-time streaming pipelines and backend data services for traffic, fleet, geospatial and external data, enabling near real-time operational monitoring and analytics.

Built scalable ELT pipelines using PySpark on Databricks (Delta Lake) integrated with Snowflake to process multi-year historical datasets for inventory forecasting. Designed a Streamlit self-service analytics app with ChatGPT and dbt, enabling business users to query and visualize data without SQL expertise.

Designed a real-time anomaly detection microservice (Spark Streaming) to analyze manufacturing logs and trigger proactive alerts. Implemented CDC architecture using Debezium and Hudi enabling incremental processing, ACID upserts and low-latency analytics.

Engineered secure data pipelines on Cloudera Hadoop ecosystem (Hive, Spark), managing sensitive PII data in accordance with governance standards. Developed a file compaction pipeline with custom checkpointing, reducing processing time by ~60%.

Implemented real-time ingestion pipelines on Cloudera using Spark Streaming and Kafka, integrating Schema Registry to stabilize schemas and reduce manual interventions.

AWS Certified Solutions Architect - Associate certification is in progress.

Earned the SnowPro Core Certification in June 2023.