Skip to main content
HimalayasHimalayas logo
Prashant JhaPJ
Looking for a job

Prashant Jha

@prashantjha

LLM Engineer building production RAG systems and evaluation pipelines to ship reliable AI.

India
Message

What I'm looking for

I’m looking for roles where I can build production RAG and LLM evaluation systems—turning messy user data into reliable, testable AI. I want to ship end-to-end pipelines, collaborate with product teams, and keep improving quality through rigorous metrics and iteration.

I’m an LLM Engineer and AI Engineer with 5 years of experience, transitioning from data science roots into full-stack LLM engineering. I focus on production-ready LLM pipelines—especially Retrieval Augmented Generation (RAG) and generative AI systems that behave consistently in the real world.

In my current work, I built an end-to-end AI job aggregation pipeline collecting 150+ daily remote AI/ML job postings with multi-region and multi-query coverage. I implemented a two-stage semantic filtering approach using embeddings and structured evaluation (OpenAI GPT models), reducing raw listings to ~30 highly relevant results.

I also designed evaluation-focused systems—like golden dataset generation for Copilot features and a document-grounding pipeline that creates reduced summaries and query-specific evidence views from enterprise DOCX/PPTX/PDF content. I’ve delivered multimodal extraction for visually rich documents, improved throughput with multithreaded batch execution and caching, and earned recognition such as the Winner of Smart India Hackathon 2020, a Stroke of Excellence Award 2023, and a published research paper on automated traffic management.

Experience

Work history, roles, and key accomplishments

Microsoft logoMI
Current

AI Job Discovery Engineer

Mar 2026 - Present (3 months)

Built an end-to-end job aggregation pipeline (Microsoft via Volga Partners) to collect 150+ daily AI/ML job postings across RemoteOK, We Work Remotely, and Google Jobs. Implemented two-stage semantic filtering with text-embedding-3-small to reduce 150+ raw listings to ~30 highly relevant results.

Microsoft logoMI

LLM Evaluation Data Scientist

Jun 2021 - Apr 2024 (2 years 10 months)

Designed and built golden datasets to evaluate Microsoft Copilot features across multimodal scenarios (image upload, image generation, file handling, and multi-turn queries). Created LLM-driven evaluation pipelines with human-in-the-loop validation to ensure quality, correctness, and scenario coverage.

Education

Degrees, certifications, and relevant coursework

JC

Jabalpur Engineering College

Bachelor of Engineering, Computer Science

2016 - 2020

Earned a Bachelor of Engineering in Computer Science at Jabalpur Engineering College from 2016 to 2020.

Find your dream job

Sign up now and join over 100,000 remote workers who receive personalized job alerts, curated job matches, and more for free!

Sign up
Himalayas profile for an example user named Frankie Sullivan