Sarthak Karnatak
@sarthakkarnatak
AI Trainer and RLHF Evaluator improving LLM quality, safety, and reasoning through structured evaluation and prompt optimization.
IndiaWhat I'm looking for
I’m an AI Trainer and RLHF Evaluator focused on improving large language model outputs through rigorous, rubric-based assessment. I combine hands-on prompt engineering with safety alignment to spot reasoning gaps, hallucinations, and policy risks early.
At Outlier AI, I design prompts and golden responses for supervised training, run side-by-side model comparisons, and lead multi-turn evaluation to verify instruction retention, contextual consistency, and coherent dialogue progression. I also contribute to scalable training workflows using structured JSON task formats and standardized annotation guidelines.
Beyond LLM evaluation, I bring 8+ years of customer experience leadership—managing teams, improving CSAT, reducing response times, and building scalable processes. My approach stays practical, quality-driven, and collaborative, including cross-functional coordination in fully remote environments.
Experience
Work history, roles, and key accomplishments
Evaluated and improved large language model outputs using RLHF and rubric-based scoring across business, operations, and creative domains. Built golden responses and prompts, performed side-by-side comparisons to identify hallucinations and safety risks, and achieved Oracle-level contributor ratings for sustained accuracy and policy compliance.
Led a 30+ agent customer support team, improving CSAT by 15% and increasing operational efficiency. Implemented AI chatbot solutions that reduced ticket volume by 30% and first response time by 25%, while cutting average handle time by 20% and boosting agent productivity by 35%.
Improved CSAT by 15% and reduced response time by 25% through workflow changes and macro optimization. Managed 5,000+ monthly customer queries across Twitter, Facebook, and other social platforms, using sentiment analysis to shape engagement strategies.
L2 Customer Experience Associate
Flipkart
Sep 2016 - May 2018 (1 year 8 months)
Resolved 95%+ of customer escalations with high satisfaction outcomes and minimal repeat contacts. Maintained CSAT above 90% across review cycles while ensuring 100% adherence to process knowledge and compliance standards.
Education
Degrees, certifications, and relevant coursework
College of Engineering Roorkee
Bachelor of Engineering, Computer Science
2014 -
Bachelor of Engineering program with a minor in Computer Science at College of Engineering Roorkee starting in 2014.
Availability
Location
IndiaAuthorized to work in
Social media
Job categories
Skills
Interested in hiring Sarthak?
You can contact Sarthak and 90k+ other talented remote workers on Himalayas.
Message SarthakFind your dream job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
