ANUSHREE MISHRA

I’m a data annotation and AI evaluation specialist with 5+ years of experience supporting Machine Learning model development. With a PhD in Linguistics (syntax, semantics) and direct industry experience at OpenAI, I focus on designing evaluation rubrics and quality frameworks that make AI outputs more reliable and linguistically grounded.

As an AI Linguistics Consultant (Contract) for OpenAI (via Mercor), I supported ML model training by annotating and evaluating 1,000+ prompts to identify systematic failure modes and recurring error patterns. I designed and iterated structured evaluation rubrics and taxonomies across accuracy, factual consistency, safety, helpfulness, and linguistic naturalness—especially for Hindi-Urdu and Bangla—while operationalising pragmatic and register judgments to improve inter-reviewer consistency by ~30%.

I also led and calibrated a distributed team of 3 raters, building onboarding guidelines and resolving annotation disagreements on edge cases and ambiguous outputs. I produced clear, stakeholder-ready documentation translating model behaviour patterns, evaluation findings, and risk signals into actionable reports for cross-functional product and safety teams.

Beyond industry work, I’m a Research Fellow (fully-funded fellowship) at University of Konstanz, where I designed native speaker elicitation studies and built annotated datasets for focus-prosody interactions and clause-level structural variation. Earlier, as a Doctoral Researcher and Guest Faculty at EFLU, I conducted 5 years of doctoral research (ellipsis, wh-movement, scope, information structure) and delivered linguistics courses to 200+ students.