LLM Data Researcher

Role Overview

We are looking for a strategic, data-savvy individual to lead efforts in evaluating data quality and its return on investment—particularly as it impacts downstream model outcomes in terms of improving their performance on challenging benchmarks and workflows in the real-world. This is a high-leverage, cross-functional role focused on delivery excellence and maximizing client impact through smarter data practices.

Rather than training models directly, this role is about answering foundational questions such as:

• What types of data move the needle most on model performance?
• How do we build benchmarks that matter and reflect real-world use cases?
• Where do we invest next to scale high-quality data pipelines?

Key Responsibilities

• Define and implement strategies to assess the ROI of data across training and fine-tuning pipelines.
• Build and maintain benchmarks that measure performance across key client and internal objectives.
• Collaborate with data operations, research, and delivery teams to align on quality standards and data priorities.
• Develop systems and tooling for continuous data evaluation—measuring what matters, where it matters.
• Drive human-in-the-loop quality processes including pre-delivery validation and annotation feedback loops.
• Identify data gaps and lead targeted acquisitions or refinements, guided by performance metrics.
• Define and/or leverage comprehensive task taxonomy frameworks to structure data annotation efforts and improve training signal quality.
• Translate research insights and data evaluations into client-facing value through better delivery and prioritization.

Required Qualifications

• Strong grasp of LLMs and data-model dynamics (but this is not a model training role).
• Knowledge of the latest trends in Generative AI and data that is useful for improving foundation models
• Proven track record in benchmark development, model evaluation, or data-centric infrastructure.
• Strategic thinker with a bias toward impact: can connect data quality work directly to client value.
• Experience designing and interpreting metrics that inform delivery performance.
• Familiarity with annotation workflows, validation processes, and scalable QA systems.
• Solid ML or data science foundation—able to reason about training impact from a data point-of-view.

Preferred

• Experience with feedback-driven annotation loops and pre-delivery QA.
• Hands-on experience with taxonomy frameworks and structured data labeling.
• Background in data-centric AI or related research, Linguistics as applied to data annotation efforts, or related fields.