Open to opportunities

Benjamin Scates

@benjaminscates

Message

I build reliable AI evaluation, data, and prompt workflows for language models.

United States

Message

What I'm looking for

I'm looking to apply AI evaluation, prompt engineering, data pipelines, and reproducible research practices to improve the reliability, robustness, and trustworthiness of AI systems.

At Keywords Studios, I collaborate with Amazon AI stakeholders to turn research goals into AI/ML problem statements, evaluation criteria, success metrics, and validation plans. I build reproducible data and evaluation pipelines for complex AI programs, including 10,514+ high-complexity examples.

I lead and mentor a distributed team of 15 researchers while training and onboarding 80+ contributors. My work spans Python experimentation, benchmark test sets, error analysis, prompt and harness engineering, and multimodal agentic AI evaluation focused on technical correctness, reasoning quality, and reliability.

Previously at Outlier AI, I created 1,756 NLP prompts and training examples for systems including ChatGPT, Google Gemini, and Meta AI, and refined 1,000+ prompts and labeled examples to improve model performance. I bring a Cognitive & Behavioral Neuroscience background and enjoy translating rigorous research into trustworthy model, data, and policy improvements.

Experience

Work history, roles, and key accomplishments

Current

Senior Research Associate - AI

Current

Keywords Studios

Jan 2025 - Present (1 year 8 months)

Collaborated with Amazon AI stakeholders to translate research goals into AI/ML problem statements and evaluation criteria. Managed a distributed team of 15 researchers and designed reproducible evaluation pipelines for over 10,000 high-complexity examples.