At Keywords Studios, supporting Amazon AI Operations, I evaluated Amazon Nova Act outputs against task instructions and scoring rubrics. I checked task completion, accuracy, reasoning quality, and instruction following, and flagged hallucinations and unsupported claims.
I also reviewed training-data annotations and validated structured outputs against JSON Schema. I developed reusable scoring guidelines for consistent model evaluation and annotation review.
In my freelance work with Handshake AI Fellowship | Project Dynamo, I authored a Terminal-Bench 2 task with an executable reference solution, a Pytest verifier, and a Docker environment. I also developed Signal Siege, a strategy-game benchmark with legality checks and anti-cheat safeguards.
I built an AI Ops Triage Platform with a six-stage LangGraph incident workflow, Gemini, and ChromaDB. My AI Knowledge Operations Platform combines RAG, semantic search, BM25 retrieval, and RAGAS metrics, with LangSmith tracing.
