Open to opportunities

Matthew George

@matthewgeorge

Senior machine learning engineer building model-agnostic LLM orchestration and evaluation pipelines for reliable, production-ready AI.

United States

Message

What I'm looking for

I’m looking for a team where I can own LLM systems end-to-end—evaluation, observability, and production deployment—using model-agnostic infrastructure that’s measurable, reliable, and iterates fast.

I’m a Senior Machine Learning Engineer specializing in LLM orchestration, tool-calling systems, and multi-model evaluation. I build model-agnostic infrastructure that makes AI reliable, measurable, and production-ready, with a focus on structured outputs, observability, and repeatable deployment workflows.

Most recently, I built and owned a multi-agent LLM system with Pydantic AI—planner, executor, and application agents—for Sidekick ticket triage and resolution. I implemented sentiment analysis and escalation detection feeding structured results into PostgreSQL, and I owned the LLM evaluation pipeline using generative AI judges (Pydantic AI + OpenAI GPT-4.1) to measure accuracy, completeness, hallucination rate, and action validation. I also set up distributed tracing with Honeycomb and OpenTelemetry, instrumented runtime metrics with Prometheus, and integrated eval/deployment into CI/CD with GitHub Actions while deploying on AWS Lambda and managing infra with Pulumi.

Experience

Work history, roles, and key accomplishments

Current

Senior Machine Learning Engineer

Current

Fixify

Mar 2025 - Present (1 year 1 month)

Built and maintained a multi-agent LLM orchestration system with planner/executor/application agents powering Sidekick ticket triage and resolution workflows. Implemented agent evals (classification accuracy, completeness, hallucination rate, action validation), event-driven execution with AWS services, and observability via Honeycomb/OpenTelemetry and Prometheus; deployed agent services with CI/C

Orchestration PydanticAI OpenAI API Anthropic: Claude AWS Lambda PostgreSQL Clickhouse OpenTelemetry Prometheus

Machine Learning Engineer

Notion

Nov 2023 - Feb 2025 (1 year 3 months)

Owned post-transcription pipelines for action item extraction and meeting summarization using Claude 3.5 Sonnet and GPT-4o with structured output parsing and citations. Built LLM-as-judge evaluation in Braintrust, integrated Fireworks AI Whisper with Baseten Labs for batching/latency tuning and 16-locale multilingual support, and delivered an Exa-powered agentic research loop with workspace RAG (t

LLM As Judge Braintrust GPT 4o Whisper Baseten Exa Vector Search

Machine Learning Engineer

Zendesk

Sep 2020 - Oct 2023 (3 years 1 month)

Trained and evaluated intent classification, sentiment analysis, and language detection models for Intelligent Triage, enabling automated ticket routing across hundreds of categories. Improved the Answer Bot deep learning recommendation pipeline on millions of support conversations, built AWS-based training pipelines with Kafka/Spark/Airflow, deployed services with Docker/Kubernetes/SageMaker/MLfl

Sentiment Analysis Recommendation Systems Kafka Apache Spark Airflow AWS Sagemaker Kubernetes MLFlow

Data Scientist

Kustomer

Sep 2018 - Aug 2020 (1 year 11 months)

Developed NLP models for conversation classification and sentiment scoring using agent-labeled data stored in MongoDB. Built Python-based data pipelines and feature engineering for model training, supported intelligent routing based on agent skill/availability, helped integrate Reply.ai chatbot NLP/deflection models, and created dashboards to track model performance and agent productivity.

NLP Sentiment Scoring Python MongoDB Data Pipelines Routing Dashboards Reply.Ai Integration

Data Science Intern

Microsoft

Jul 2017 - Sep 2017 (2 months)

Engineered full-stack cloud applications using the Microsoft Stack to demonstrate scalable cloud architectures on Azure. Collaborated with a cross-functional team to build high-performance, strict-typing frontends following modern software engineering best practices for enterprise tools.

Azure Cloud Architecture Full Stack Frontend Engineering TypeScript Scalable Systems