Open to opportunities

Stephen Koo

@stephenkoo

Message

Senior software engineer specializing in scalable AI infrastructure and production LLM integrations.

United States

Message

What I'm looking for

I seek roles building production-grade AI/ML infrastructure and scalable backend systems, with cross-functional impact, mentorship opportunities, and strong reliability focus.

I am a Senior Software Engineer with over eight years of experience designing and operating scalable backend and AI infrastructure systems for large-scale production environments.

At Google, I led design and implementation of AI inference orchestration, routing, and caching layers (Golang, Go on GKE), delivering measurable latency reductions and multi-region high availability for Gmail and Google Chat.

I build end-to-end solutions spanning services, CI/CD, observability, and front-end experiences using React/TypeScript and Next.js, and I productionize LLM-backed features with Kubernetes, Vertex AI/Gemini, and retrieval-augmented generation.

I partner cross-functionally with ML, product, and privacy teams, mentor engineers, and focus on reliability, performance, and enterprise-grade AI capabilities that drive product impact.

Experience

Work history, roles, and key accomplishments

Current

Senior Software Engineer

Current

Google

Sep 2017 - Present (8 years 9 months)

Led the design and implementation of the AI Inference Orchestration Service (Golang), a critical component responsible for routing requests between Workspace surfaces and ML models. Optimized the Golang request batching layer and implemented in-memory caching strategies, contributing to a team-wide 30% reduction in P95 inference latency

Golang Kubernetes Vertex AI RAG Python React Typescript Cloud Build

Education

Degrees, certifications, and relevant coursework

Stanford University

Master of Science, Computer Science (Artificial Intelligence)

2015 - 2017

Completed a Master of Science in Computer Science with a focus on Artificial Intelligence, covering advanced AI topics and research-driven coursework.

Stanford University

Bachelor of Science, Computer Science (Artificial Intelligence)

2011 - 2015

Completed a Bachelor of Science in Computer Science with a focus on Artificial Intelligence, including foundational coursework in algorithms, systems, and AI.