Open to opportunities

Jack Wang

@jackwang

Message

Production-focused ML engineer specializing in LLMs, RAG, MLOps, and low-latency systems.

United States

Message

What I'm looking for

I seek roles building production LLMs and MLOps at scale—focused on low-latency serving, robust deployment, and measurable product impact.

I am a production-focused machine learning engineer with 7+ years shipping scalable ML systems across fintech and AI platforms, specializing in adapting foundation models through fine-tuning and RAG architectures.

I build MLOps infrastructure, real-time data pipelines, and low-latency model serving that power customer-facing LLM applications, and I have delivered sub-200ms p95 inference latency and production RAG systems for enterprise document repositories.

I combine deep knowledge of ML frameworks (PyTorch, TensorFlow, JAX), vector stores, and cloud infrastructure with systems engineering practices—mentoring engineers, establishing automated deployment and monitoring, and driving reproducible model evaluation in production.

Experience

Work history, roles, and key accomplishments

Current

Senior Software Engineer

Current

Scale AI

Dec 2022 - Present (3 years 2 months)

Architected production LLM inference infrastructure delivering sub-200ms p95 latency and built end-to-end RAG systems and real-time embedding services powering customer-facing LLM applications.

VLLM Triton Inference Server Pinecone fastAPI ONNX Runtime Kubeflow Apache Beam Databricks Lang Chain

Machine Learning Engineer

Two Sigma

Jul 2019 - Dec 2022 (3 years 5 months)

Developed gradient-boosted and neural models and built real-time feature pipelines with millisecond latency for quantitative trading, plus distributed GPU training and automated hyperparameter optimization.

Apache Flink Kafka Polars DuckDB Kubernetes Horovod SHAP C++gRPC

Software Engineer Intern

Uber

May 2018 - Aug 2018 (3 months)

Built TensorFlow demand-prediction models and feature pipelines on Spark, implemented A/B testing frameworks, and optimized inference latency via ONNX conversion and batching.