Open to opportunities

Jack Wang

@jackwang2

Message

Senior software engineer building scalable AI/ML and LLM-powered production systems.

United States

Message

What I'm looking for

I’m looking to build reliable, low-latency LLM/AI systems end to end—focused on evaluation rigor, scalable cloud/data engineering, and measurable impact that evolves with emerging AI technologies.

I’m a Senior Software Engineer with 10+ years of experience building scalable, production-grade systems with a strong focus on AI/ML and LLM-powered applications.

At Scale AI, I built an LLM-powered data labeling platform using Python and OpenAI APIs, replacing manual workflows with prompt engineering frameworks, structured JSON outputs, and validation layers for deterministic labeling. I developed RAG with embedding pipelines and Pinecone to solve hallucinations using semantic search, relevance ranking, and context window optimization.

I also designed multi-stage AI inference pipelines with FastAPI, adding validated and enriched model outputs across services to prevent failure propagation via confidence scoring, retry mechanisms, and rule-based guardrails. To improve throughput, I reduced latency in LLM API workflows with asynchronous processing using Celery and Redis, plus batching and embedding caching.

Beyond that, I built a real-time ingestion pipeline with Kafka, containerized services with Docker, and scaled inference workloads with Kubernetes under variable traffic. I bring a production mindset to evaluation—prompt versioning, A/B testing, and model evaluation pipelines—and I mentor engineers on LLM system design and production ML best practices.

Experience

Work history, roles, and key accomplishments

Current

Senior Software Engineer

Current

Scale AI

Apr 2022 - Present (4 years 2 months)

Built an LLM-powered data labeling platform using Python and OpenAI APIs, introducing prompt engineering, structured JSON outputs, and validation layers to improve labeling consistency. Developed RAG and multi-stage inference systems with FastAPI and Pinecone, reduced LLM workflow latency via asynchronous processing and caching, and implemented Kafka-based real-time ingestion with fault tolerance

OpenAI API LLMs (OpenAI Prompt Engineering Retrieval Augmented Generation (RAG)fastAPI Pinecone Kafka Celery Redis Kubernetes

Machine Learning Engineer

Two Sigma

Feb 2019 - Mar 2022 (3 years 1 month)

Built a large-scale time-series prediction system in Python to address noisy and non-stationary data using feature engineering and normalization. Developed ensemble models (XGBoost, LightGBM), distributed training/backtesting pipelines with Apache Spark, automated evaluation/backtesting frameworks, and data validation and PostgreSQL storage to support reliable production deployment.

Time Series Models Feature Engineering XGBoost LightGBM Cross Validation Hyperparameter Tuning Apache Spark Data Validation PostgreSQL Model Evaluation

Software Engineer Intern

Uber

May 2018 - Aug 2018 (3 months)

Built backend Python tools to process large-scale operational datasets for internal analytics, reducing reliance on manual extraction workflows. Developed Flask REST APIs for processed data and improved reliability with validation checks, error handling, and automated unit tests using PyTest.

Python Flask REST APIs Backend Data Validation Error Handling Pytest ETL

Software Engineer Intern

Kloudless

Sep 2017 - Dec 2017 (3 months)

Built frontend dashboards with React and Redux to visualize third-party API integrations. Designed reusable UI components, implemented MongoDB schemas for integration metadata, and added integration testing with Mocha and Chai to validate API workflows.

React Redux JavaScript HTML CSS MongoDB API Integration Mocha Chai

Software Engineer Intern

Oracle

May 2017 - Aug 2017 (3 months)

Developed JavaScript UI features for internal engineering tools and built Python backend scripts to process structured enterprise data. Implemented REST endpoints for internal system integration and improved backend reliability with unit tests using JUnit.

JavaScript Python Unit Testing Junit System Integration