At Deep Inference, I built and evaluated benchmarking workflows for large language model inference, measuring time-to-first-token, inter-token latency, throughput, and GPU memory use.
I investigated inference bottlenecks across model execution, batching, memory management, and GPU utilization. I also evaluated vLLM and SGLang serving configurations, comparing trade-offs in concurrent requests, KV cache use, throughput, and latency.
At Infinity, I developed and tested Python and C++ components for machine-learning workloads, with an emphasis on execution efficiency, correctness, and reproducibility. I investigated bottlenecks with profiling tools and evaluated GPU kernels for numerical correctness, latency, memory access patterns, and scalability.
In my projects, I built a reproducible LLM inference benchmarking framework and implemented a custom Triton kernel, verifying its numerical correctness against PyTorch. I profiled baseline and optimized versions to study memory access, block sizes, occupancy, and launch overhead.
