Senior ML Engineer (Token Factory)

The role

Token Factory is a part of Nebius Cloud, one of the world’s largest GPU clouds, running tens of thousands of GPUs. We are building an inference & fine-tuning platform that makes every kind of foundation model — text, vision, audio, and emerging multimodal architectures — fast, reliable, and effortless to train & deploy at massive scale.

• Advanced Fine-Tuning: Enhancing fine-tuning methodologies - both LoRA-based and full-parameter - for cutting-edge LLMs (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-4.7), focusing on both model quality and training efficiency.

• Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. This involves building model training and evaluation pipelines in JAX for speculative decoding, experimenting with architectures (dense/MoE, auto-regressive/parallel), and deriving scaling laws to guide resource allocation.
• Low Precision Training & Inference: Investigating low-precision (FP8, NVFP4/MXFP4) methodologies for supervised fine-tuning and reinforcement learning - spanning both inference and training - optimized for modern hardware

We expect you to have:

• A profound understanding of theoretical foundations of machine learning and reinforcement learning.

• Deep expertise in modern deep learning for language processing and generation

• Experience with training large models on multiple computational nodes

• Reasonable understanding of performance aspects of large neural network training (sharding strategies, custom kernels, hardware features etc.)

• Strong software engineering skills (we mostly use Python)

• Deep experience with modern deep learning frameworks (we use JAX)

• Proficiency in contemporary software engineering approaches, including CI/CD, version control and unit testing

• Strong communication and leadership abilities

Nice to have:

• Previous experience working with language models or other similar NLP technologies.

• Familiarity with important ideas in LLM space, such as MHA, RoPE, ZeRO/FSDP, Flash Attention, quantization

• A track record of building and delivering products (not necessarily ML-related) in a dynamic startup-like environment.

• Strong engineering skills, including experience in developing large distributed systems or high-load web services.

• Open-source projects that showcase your engineering prowess

• Excellent command of the English language, alongside superior writing, articulation, and communication skills.