Machine Learning Engineering Manager - LLM Serving (Remote - US)

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Machine Learning Engineering Manager - LLM Serving & Infrastructure in the United States.

This role offers the opportunity to lead and shape the development of high-scale, low-latency LLM serving infrastructure for advanced personalization and recommendation systems. You will manage a talented engineering team while collaborating closely with machine learning researchers, data scientists, and product teams to deliver robust, scalable, and cost-efficient ML systems. The position focuses on deploying and maintaining LLMs across production environments, optimizing performance and reliability, and driving adoption of innovative AI-powered features. This is a highly visible role where your technical leadership and strategic vision will directly impact millions of users, improving their experience through intelligent recommendations and interactions.

Accountabilities:

• Lead a high-performing engineering team to design, build, and deploy large-scale LLM serving infrastructure.
• Drive the implementation of a unified serving layer for multiple LLM models, supporting batch, offline evaluation, and real-time inference.
• Oversee the development and management of the Model Registry, including deployment, versioning, and monitoring of LLMs across production environments.
• Collaborate with internal product and personalization teams to integrate LLM-powered features seamlessly into recommendation systems.
• Establish standardized technical interfaces, protocols, and best practices for efficient model deployment, scaling, and operational monitoring.
• Optimize serving infrastructure for latency, cost, and reliability, leveraging techniques like quantization, pruning, and efficient batching.
• Develop observability and monitoring frameworks, including dashboards, alerting, and SLA tracking for inference traffic and system health.

Requirements

• 5+ years of software or machine learning engineering experience, including 2+ years managing engineering teams.
• Hands-on experience in ML engineering, building and scaling production-quality ML systems and datasets.
• Strong technical expertise in LLM infrastructure, recommendation or personalization systems, and high-volume, high-velocity ML platforms.
• Proven ability to lead complex, multi-partner projects with federated contribution models.
• Experience designing loosely coupled, scalable systems with clear separation of concerns and robust APIs.
• Familiarity with CI/CD pipelines, experiment tracking, and results visualization tools (e.g., MLFlow).
• Excellent leadership, communication, and collaboration skills, with the ability to influence peers and stakeholders.
• Pragmatic, results-oriented mindset, balancing speed and production rigor.

Benefits

• Competitive salary range: $176,166 - $251,666 plus equity.
• Comprehensive health insurance coverage.
• Six months paid parental leave.
• 401(k) retirement plan.
• Monthly meal allowance.
• 23 paid vacation days, 13 flexible holidays, and paid sick leave.
• Flexible remote/hybrid work options with occasional in-person meetings.
• Opportunity to work on cutting-edge ML infrastructure impacting millions of users globally.

Jobgether is a Talent Matching Platform that partners with companies worldwide to efficiently connect top talent with the right opportunities through AI-driven job matching.

When you apply, your profile goes through our AI-powered screening process designed to identify top talent efficiently and fairly.

🔍 Our AI evaluates your CV and LinkedIn profile thoroughly, analyzing your skills, experience, and achievements.
📊 It compares your profile to the job’s core requirements and past success factors to determine your match score.
🎯 Based on this analysis, we automatically shortlist the 3 candidates with the highest match to the role.
🧠 When necessary, our human team may perform an additional manual review to ensure no strong profile is missed.

The process is transparent, skills-based, and free of bias — focusing solely on your fit for the role. Once the shortlist is completed, we share it directly with the company that owns the job opening. The final decision and next steps (such as interviews or additional assessments) are then made by their internal hiring team.

Thank you for your interest!