Senior Software Engineer – Applied AI

Job Description:

• Contribute to all phases of product development, from ideation and prototyping to implementation and iteration
• Build and evolve both frontend product experiences and backend services that power them
• Fine-tuning existing models and optimize their performance for Mainstay use cases
• Design reliable, scalable backend services to support AI-powered features in production
• Leverage AI to modernize and automate legacy workflows in the real estate industry
• Evaluate and benchmark LLM performance across latency, cost, accuracy, and hallucination rates
• Tune prompts, chain-of-thought logic, and implement retrieval strategies such as RAG and vector stores
• Monitor performance, safety, and cost of LLM-driven features in production environments
• Define and build useful LLM-powered experiences in partnership with cross-functional teams
• Mentor and support the growth of other engineers on the team
• Collaborate closely with product managers, designers, and operators to identify and solve customer pain points

Requirements:

• 5+ years of experience building and delivering production-grade software, ideally in fast-paced, high-growth startup environment
• At least 3 years of hands-on experience developing and deploying production-ready AI features (LLM applications, fine-tuning, RAG systems, guardrails, evals, etc.)
• Strong full-stack engineering skills with a focus on backend scalability and system reliability
• Experience fine-tuning and deploying LLMs
• Deep understanding of LLM evaluation methods, including prompt testing, retrieval precision, hallucination detection, and cost/performance tradeoffs
• Experience building custom eval pipelines using techniques like model-as-a-judge or embedding similarity
• Hands-on work with agentic frameworks (e.g. LangGraph, AutoGen, CrewAI) or custom agent systems
• Clear, concise communicator - essential for our remote, highly collaborative team
• Proactive and ownership-driven - comfortable navigating ambiguity and shifting priorities
• Customer-focused mindset with a drive to build solutions that solve real problems
• Experience with AI/ML tools or APIs; bonus if you’ve worked with Python-based RAG, voice agents, or AI task automation.

Benefits:

• Trust is at the core of everything we do.
• We build trust with everyone -- from candidates to employees to partners.
• Providing reasonable accommodations throughout the recruitment process for candidates with disabilities, pregnancy, religious beliefs, or other reasons protected by applicable laws.