Looking for a job

Jai Sipani

@jaisipani

Message

I build production LLM agents and distributed systems that improve cost, reliability, and scale.

India

Message

I've built production LLM and distributed systems at Interface AI, including a multi-provider agent runtime with tool calling, context management, prompt caching, retries, and per-run cost accounting.

I built an evaluation harness across 130+ tools and 300+ scenarios, and model routing that reduced inference spend by about 27% while maintaining equal evaluation pass rates. I also architected a multi-tenant sourcing engine for 10M+ profiles and led a zero-downtime migration from GCP to AWS ECS Fargate.

Previously, I delivered real-time speech, RAG, monitoring, IAM, MFA, and SSO systems, from 4K+ concurrent ASR streams to Shopify and WordPress 2FA integrations.

Experience

Work history, roles, and key accomplishments

Current

Lead Engineer

Current

Interface AI

Feb 2025 - Present (1 year 7 months)

Built a multi-provider agent runtime from scratch, including tool-calling loops, token-budgeted context management, and per-run cost accounting. Also built an eval harness scoring 130+ tools across 300+ scenarios and an intent-based model routing system that cut inference spend ~27%.

Anthropic OpenAI Vertex Tool calling Context Management Prompt Caching Model Evaluation Harnesses Model Routing RabbitMQ Kafka Python AWS GCP AWS ECS Fargate Vector Search Reranking

Senior Software Engineer

Interface AI

Mar 2023 - Jan 2025 (1 year 10 months)

Built a real-time ASR microservice handling 1M+ calls at 200-300 ms p95 and 4K+ concurrent streams. Fine-tuned transcription models with LoRA, cutting error rates 30%, and built an async ETL pipeline with Kafka and Celery.

Azure AI Speech ASR LoRAs Kafka Celery RAG Chunking Embeddings Reranking Python Redis S3

Software Engineer

Tekmonks Inc.

May 2022 - Mar 2023 (10 months)

Built an in-house agent-based system for real-time system and database health monitoring. Created a real-time React dashboard for access logs and a web-components framework that cut time-to-market by 30%.

React Web Components Monitoring Audit Logs DDoS Detection SQL Injection

Software Engineer

miniOrange Security Pvt. Ltd.

Sep 2020 - Apr 2022 (1 year 7 months)

Delivered IAM solutions including MFA and SSO for WordPress and Shopify. Built 2FA plugins integrating with 30+ forms and integrated SMS gateways for OTP delivery.