TrueFoundry provides a Kubernetes-based, cloud-agnostic PaaS that automates ML workflows—model deployment, autoscaling, monitoring, RAG/agent workflows, GPU/memory optimization, and centralized access controls for multi-cloud and on-premises environments.
<p><strong>About TrueFoundry<br></strong><br>Every production AI system, whether it's powering customer support, writing code, analyzing financial data, or diagnosing medical conditions, needs the same foundational infrastructure.A way to route between models. A way to manage tools and integrate them securely. A way to orchestrate agents and enforce governance. A unified compute layer to run it all.</p> <p><strong>That infrastructure layer is being built right now.</strong></p> <p>We are looking for a Staff ML Platform Engineer to join the team.</p> <h2><strong>The Problem We're Solving</strong></h2> <p>Companies are moving beyond simple chatbots to production agentic systems. These systems route between OpenAI, Anthropic, Google, and self-hosted models. They integrate dozens of tools via protocols like MCP. They orchestrate multi-agent workflows where agents coordinate with other agents.</p> <p>The infrastructure to support this doesn't exist yet. You can't just duct-tape together a few API calls and call it production-ready.</p> <p>You need a control plane that handles:</p> <ul> <li>Intelligent routing with observability, cost policies, and fallback logic</li> <li>Centralized tool and MCP server management with security and lifecycle controls</li> <li>Agent orchestration with governance and guardrails</li> <li>A unified compute layer to run self-hosted models, custom tools, and agents</li> </ul> <p><strong>AI Gateway</strong> is the control plane, five composable components (Prompts, LLM Gateway, MCP Gateway, Guardrails, Agent Gateway) that handle routing, orchestration, and governance.</p> <p>We're Series A, backed by Intel Capital and Sequoia. Companies like CVS, Mastercard, Siemens, Paytm, Synopsys, and Zscaler run production AI workloads on our platform.</p> <p>We're looking for <strong>ML Engineers</strong> who are passionate about scaling deep learning workloads, optimizing multi-GPU training, and shipping production-grade solutions. If you live and breathe PyTorch, multi-node training, and love solving gnarly infra challenges this is your place.</p> <h3><strong>What You’ll Work On</strong></h3> <ul> <li>Write clean, modular, and scalable Python code, with a strong emphasis on reliability and performance.</li> <li>Build a platform for training and fine-tuning large-scale ML models across multi-GPU, multi-node clusters with PyTorch, Kubeflow, and other orchestration tools.</li> <li>Own the infrastructure and code that enable high-throughput, low-latency inference pipelines for state-of-the-art models.</li> <li>Build a platform for developing, deploying, and evaluating agentic applications for our end customers. </li> <li>Help shape internal standards and best practices across the engineering team for high-scale ML workloads.</li> </ul> <h3><strong>What We’re Looking For</strong></h3> <ul> <li>5+ years of hands-on experience building and deploying ML systems at scale.</li> <li>5+ years of writing production-quality, high-performance code.</li> <li>Deep experience with multi-GPU/multi-node training, ideally with PyTorch as your primary framework.</li> <li>Experience working with Torch, high-level ML frameworks, and inference engines (vLLM or TensorRT).</li> <li>Experience with Kubernetes is highly preferred; exposure to Kubernetes-native tools is a huge plus.</li> <li>A pragmatic mindset: you know when to optimize and when to ship.</li> <li>Bonus: Familiarity with open-source LLM training/fine-tuning.</li> </ul> <p><strong>Traits we are looking for:</strong> Ownership, ability to execute, hustle and think out of the box, data-driven decision making, be comfortable with more unknowns than knowns.</p> <h3><strong>Perks of Working at TrueFoundry</strong></h3> <ul> <li>Join a fast-growing Series A, Bay Area-based startup building cutting-edge AI infrastructure.</li> <li>Comprehensive health insurance for you and your family.</li> <li>Flexible hybrid work, 3 days a week in the office (Tuesday, Wednesday and Thursday), with flexibility around the schedule.</li> <li>Lunch and snacks are on us whenever you come into the office.</li> <li>Work from another TrueFoundry office for up to 3 months each year, whether that's our London or Bay Area office, if you'd like a change of scenery.</li> </ul>