TrueFoundry provides a Kubernetes-based, cloud-agnostic PaaS that automates ML workflows—model deployment, autoscaling, monitoring, RAG/agent workflows, GPU/memory optimization, and centralized access controls for multi-cloud and on-premises environments.
<p><strong>About TrueFoundry<br></strong><br>Every production AI system, whether it's powering customer support, writing code, analyzing financial data, or diagnosing medical conditions, needs the same foundational infrastructure.A way to route between models. A way to manage tools and integrate them securely. A way to orchestrate agents and enforce governance. A unified compute layer to run it all.</p> <p><strong>That infrastructure layer is being built right now.</strong></p> <p>We are looking for a Senior Software Engineer-Core Team.</p> <h2><strong>The Problem We're Solving</strong></h2> <p>Companies are moving beyond simple chatbots to production agentic systems. These systems route between OpenAI, Anthropic, Google, and self-hosted models. They integrate dozens of tools via protocols like MCP. They orchestrate multi-agent workflows where agents coordinate with other agents.</p> <p>The infrastructure to support this doesn't exist yet. You can't just duct-tape together a few API calls and call it production-ready.</p> <p>You need a control plane that handles:</p> <ul> <li>Intelligent routing with observability, cost policies, and fallback logic</li> <li>Centralized tool and MCP server management with security and lifecycle controls</li> <li>Agent orchestration with governance and guardrails</li> <li>A unified compute layer to run self-hosted models, custom tools, and agents</li> </ul> <p><strong>AI Gateway</strong> is the control plane: five composable components (Prompts, LLM Gateway, MCP Gateway, Guardrails, Agent Gateway) that handle routing, orchestration, and governance.</p> <p>We're Series A, backed by Intel Capital and Sequoia. Companies like CVS, Mastercard, Siemens, Paytm, Synopsys, and Zscaler run production AI workloads on our platform.</p> <p>You will:</p> <ul> <li>Solve some of the most complex Engineering problems and drive it alongside a team of engineers & ML researchers.</li> <li>Build a deep, holistic understanding of the TrueFoundry platform across all components and shape the product vision and implementation.</li> <li>Act as the technical face of engineering for customer-related discussions and escalations</li> <li>Guide and unblock engineers across projects in the US region</li> <li>Partner closely with our CTO and India-based engineering team to drive system design, architecture, and implementation of complex products</li> <li>Lead technical design, critical customer problem-solving, and platform scalability initiatives end-to-end</li> </ul> <p>This is a high-ownership, high-impact role designed for an engineer who loves combining world-class systems thinking with real-world execution.</p> <h3><strong>What You’ll Do:</strong></h3> <ul> <li>Develop deep expertise across TrueFoundry’s platform stack — infrastructure, deployment systems, LLM/ML orchestration, observability, cost optimization, and more</li> <li>Drive the system architecture and design for complex, distributed, cloud-native systems</li> <li>Act as the technical point-of-contact for enterprise customer engineering needs and escalations</li> <li>Lead and participate in design reviews, code reviews, and critical incident responses</li> <li>Collaborate closely with the CTO on architectural decisions, scaling strategies, and technical roadmap prioritization</li> <li>Guide and mentor US-based engineers across multiple initiatives, helping them deliver high-quality, scalable systems</li> <li>Identify and drive technical debt cleanup, performance improvements, and resilience upgrades across the platform</li> <li>Bring a product engineering mindset, ensuring that customer needs and feedback translate into scalable engineering solutions</li> </ul> <h3><strong>Who You Are:</strong></h3> <ul> <li>4-8 years of strong backend/systems engineering experience at top technology companies or startups</li> <li>Deep expertise in distributed systems, cloud-native architectures, and scalable system design</li> <li>Strong working knowledge of Kubernetes, containerized workloads, and infrastructure engineering</li> <li>Practical experience building or deploying ML/GenAI applications (or closely working with ML/DS teams)</li> <li>Skilled in programming languages such as Python, Golang, TypeScript, and Node.js</li> <li>Solid understanding of system observability, resiliency design, and SRE practices</li> <li>Strong technical leadership and communication skills — able to work with both customers and engineering teams</li> <li>Ability to think strategically while also executing hands-on when required</li> </ul> <h3><strong>Perks of Working at TrueFoundry</strong></h3> <ul> <li>Join a fast-growing Series A, Bay Area-based startup building cutting-edge AI infrastructure.</li> <li>Comprehensive health insurance for you and your family.</li> <li>Flexible hybrid work, 3 days a week in the office (Tuesday, Wednesday and Thursday), with flexibility around the schedule.</li> <li>Lunch and snacks are on us whenever you come into the office.</li> <li>Work from another TrueFoundry office for up to 3 months each year, whether that's our London or Bay Area office, if you'd like a change of scenery.</li> </ul>