ML Infrastructure Engineer
Company hidden until unlock
- Work model
- Onsite
- Location
- San Mateo
- Posted
- 5d ago
pythongorustjavac++awsgcpazurekubernetesdockertensorflow
Company hidden until unlock
ABOUT THE ROLE This is a hands-on ML Infrastructure Engineer role at an early-stage enterprise AI startup, where you'll own the end-to-end inference and model-serving infrastructure that keeps production AI agents running reliably and at scale. You'll sit at the intersection of ML and platform engineering, directly shaping the systems that power real-world, high-stakes deployments in regulated industries like insurance, banking, and healthcare. WHAT YOU'LL DO - Own inference and model-serving infrastructure end to end, from design through production deployment. - Build and scale systems that enable AI agents to run reliably and efficiently under increasing concurrency. - Collaborate closely with ML and infrastructure teams to ensure seamless integration and performance optimization. - Identify infrastructure bottlenecks and drive cross-functional solutions across engineering teams. WHAT WE'RE LOOKING FOR - 5+ years of experience building and operating ML inference systems, model-serving platforms, or ML infrastructure in production. - Hands-on experience designing and scaling inference-serving infrastructure using frameworks such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems. - Strong track record optimizing production ML systems for latency, throughput, and reliability at scale. - Experience with containerization and orchestration (Docker, Kubernetes) for deploying and scaling ML workloads. - Experience building distributed systems that handle concurrent requests and manage resource allocation under load. - Proficiency with observability and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing). - Cloud platform experience on AWS, GCP, or Azure for deploying and managing ML systems. - Proficiency in at least one systems or backend language — Python, Go, Rust, C++, or Java. - Nice to have: experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune); real-time or low-latency inference systems; agentic or multi-step reasoning pipelines; enterprise data infrastructure or integration platforms. LOCATION On-site in San Mateo, CA. No visa sponsorship is available for this role.