Together AI is an AI neocloud that provides AI-focused cloud infrastructure and tools to support adoption of open-source models.
<h3><strong>About the Role</strong></h3> <p>Together AI is building the best inference infrastructure for voice applications. Our Voice AI platform powers production-grade, real-time voice agents and applications — serving speech-to-text and text-to-speech models with best-in-class latency and reliability.</p> <p>We're looking for a Staff ML Engineer to drive the model serving layer for voice workloads. You'll work hands-on with inference engines like TRT-LLM and SGLang to optimize how we serve models like Whisper, Parakeet, Orpheus, and Kokoro — pushing latency and throughput to the frontier. You'll profile GPU utilization, design batching strategies for streaming audio, and ensure new model architectures can go from research to production quickly.</p> <p>This is a foundational hire on a small, high-impact team. Voice inference has unique challenges — streaming audio, tokenization, real-time latency budgets — that require dedicated ML engineering focus. You'll shape how Together serves voice models as the industry moves from pipeline architectures (ASR → LLM → TTS) toward end-to-end speech-to-speech.</p> <ul> <li>Own the model serving stack that powers Together's voice platform across STT, TTS, and speech-to-speech.</li> <li>Work directly with state-of-the-art accelerators (H100s, H200s, B200s) to optimize voice model inference.</li> <li>Collaborate with model partners (Cartesia, Deepgram, Rime, and others) to bring their models to production on Together's infrastructure.</li> <li>Build quality evaluation frameworks that guide model selection for customers and inform the roadmap.</li> <li>Join a small, early-stage team with outsized impact on a fast-growing product area.</li> </ul> <p class="font-claude-response-body break-words whitespace-normal leading-[1.7]"> </p> <p class="font-claude-response-body break-words whitespace-normal leading-[1.7]"><strong>Responsibilities</strong></p> <ul class="[li_&]:mb-0 [li_&]:mt-1 [li_&]:gap-1 [&:not(:last-child)_ul]:pb-1 [&:not(:last-child)_ol]:pb-1 list-disc flex flex-col gap-1 pl-8 mb-3"> <li class="font-claude-response-body whitespace-normal break-words pl-2">Own the voice inference roadmap end-to-end — define and execute the technical strategy for optimizing STT, TTS, and speech-to-speech models across Together's infrastructure, with a clear-eyed view of where the field is heading and how to position the platform ahead of it.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Drive best-in-class inference performance — architect and implement systems targeting leading TTFB, throughput, and GPU utilization for voice workloads; set the performance bar others in the industry measure against, not just catch up to.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Lead productionization of voice models at scale — design the serving architecture for serverless and dedicated endpoints, including batching strategies, streaming inference pipelines, and memory management tailored to real-time audio; own reliability and latency SLAs.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Build the voice evaluation platform — design a rigorous, extensible evaluation framework covering WER across accents, languages, and noise conditions for STT; naturalness, latency, and pronunciation fidelity for TTS; establish the internal benchmark methodology that informs model selection and roadmap decisions.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Shape the architecture for next-generation model support — anticipate and enable emerging model paradigms — audio-native LLMs, codec-based architectures (SNAC, Encodec), and end-to-end speech-to-speech systems — before they're mainstream, not after.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Serve as the technical DRI for model partner integrations — lead deep collaboration with partners such as Cartesia, Deepgram, and Rime; own the full lifecycle from integration to optimization to ongoing performance accountability.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Diagnose and resolve the hardest performance problems in the stack — conduct systematic profiling and root-cause analysis from GPU kernel behavior to framework-level bottlenecks; drive shipped improvements with documented, measurable impact.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Influence platform architecture across the organization — partner with platform engineering leadership to ensure the serving layer is built for the latency and reliability demands of real-time voice APIs; your technical decisions should raise the ceiling for the whole team.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Define and scale voice fine-tuning capabilities — lead the technical direction for enabling customers to fine-tune STT and TTS models on Together's infrastructure, establishing the primitives for differentiated voice experiences.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Lay technical foundations for a category-defining product surface — architect systems with enough foresight that they support multiple new voice products with minimal rework; think in terms of platforms, not point solutions.</li> </ul> <p class="font-claude-response-body break-words whitespace-normal leading-[1.7]"><strong>Requirements</strong></p> <ul class="[li_&]:mb-0 [li_&]:mt-1 [li_&]:gap-1 [&:not(:last-child)_ul]:pb-1 [&:not(:last-child)_ol]:pb-1 list-disc flex flex-col gap-1 pl-8 mb-3"> <li class="font-claude-response-body whitespace-normal break-words pl-2">8+ years of ML engineering experience, with a demonstrated focus on model serving, inference optimization, or ML infrastructure at production scale — including systems you've owned from design through live traffic.</li>