Research Engineer, Large-Scale Training

About the company

Together AI is an AI neocloud that provides AI-focused cloud infrastructure and tools to support adoption of open-source models.

About this role

<h3><span style="font-family: helvetica, arial, sans-serif;">About the Role</span></h3> <p class="isSelectedEnd"><span style="font-family: helvetica, arial, sans-serif;">The Model Shaping team at Together AI works on products and research for tailoring open foundation models to downstream applications. We build services that allow machine learning developers to choose the best models for their tasks and further improve these models using domain-specific data. In addition, we develop new methods for more efficient model training and evaluation, drawing inspiration from a broad spectrum of ideas across machine learning, natural language processing, and ML systems.</span></p> <p class="isSelectedEnd"><span style="font-family: helvetica, arial, sans-serif;">As a Research Engineer on the Scaling Team within Model Shaping, you will turn cutting-edge research on efficient foundation model training into robust, high-performance systems. You will profile and optimize Together's training infrastructure, identify performance bottlenecks across the stack, and implement state-of-the-art techniques from both the research literature and our own scientists in production environments.</span></p> <p class="isSelectedEnd"><span style="font-family: helvetica, arial, sans-serif;">Your work will directly shape the fine-tuning experience of Together's customers. You will rapidly bring newly released open-source models onto the Model Shaping platform, ensuring they train efficiently and reliably across diverse customer workloads. Working closely with Research Scientists, you will also build the experimental infrastructure that accelerates research and enables validated ideas to be deployed reliably at scale.</span></p> <h3><span style="font-family: helvetica, arial, sans-serif;">Responsibilities</span></h3> <ul data-spread="false"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Design, implement, and optimize core components of Together's large-scale training infrastructure.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Integrate new model architectures, validate training correctness and convergence, and optimize performance for production fine-tuning workloads.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Profile distributed training workloads to identify and eliminate bottlenecks across compute, memory, and communication.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Design and execute experiments to validate performance hypotheses and benchmark new approaches against state-of-the-art methods.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Partner closely with Research Scientists to productionize novel training methods and contribute to publications and open-source releases.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Rapidly enable support for newly released open-source foundation models on the Together platform.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Build and maintain experimental infrastructure that accelerates research while ensuring production-quality reliability and scalability.</span></li> </ul> <h3><span style="font-family: helvetica, arial, sans-serif;">Requirements</span></h3> <ul data-spread="false"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Demonstrated ability to independently take ambiguous performance or infrastructure problems from investigation through deployment.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong programming skills in Python and PyTorch, with an emphasis on writing efficient, maintainable code.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Hands-on experience training or fine-tuning large neural networks in multi-GPU or multi-node environments.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Solid understanding of ML systems fundamentals, including GPU architecture, mixed-precision training, and distributed training paradigms such as data, tensor, pipeline, or expert parallelism.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong communication skills and the ability to collaborate effectively with both researchers and engineers.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Passion for staying current with advances in AI research and applying them to real-world systems.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Excitement about translating cutting-edge research into production systems that deliver customer impact.</span></li> </ul> <h3 class="isSelectedEnd"><span style="font-family: helvetica, arial, sans-serif;"><strong>Nice to Have</strong></span></h3> <ul data-spread="false"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience writing optimized NVIDIA GPU kernels using CUDA or Triton, or implementing communication collectives with technologies such as NCCL or NVSHMEM.</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience with large-scale training frameworks