Phonely is a San Francisco-based provider of an AI voice platform that helps businesses optimize phone conversations.
At some point in the future, every business will answer their phone with voice AI. We are building the platform that makes it possible, and we are making our models better at real conversations than any general-purpose model on the market. Phonely builds conversational voice AI agents for high volume phone workflows. Our customers use us to qualify leads, book appointments, route calls, and resolve real customer conversations in production. The quality of those conversations comes down to our models, and that is where you come in. About the Role We're looking for an Applied Machine Learning Researcher to make our voice AI agents more natural, more reliable, and faster. You'll work on the core ML systems behind Phonely, fine-tuning models to beat frontier models on the conversational tasks that actually matter to our customers. This is applied research. Your work ships. Success here is measured in production model behavior getting better, not in prototypes or papers. You'll partner closely with engineering, product, QA, and the customer-facing teams to understand where models break in the real world, design rigorous experiments, improve the training data, and ship better models. NOTE: This is a hands-on research and engineering role. You'll write a lot of Python, dig through a lot of real production conversations, and own the numbers. It's demanding and it moves fast. This is a remote role, preferably based in Australia. What You'll Work On • Research and improve LLM behavior for real-time voice conversations • Design and run fine-tuning experiments across data, model, and evaluation strategies • Build evaluation frameworks for model quality, workflow-following, naturalness, reliability, task completion, and overall conversation quality • Analyze production conversations to find failure modes and opportunities to improve • Develop data curation, labeling, and synthetic data strategies • Compare model architectures, training approaches, prompts, and datasets • Investigate regressions and explain clearly why model behavior improves or degrades • Work with engineering to deploy research improvements safely and efficiently • Help define model release criteria, eval gates, and quality benchmarks What You'll Bring • Strong experience with machine learning and modern language models • Hands-on experience fine-tuning, evaluating, or adapting language models • Strong Python skills and comfort working with messy, real-world datasets • Ability to design rigorous experiments and interpret the results clearly • Experience building or improving evaluation systems for AI models • Strong analytical skills and the ability to debug model behavior • Clear written and spoken communication, so you can explain findings to technical and non-technical teammates alike • A practical mindset: you care about production impact, latency, reliability, and customer outcomes Nice to Have • A PhD in machine learning, NLP, or a related field • Experience with conversational AI, voice AI, or customer-support automation • Experience with SFT, preference tuning, DPO, RFT, GRPO, RLHF, LoRA, or QLoRA • Familiarity with model serving, inference optimization, or vLLM • Experience with synthetic data generation and data quality pipelines • Experience working in a startup or fast-moving product environment Why Join Phonely We're a group of ex-athletes, founders, and builders with low egos and a high belief that life is not about taking the easy road, but challenging ourselves to find the most we can be. Even with a remote team, we stay close: we're big on staying active, we back each other, and we care about the people we work with as much as the work itself. A few times a year we all get together in person and rent out Airbnbs in cool places (Rocky Mountains, Costa Rica, Indonesia) so you can see the world while on the grind. Interview Process 1. 15-minute intro call to evaluate fit 2. Deep dive on your ML and fine-tuning experience with a team member 3. Technical exercise or case study on a real model-quality problem 4. Final conversation with leadership