Backend Software Engineer (Research team)
- Employment
- Full Time
- Location
- London & San Francisco
- Posted
- 11d ago
llm
Application deadline: We are conducting interviews actively and aim to fill this role as soon as we find someone suitable. ABOUT THE OPPORTUNITY We’re looking for Backend Software Engineers who are excited to build tools for frontier AGI safety research, e.g. building and maintaining evals libraries and tools for monitoring and controlling our own LLM traffic. REPRESENTATIVE PROJECTS Here is a list of example projects which you might build and ship in your first 6 months. - Internal tooling for efficiently running and analyzing evaluations. For example, a tool that quickly investigates thousands of agentic eval runs in parallel and surfaces interesting information automatically - Automated evaluation pipelines to minimize the time from getting access to a new model for pre-deployment testing to analyzing the most important results and sharing them - Orchestration tools that allow researchers to run thousands of agentic evaluations in parallel on remote machines with high security and reliability - LLM proxy service that enables us to monitor all of our coding agent traffic in real time and identify undesired behavior automatically (in the spirit of Control) - LLM agents and MCP tools to automate internal software engineering and research tasks, with sandboxes to prevent major failures - CI pipeline optimisations to reduce execution time and eliminate flaky tests - Telemetry API and instrumentation of our existing tools, allowing us to monitor usage and improve reliability - Data warehousing pipeline and service to store thousands of eval transcripts which researchers can study and build datasets from - Upstream improvements to the Inspect framework and ecosystem, e.g. support for evaluating modern agentic scaffolds.