Principal AI Ops Engineer

About this role

<div class="content-intro"><p>We are a healthcare technology company that provides platforms and solutions to improve the management and access of cost-effective pharmacy benefits. Our technology helps enterprise and partnership clients simplify their businesses and helps consumers save on prescriptions.</p> <p>As a leader in SaaS technology for healthcare, we offer innovative solutions with integrated intelligence on a single enterprise platform that connects the pharmacy ecosystem.&nbsp; With our expertise and modern, modular platform, our partners use real-time data to transform their business performance and optimize their innovative models in the marketplace.</p></div><p><strong>Position Summary:</strong></p> <p>The Principal AIOps Engineer will build the platform that makes AI cheap, fast, safe, and observable at RxSense. As a direct report to the Director of AI Engineering, this role will own the infrastructure that every AI-powered product at RxSense depends on. This is a hands-on-keyboard position from day one, partnering with AI engineers, software engineers, data scientists, security, and finance to deliver deployment pipelines, agent runtime, eval frameworks, self-hosted model serving, and the developer harness that determines how fast every other engineer in the company can ship.</p> <p><strong>Essential Duties and Responsibilities:</strong></p> <ul> <li>Build and maintain end-to-end deployment pipelines for AI-powered applications, including artifact builds, environment promotion, rollback, and observability hooks. Drive new greenfield deployment platforms from initial build to the default that AI teams ship on.</li> <li>Stand up and operate the runtime and lifecycle infrastructure for production agents, including deployment, versioning, monitoring, rate-limiting, and retirement. Define the deployment contract (config, secrets, tools, memory, evals) and the operational SLOs.</li> <li>Own how the organization provisions, rotates, scopes, and meters access to model provider APIs (Anthropic, OpenAI, and others). Build a key management layer that enforces per-team and per-app quotas, prevents leakage, and gives finance and engineering a clear view of spend.</li> <li>Build evals into the CI/CD pipeline so no agent or LLM-powered service ships without passing a defined eval bar. Design the framework so product teams can author their own evals against a shared harness, and so eval results gate promotion across environments.</li> <li>Stand up self-hosted inference for workloads where managed APIs aren't the right fit, including latency-sensitive paths, regulated data, cost optimization, and vendor redundancy. Own the serving stack, the autoscaling and GPU economics behind it, and the playbook for when a workload belongs to a managed provider versus internal infrastructure.</li> <li>Design and build the shared developer harness that every AI-powered service uses: prompt management, model routing, retries, tracing, eval hooks, and policy enforcement. Set the abstractions that determine how fast every other AI engineer can ship for the next three years.</li> <li>Partner with finance on cost visibility, including token accounting, per-feature cost attribution, and real-time spend observability.</li> <li>Write documentation, runbooks, and clear interfaces so the platform is adoptable by other engineering teams without hand-holding.</li> <li>Participate in code review and promote collaboration and best practices including simplicity, automation, sound design patterns, test coverage, and reusability.</li> </ul> <p><strong>Education/Experience/Competencies:</strong></p> <ul> <li>BS (or higher, e.g., MS or Ph.D.) in Computer Science or related technical field involving coding, or equivalent technical experience.</li> <li>6+ years of platform, infrastructure, or DevOps engineering, with at least 2 years building production infrastructure for AI/ML or LLM-powered systems. We care more about depth and drive than years on a resume.</li> <li>Deep hands-on experience designing and operating CI/CD pipelines for high-velocity engineering organizations, including artifact management, environment promotion, and progressive rollout.</li> <li>Strong AWS background, comfortable down to the IAM, networking, and container orchestration layers.</li> <li>Proven track record building developer platforms or internal tools that other engineering teams adopted by choice, not by mandate.</li> <li>Production experience with LLM-powered applications, including prompt management, model routing, retries, tracing, and the operational realities of running agents or chains in production.</li> <li>Hands-on coding fluency in Python or TypeScript, ideally both. This is a keyboard role, not an architecture-only role.</li> <li>Comfortable operating in a polyglot environment. The RxSense AI engineering stack spans Python, .NET, and TypeScript, and you will deploy and support services across all three.</li> <li>Comfortable owning the cost and reliability conversation with both engineering leadership and finance partners.</li> <li>Strong written communication and a bias toward documentation, runbooks, and clear interfaces.</li> <li>Proven analytical thinking and problem-solving skills.</li> <li>Excellent communication skills, both verbal and written.</li> </ul> <p><strong>Bonus Qualifications:</strong></p> <ul> <li>Direct experience integrating with Anthropic, OpenAI, or other frontier model provider APIs at scale, including key management, quota enforcement, and capacity planning.</li> <li>Hands-on experience self-hosting models with vLLM, TGI, SGLang, or similar inference servers, including GPU autoscaling and cost optimization.</li> <li>Built or contributed to an eval framework that gated production deployments.</li> <li>Familiarity with agent runtimes and frameworks such as the Claude Agent SDK, LangGraph, or in-house equivalents.</li> <li>Working familiarity with .NET, enough to read code, debug a deploy, and pair with se