Astronomer provides Astro, a DataOps platform and Airflow-as-a-Service that enables teams to build, run, observe Apache Airflow pipelines with lineage, data quality, dbt integration, governance and Kubernetes-native deployments.
Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than 800 of the world's leading enterprises, Astronomer lets businesses do more with their data. To learn more, visit www.astronomer.io http://www.astronomer.io. ABOUT THIS ROLE Apache Airflow is one of the most popular open-source data platform tools. It powers the data platforms at nearly every large company and fast-growing startups: Airbnb, Uber, OpenAI, Anthropic, Nike, Capital One, Disney all use Airflow extensively. At Astronomer, we’re the largest contributors to the project and are building commercial products around Airflow to make it easier to use, run, and scale. We’re in a unique position: as the company behind Apache Airflow, we see data as it moves across entire organizations—from raw ingestion to production dashboards, machine learning models, and AI products. Leveraging this vantage point, our R&D team is developing a global context layer for data—an intelligence layer that powers search and discovery, code generation for data and analytics, and automated root cause analysis. LLMs are already quite good at writing Python and SQL code against data platforms; we think this context layer will give them the metadata necessary for data practitioners everywhere to use LLMs effectively. As a Software Engineer on this team, you’ll help design and build this foundation and the applications around it. You’ll work on some of the hardest and most exciting challenges in data—search, information retrieval, and AI for data pracitioners—while collaborating with a small, highly skilled team that values velocity, creativity, and impact. This role sits at the intersection of applied research, software engineering, and product: we think it takes someone who can work across the stack to build, release, and scale products successfully in this space. Hybrid Work Model: For this role, you will embrace a flexible hybrid work model with at least 3 days per week in our New York City office. WHAT YOU GET TO DO: - Shape the future of AI for data engineering - build intelligent systems that understand, reason about, and optimize the flow of data across entire organizations. - Design and engineer the brain of Astronomer’s context layer, crafting components that power data modeling, semantic search, retrieval, and code generation. - Push the boundaries of applied AI - experiment with LLMs, embeddings, and cutting-edge retrieval techniques to create developer tools that deliver insights to you and our customers. - Turn research into reality - work side by side with R&D and product teams to bring early AI concepts to life in the product experience. - Solve high-impact information retrieval and search challenges at a global scale, leveraging Astronomer’s unparalleled visibility into data pipelines across industries. - Influence the technical vision and architecture for the next generation of AI-driven data products. - Represent Astronomer in the community - through open-source contributions, technical talks, and publications that showcase our leadership in AI and data innovation. WHAT YOU BRING TO THE ROLE: - 5-8 years of software engineering experience with Python or Go - Empathy for users, and a deep interest in improving the workflows of data professionals. - Familiarity with early-stage product development; comfortable working with ambiguity in a fast-changing field. - Experience with LLMs, vector databases, embeddings, or other applied AI areas—or a strong desire to dive in. - A creative, experimental mindset: you enjoy exploring uncharted areas, validating hypotheses, and learning through iteration. - Strong collaboration and communication skills—you can explain complex systems clearly to both technical and non-technical audiences. - A collaborative approach and comfort working in an evolving, research-driven environment where ideas move quickly. BONUS POINTS IF YOU HAVE: - A passion for AI systems for data, developer tools, or machine learning infrastructure. - Familiarity with Apache Airflow or other orchestration tools. - Demonstrated contributions to open source projects. - Experience in search, IR, or large-scale data infrastructure. - Exposure to early-stage startups or R&D organizations where ambiguity is the norm. - Experience building out agentic systems on top of frontier models. The estimated total compensation for this role ranges from $200,000 - $230,000 based on leveling and geography, along with an equity component and a comprehensive benefits package. This range is merely an estimate; actual compensation may deviate from this range based on skills, experience, and qualifications. #LI-Fulltime #LI-Hybrid At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.