Full-Stack Software Engineer (Data Pipeline Focus) DescriptionIn this role, you'll contribute across the stack: developing ingest pipelines, building scalable REST APIs, and facilitating data exploration and understanding. The platform supports large-scale data ingestion, complex queries, and interactive analysis. While your primary focus will be on the data-pipeline layer, you’ll collaborate closely with other sub-teams to ensure end-to-end functionality and performance. We’re looking for someone excited to work across the system and to improve team processes and tooling, especially for faster integration of new data sources.ResponsibilitiesLead the design and implementation of data-processing workflowsManage all aspects of the data-processing lifecycle (collection, discovery, analysis, cleaning, modeling, transformation, enrichment, validation)Develop and maintain data models and JSON Schemas to ensure integrity and consistencyCollaborate with analysts and engineers to meet data requirementsManage and optimize data storage/retrieval in Elasticsearch and Dgraph (plus MongoDB and Redis)Orchestrate dataflow using Apache NiFiMentor teammates on best practices for data processing and software engineeringUse AI platforms to support hybrid automated/manual data transformation, code generation, and schema managementWork with analysts, product owners, and engineers to ensure solutions meet operational needsPropose and implement process improvements for faster delivery of new data sourcesRequired Skills & ExperienceStrong data-wrangling and dataflow background (discovery, mining, cleaning, exploration, enrichment, validation)Proficiency in JSON and JSON Schemas (or similar)Solid data-modeling experienceExperience with NoSQL databases (Elasticsearch, MongoDB, Redis, graph DBs)Familiarity with dataflow tools such as Apache NiFiExtensive experience in Python or Java (both preferred)Experience using generative AI for code and data transformationGit for version control; Maven for build automationComfortable in a Linux development environmentFamiliarity with Atlassian tools (Jira, Confluence)Strong communication and teamwork skillsNice to HaveExperience with various corporate data formatsKnowledge of Kafka or RabbitMQProficiency in Java/Spring (Boot, MVC/REST, Security, Data)AWS (EC2, S3, Lambda) experienceAPI design for data servicesFrontend experience (modern JS + Vue.js or similar)CI/CD (e.g., Jenkins), automated testing (JUnit)Docker, Kubernetes, and other containerization techDevOps tools (Packer, Terraform, Ansible)Qualifications12+ years of relevant experience and a B.S. in a technical discipline(Four additional years of experience may substitute for a degree)