Senior Data Engineer

About this role

<p><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Job Title: </strong>Senior Data Engineer</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Key Skills:</strong>&nbsp;DBT, Python, SQL, Snowflake, Databricks</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Location:</strong>&nbsp;Brazil</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Mode:&nbsp;</strong>Remote</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">We at&nbsp;Coforge&nbsp;are hiring&nbsp;<strong>Senior Data Engineer</strong>&nbsp;with the following skill set.</span><br><br><strong><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Key Responsibilities</span></strong></p> <ul> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Delivery &amp; Ownership</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Own the end-to-end delivery of data pipelines and data products, from design through deployment and production support</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Develop, maintain, and optimise scalable data pipelines using Azure Databricks, DBT, and Python</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Ensure solutions are production-ready, reliable, and meet defined SLAs</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Migration &amp; Platform Engineering</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Execute and support migration of data from Snowflake (AWS) to Azure using Databricks, DBT, and Lakebridge</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Build and manage infrastructure-as-code using Terraform to enable scalable and repeatable deployments</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Ensure seamless integration between legacy systems and the Azure data platform</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Data Quality, Observability &amp; Governance</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Implement data quality frameworks (e.g., DBT testing, validation rules) to ensure trusted data pipelines</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Enable observability through logging, monitoring, and alerting of pipelines and data workflows</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Support data governance standards including data lineage, consistency, and compliance requirements</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Architecture &amp; Engineering Excellence</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Contribute to the design and architecture of scalable data solutions, aligned with lakehouse principles</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Apply best practices in data modelling, performance optimisation, and cost efficiency</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Follow and promote engineering standards, including modular design and reusable components</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Production Support &amp; Reliability</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Monitor and maintain data pipelines in production, ensuring high availability and performance</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Troubleshoot issues and perform root cause analysis (RCA) for incidents</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Continuously improve system stability, resilience, and operational processes</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Collaboration &amp; Technical Leadership</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Work closely with cross-functional teams (data, product, architecture, business) to translate requirements into scalable technical solutions</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Provide guidance and mentorship to junior engineers through code reviews and best practices</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Contribute to documentation of architecture, pipelines, and operational processes</li> </ul> <p>&nbsp;</p> <p><strong><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Required Skills &amp; Qualifications</span></strong></p> <ul> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Strong hands-on expertise with:&nbsp;Azure Databricks,&nbsp;DBT,&nbsp;Snowflake,&nbsp;Python and SQL</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Proven experience building and managing large-scale data pipelines and transformations</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Experience working with high-volume datasets (e.g., terabytes, complex data models, multiple pipelines)</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Solid understanding of:&nbsp;Data modelling and ETL/ELT frameworks,&nbsp;Distributed data systems and lakehouse architectures,&nbsp;Experience with Terraform and infrastructure-as-code principles</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Strong experience with CI/CD pipelines and Git-based version control</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Experience supporting production systems, including monitoring, alerting, and incident resolution</li> <li style="font-family: arial, helvetica, sans-se