Data Engineer - QA

Reporting to the Engineering Manager, you will join a feature team focused on the design, development, and quality assurance of the Data Platform component of the ThunderSoft Cloud Platform.

As part of the Data Platform team, you will play a key role in ensuring data reliability, accuracy, and consistency across the platform, supporting critical business and analytics use cases.

You will contribute to building a robust data quality framework within a modern cloud environment (Databricks / AWS), ensuring that data pipelines are not only performant but also trustworthy and testable.

Mission

You will join the Data Platform team as a Data Engineer with a strong focus on Data Quality and QA practices. You will be responsible for embedding quality assurance, validation, and observability mechanisms into data pipelines and ensuring that data products meet the highest standards of reliability and integrity.

This includes designing and implementing automated data tests, monitoring systems, and quality frameworks, as well as collaborating with data engineers, analysts, and product teams to establish data quality standards and best practices.

Responsibilities

• Define, implement, and maintain data quality rules and validation frameworks.
• Design and integrate automated tests into ETL/ELT pipelines (unit, integration, regression tests for data).
• Design, build, and enhance data pipelines with built-in quality checks and assertions.
• Implement data observability and monitoring solutions (alerts, anomaly detection).
• Analyze and troubleshoot data quality issues in production environments.
• Validate data transformations and ensure alignment with business logic and functional expectations.
• Collaborate closely with data engineers, data analysts, and business teams to define data contracts and acceptance criteria.
• Ensure production reliability, including monitoring pipelines and responding to incidents.
• Promote and enforce QA best practices in data engineering (test automation).

Qualifications

• Master’s degree (BAC+5) with at least 4 years of experience as a Data Engineer, Data QA Engineer, or in a similar role.
• Strong expertise in Python, with experience in data validation frameworks and testing libraries.
• Solid experience in ETL/ELT pipelines with embedded testing practices.
• Hands‑on experience with Databricks and Spark, including data validation at scale.
• Strong SQL skills with ability to write complex validation queries.
• Experience with cloud environments (AWS preferred).
• Understanding of CI/CD pipelines and test automation strategies for data platforms.
• Experience working in an agile environment and commitment to agile principles.

Additional Skills

• Upper‑intermediate English level and ability to communicate effectively.
• Strong analytical and problem‑solving mindset.
• Attention to detail and strong ownership of data quality.
• Strong team spirit and collaboration mindset.
• Comfortable working in an Agile/SAFE environment.