QA Engineer

# QA Engineer : 200 systems in production, and they have to hold up

Past a certain number of systems in production, the constraint is no longer building. It is guaranteeing that it holds. That is this role, and it is new. A useful clarification: this is QA for AI systems, not interface testing. You test non-deterministic outputs, not buttons.

Postuler/ Le poste## What you take on### You write an evaluation suite, and it lies

You build a test set for a client's support agent. It passes at 95%. Two weeks later a user asks a question phrased unlike anything in your test set, and the system answers nonsense. Your job: maintaining a test set that keeps closing in on real traffic, week after week.### You break an n8n workflow for good reasons

And check that nothing else breaks with it. A node is changed to fix a bug; a downstream step that depended on a precise output format fails silently. Your regression tests catch that before the client does.### You are the last gate before delivery

Business test set, the client's edge cases, checking that the outputs are presentable to a leader who will use them to decide. You say yes or no to going live.### You verify a number the company repeats everywhere“7 hours a week recovered on average” is a central commercial argument, and a verifiable claim. You build the method to audit it project by project, and you say when it does not hold for a given case.### You watch for what degrades without warning

Quality dropping over several weeks with no alert, token costs blowing up. You build the signals that make those drifts visible before the client sees them.### You write a lot, and to be understood immediately

Acceptance reports, non-conformity notes, drift write-ups: readable in thirty seconds by someone who has not followed the project./ Your first 90 days

Month 1, you audit what exists and tell us where it breaks most often. Month 2, you put the evaluation foundation in place on a live system. Month 3, no system ships to a client without going through your acceptance./ Le profil## Ce qu'on cherche

Evaluation techniques: Golden datasets, scoring rubrics, LLM-as-judge calibrated against human annotation, regression testing

Tools: Solid Python, an eval framework (promptfoo, Deep

Eval or equivalent), reading and testing n8n workflows, CI/CDNon-determinism: Thinks in acceptable quality ranges, tells normal variation from a real regression

Writing: Clear reports, readable by a non-technical audience

Field work: 3 to 8 years in QA, test engineering or SDET, part of it on AI systems in production, comfortable saying no under pressure

Ce qu'on ne cherche pas

Someone who executes test plans written by others. Here, you write the strategy./ Le contexte## Where it differs from neighbouring roles| Role | What they do | Who owns the outcome || --- | --- | --- || Generalist QA or manual tester | Predefined scenarios on deterministic software | Functional coverage owns the outcome || MLOps or Platform Engineer | Infrastructure that runs the models | Technical availability owns the outcome || Data Scientist or ML Engineer | Designs and trains the models | Upstream model performance owns the outcome || QA Engineer, AI Systems, AI Makers | Builds the evaluations, decides go or no-go | You own the reliability of the published numbers |/ Where

Remote, in a time zone compatible with France and Morocco (Europe, North Africa or nearby), with no residency requirement, and occasional time in our offices.## Is this role for you?

No cover letter. The form takes five minutes and starts with showing what you have built.

#J-18808-Ljbffr