LLMOps / AI Runtime Engineer

Level of qualifications required : Graduate degree or equivalent

Fonction : Support functions

Level of experience : From 5 to 12 years

Context

Following the priorities established in May 2024 by the Seoul Declaration for Safe, Innovative and Inclusive AI, to which France is a signatory, the French government decided to create INESIA, an institute whose mission is to bring together , without creating a new legal entity, national stakeholders involved in AI evaluation and safety , in particular:

the French Cybersecurity Agency (ANSSI),

the National Laboratory of Metrology and Testing (LNE),

the Digital Regulation Expertise Center (PEReN),

and the French National Institute for Research in Digital Science and Technology (Inria).

Within this framework, Inria primarily contributes to activities related to systemic risk analysis in the field of national security, as well as the evaluation of the performance and reliability of AI models.

This work is strategically coordinated with Inria’s AI Evaluation research program and materializes through the design and development of an AI evaluation platform, particularly focused on systems based on Large Language Models (LLMs).

The platform aims to provide an integrated, secure, and robust environment supporting the program’s research projects, while enabling the development of evaluation applications such as benchmarking campaigns and red teaming exercises. It relies on open-source tools from the AI ecosystem as well as internally developed components.

You will join a team operating in a fast-paced, iterative development environment: the platform will evolve progressively through regular operational deliverables. We are looking for individuals capable of proposing solutions, making technical trade-offs, and transforming technical requirements into operational systems.

This position is at the core of the platform’s value proposition: enabling the evaluation of sensitive LLM applications in realistic, controlled, and secure environments. It offers the opportunity to contribute to a strategic and ambitious project at the heart of current challenges related to AI safety, transparency, and governance, spanning technical, scientific, and societal dimensions.

Assignment

Design, develop, and operate the runtime environment manager used to deploy, version, and reproduce AI systems across diverse execution contexts.

Main activities

• Manage the deployment of LLM-based systems:
  • inference engines
  • RAG pipelines
  • agents using external tools
• Design and implement the environment definition system (formats, configuration, versioning)
• Develop build and deployment mechanisms for environments (containers, images, dependencies)
• Ensure reproducibility of execution environments through fine-grained dependency, version, and configuration management
• Integrate the environment manager with workers and orchestration systems
• Enable execution across multiple contexts:
  • local developer environments
  • HPC clusters (SLURM, OAR, etc.)
  • cloud infrastructures / Kubernetes
• Optimize environment performance and deployment times
• Contribute to technical architecture decisions related to infrastructure and reproducibility
• Document environments and usage best practices

Skills

Required Skills

• Experience deploying LLMs (vLLM, SGLang, Triton, ...) and complex systems (RAG pipelines, agents, ...)
• Strong experience with containerization technologies (Docker, Apptainer/Singularity)
• Experience with distributed environments or cluster execution
• Strong proficiency in Python and the ML ecosystem
• Familiarity with software development best practices (Git versioning, CI/CD, documentation)
• Ability to write technical documentation

Preferred Skills

• Experience with MLOps tools (ClearML, MLFlow, Kubeflow, etc.)
• Knowledge of HPC environments (OAR, Slurm)
• Familiarity with reproducible packaging tools (Guix, Nix, etc.)
• Awareness of performance optimization challenges

Additional Appreciated Skills

• Experience in academic research
• Technical English proficiency, both written and spoken
• Awareness of AI trustworthiness and safety challenges

We encourage you to apply even if you do not meet every requirement: we value candidates who are eager to learn and grow new skills.

Defence Security

This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. relating to the protection of national scientific and technical potential (PPST). Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.

Recruitment Policy

As part of its diversity policy, all Inria positions are accessible to people with disabilities.