Data Scientist (LLM)

Requirements

• 2–5 years of experience in software engineering, applied AI, or a similar role
• Strong programming skills in Python or TypeScript
• Experience building LLM-powered applications or AI agents in production environments
• Comfortable working with real-world data, including reviewing large volumes of conversations to identify issues and improvement opportunities
• Experience designing and iterating on prompts, including techniques such as chaining, few-shot prompting, or tool usage
• Familiarity with evaluating LLM performance through structured methods, such as offline evaluation and A/B testing
• Product‑minded, with a focus on delivering solutions that improve user experience and business outcomes
• Able to balance speed and quality, and iterate quickly based on data and feedback
• Strong communication skills, able to collaborate effectively with engineers, product managers, and other stakeholders
• (Desirable) Experience building internal tools or platforms
• Experience with orchestration or workflow tools, such as Temporal
• (Desirable) Experience working on customer support, SaaS, or e-commerce products

What the job involves

• As a Data Scientist focused on LLM at Gorgias, you will play a crucial role in developing, refining, and enabling our AI Agent, designed to automatically address and resolve shopper requests for our merchant clients
• Beyond prompt engineering, you'll be instrumental in scaling the AI Agent's capabilities by developing the right LLM strategies to enable our services
• You will work closely with our ML Engineers, Product Managers, and Software Engineers to create, optimize, and implement solutions that enhance the efficiency, accuracy, and functionality of our AI-driven customer support solutions
• Design and iterate on LLM‑based workflows, including prompting, chaining, and tool usage, to improve the performance of AI agents in production
• Build and improve systems that reliably handle real customer support interactions at scale
• Review large volumes of customer conversations to identify failure modes and areas for improvement
• Develop and maintain evaluation frameworks, including offline testing and online A/B experiments
• Define and track metrics to measure AI agent performance and guide iteration
• Create and maintain datasets for continuous improvement and non‑regression
• Collaborate with Product Managers and Engineers to prioritize improvements and ship high‑impact features
• Contribute to the design of internal tools and abstractions that enable scalable AI agent development across teams
• Integrate AI agents with internal systems and external e-commerce platforms
• Document methodologies, best practices, and learnings to support team scalability and knowledge sharing