Labeling & Evaluation Agent (French)
David Kennedy Recruitment · Lisbon, Portugal
David Kennedy Recruitment is working with a leading technology company specialising in artificial intelligence that is seeking to onboard a Labeling & Evaluation Agent to support the development and continuous improvement of its AI models.
The role focuses on evaluating AI-generated outputs, including chatbot responses, and providing high-quality annotations and feedback. You will assess response quality, factual accuracy, relevance, safety and policy compliance, helping improve the performance and reliability of large language models (LLMs).
Position: Labeling & Evaluation Agent (French)
Location: Lisbon, Portugal
Work model: On-site
Employment type: Full-time
DUTIES AND RESPONSIBILITIES:
- Evaluate AI-generated responses using detailed quality assessment and annotation guidelines
- Label and annotate language model outputs accurately and consistently
- Assess generated content for quality, relevance, factual accuracy, safety and policy compliance
- Apply complex annotation guidelines across different scenarios and content types
- Identify inconsistencies, edge cases and potential issues, escalating them when necessary
- Analyse structured text, language data and AI-generated content
- Maintain strict confidentiality and comply with project standards and data security requirements
- Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement
REQUIREMENTS:
- Native-level proficiency in French
- English proficiency at B2 level or above
- Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms
- Excellent analytical and critical thinking skills, with strong attention to detail
- Ability to make objective, evidence-based decisions
- Ability to interpret and consistently apply complex written guidelines and policies
- Strong reading comprehension and written language skills
- Ability to evaluate structured text, datasets and AI-generated content
- Ability to meet productivity, accuracy and quality targets
- Professionalism, integrity and discretion when handling confidential information
BENEFITS:
- Modern office in city center with open spaces, easy to access with public transportation
- Comprehensive initial training and guidance to make sure you're well prepared and comfortable at a new workplace
- Full-time employment in Lisbon
- 24/7 rotational working schedule
- 2 rotational days off
- Opportunity to work in the rapidly developing field of Generative AI and Large Language Models
- International and multicultural working environment
- Support from colleagues with coaching and mentoring opportunities
- Health and Wellbeing initiatives
- Monthly meal allowance
- And much more!!!