Head of Data Labeling

Навыки

• AI-инструменты в работе
• API (интеграции)
• Анализ данных
• Datadog / New Relic
• Дообучение моделей
• Лидерство
• LLM

О компании и продукте

• White Circle https://whitecircle.ai/ is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies – simple natural-language rules that define what an AI model should and shouldn’t do. We automatically test, enforce, and continuously improve these policies at scale.
• We’ve raised $11M from top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others
• We process over 100M+ API calls every month
• We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model

Задачи

• Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)
• Define annotation guidelines, quality standards, and evaluation frameworks
• Develop quality assurance processes, calibration sessions, and auditing systems
• Partner with AI researchers and engineers to translate research objectives into labeling workflows
• Prioritise labeling projects based on business and research needs
• Monitor operational metrics including quality, consistency, throughput, and cost
• Improve annotation tooling, automation, and workflow efficiency
• Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation
• Analyse disagreement patterns and edge cases to improve guidelines and model performance
• Manage vendor relationships and ensure consistent quality across distributed teams
• Build reporting dashboards and communicate operational insights to leadership
• Foster a culture of continuous improvement, accountability, and operational excellence
• WE'RE LOOKING FOR SOMEONE WHO
• Has experience leading data annotation or AI evaluation teams
• Has strong operational and people management skills
• Understands AI model evaluation, LLM behavior, and modern annotation workflows
• Can design scalable processes without sacrificing quality
• Communicates clearly across technical and non-technical teams
• Thrives in fast-moving startup environments
• YOU MIGHT BE A GREAT FIT IF YOU
• Have managed annotation programs for LLMs, generative AI, or machine learning
• Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation
• Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops
• Have managed distributed or global annotation teams
• Have experience with vendor management and outsourcing operations

Будет плюсом

• Familiarity with prompt engineering and AI safety policies
• SQL, Python, or data analysis experience
• Experience building internal annotation platforms or workflow automation
• Background in linguistics, cognitive science, machine learning, or data operations
• Important note
• This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content
• You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently
• WHY WHITE CIRCLE
• Competitive salary + equity
• Paid time off in line with your local regulations
• All the hardware, tools, and services you need
• Covered subscriptions for AI agents and IDEs
• Team off-sites twice a year: we’ve recently been to the Alps and Saint-Tropez

Условия

• Work from Paris (hybrid) with a relocation package available, or work from London (note: we are currently unable to provide relocation support and medical insurance for London-based roles)

Как проходит отбор

• Intro call with HR (30 min)
• Take-home exercise
• Final conversation with our CEO (45 min)
• Please submit your application in English

Паспорт вакансии

История публикации

Откуда что взялось

Отмечено то, что вывели мы. Без пометки — значение назвал работодатель.

Почему на этом месте в выдаче

Порядок выдачи объявлен контрактом: свежесть решает между днями, полнота и зарплата — внутри дня.