Your primary focus will be relevance evaluation and quality improvements:
LLM-based evaluation. We fine-tune our own models to assess relevance. This involves teaching the model to understand query intent, represent items from messy catalog data, and align model judgments with real user behavior.
Real-time quality in production. Reranking, filtering, signal computation. Latency is a strict requirement, so quality vs speed trade-off is constant.
Automated quality monitoring and agentic insights. Pipelines to detect degradations and find underperforming patterns. Agent-based systems that generate actionable recommendations for the product data and search configurations.
Требования
4+ years shipping production ML systems
Experience with search, information retrieval, or recommendation systems
Hands-on experience with fine-tuning, evaluation frameworks, and scaling LLM deployments
Strong Python and PyTorch. Fluency in SQL and data orchestration tools (Spark, Airflow)
Experience designing and running A/B tests to validate model impact
Excellent English communication skills
Experience collaborating in cross-functional teams (ranking, product, data engineering)
Условия
Work with smart and empathetic people who will help you grow and make a meaningful impact
Regular team offsite events to connect and collaborate
Fully remote team — choose where you live
Unlimited vacation time — we strongly encourage all of our employees to take at least 3 weeks per year
Work from home stipend! We want you to have the resources you need to set up your home office
Apple laptops provided for new employees
Training and development budget for every employee, refreshed each year
Maternity & Paternity leave for qualified employees
Base salary: $80K–$120K USD, depending on knowledge, skills, experience, and interview results
Stock options — offered in addition to the base salary
Среди клиентов Sephora, Petco, home24, Birkenstock