Staff Product Manager (Evals)

Навыки

• AI-агенты
• B2B-продажи
• Коммуникация
• LLM
• Machine Learning
• Ответственность за результат
• RAG

О компании и продукте

• Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise MCP and trusted by 50% of the Fortune 500, Workato’s cloud-native architecture connects every application, data source, and process to power real-time orchestration at scale. With enterprise-grade security and continuous innovation at its core, Workato provides the trusted foundation for organizations to automate with confidence and operationalize AI across the business. To learn more, visit www.workato.com
• Ultimately, Workato believes in fostering a flexible, trust-oriented culture that empowers everyone to take full ownership of their roles . We are driven by innovation and looking for team players who want to actively build our company.
• But, we also believe in balancing productivity with self-care . That’s why we offer all of our employees a vibrant and dynamic work environment along with a multitude of benefits they can enjoy inside and outside of their work lives.
• If this sounds right up your alley, please submit an application. We look forward to getting to know you!

Задачи

• We're looking for a Staff Product Manager to own evaluations for AI agents at Workato — both the internal framework that helps our teams ship better AI features, and the customer-facing tools that let builders assess and improve the agents they create
• This is a role with a dual mandate
• Internally, you'll establish how Workato evaluates agent quality, starting with Agent Studio and expanding to other teams shipping AI capabilities
• Externally, you'll build the evaluation experience that helps business technologists understand why their agents succeed or fail — and what to do about it
• The right person for this role has actually written evals
• You've built test suites, designed evaluation criteria, and debugged agent failures in the trenches
• You know the gap between "eval theory" and "eval reality," and you can translate that practitioner knowledge into products that work for both technical teams and non-technical builders
• In this role, y ou will also be responsible to
• Define and own the evaluation framework for Workato's internal AI agent features, driving adoption across teams starting with Agent Studio
• Build the customer-facing evaluation experience — how builders test, measure, and improve agents they create on Workato
• Make hard calls about what evaluation complexity to expose versus abstract, balancing rigor with approachability
• Partner closely with the Build Experience PM to ensure evaluation is integrated into the builder journey, not bolted on
• Work with ML engineers and platform teams to ground the framework in technical reality while keeping it accessible
• Establish metrics for what "good" looks like — both for internal agent quality and for customer evaluation adoption
• Spend significant time with customers understanding where they struggle to assess agent performance and what mental models they bring

Требования

• Qualifications / Experience
• 7+ years in Product Management
• Hands-on experience writing evaluations for AI/ML systems (agents, LLMs, or similar)
• Track record of shipping technical products to both internal and external users
• Experience driving adoption of frameworks or practices across engineering teams
• Strong written and verbal communication skills
• Bachelor's degree or equivalent experience
• Practitioner depth in evaluations
• You've written evals yourself — built test suites, designed rubrics, debugged why agents underperformed
• You understand evaluation methodology not only from reading about it, but from doing it
• You have opinions about what works, what doesn't, and where current approaches fall short
• Strong product management experience
• You've shipped products, driven roadmaps, and led cross-functional teams
• You know how to translate technical capabilities into user value and write specs that don't leave details to chance
• Technical translation ability
• You can take complex evaluation concepts and make them accessible to business technologists without dumbing them down
• You understand the difference between hiding complexity and organizing it
• Internal influence skills
• You've driven adoption of frameworks, practices, or tools across teams
• You can be a credible partner to ML engineers while advocating for what internal teams actually need
• Greenfield comfort
• You've defined products from ambiguity — scoped v1s, made bets with incomplete information, and iterated based on what you learned
• You don't need an existing playbook to be effective
• B2B product sensibility
• You see enterprise conventions as problems to solve, not constraints to accept

Будет плюсом

• Experience with agent architectures, RAG systems, or LLM application development
• Background in ML engineering, solutions architecture, or technical program management before PM
• Experience building developer tools or platform products
• Familiarity with evaluation frameworks (e.g., human eval pipelines, automated benchmarks, red-teaming)
• (REQ ID: 2538)

Паспорт вакансии

История публикации

Откуда что взялось

Отмечено то, что вывели мы. Без пометки — значение назвал работодатель.

Почему на этом месте в выдаче

Порядок выдачи объявлен контрактом: свежесть решает между днями, полнота и зарплата — внутри дня.