Senior Product Manager, Multilingual Ai And Evals
Okx
Singapore
Who We Are
About The Opportunity
What You’ll Be Doing
Multilingual AI Quality Evaluation
- Enhance the quality gate of the agentic pipeline — the evaluation agent that decides whether a translation is good enough to auto-publish.
- Own the trade-off between automation coverage and risk across content tiers. When the gate gets something wrong systematically rather than as a one-off, diagnose the root cause and drive the fix back into the agent.
AI Driven Localization Testing
- Build an AI-driven product testing tool that automatically detects localization defects that translation review cannot catch — truncation, layout breakage, hardcoded strings, unlocalized images, wrong number/date/currency formats.
- Ship the MVP that scans mobile/web apps to accelerate localization auditing, then drive the long-term vision of integrating localization tests into internal product testing infrastructure, so they run automatically before features go live.
Evaluation & Annotation Infrastructure
- Build the backbone that makes quality measurable and improvable: golden datasets, annotation tooling with error classification, automated evaluation, a metrics/dashboard layer, and the feedback loop that turns human evaluations into agent fine-tuning.
- Own annotation workflow integration so linguists can annotate in a standardized, real-time way.
What We Look For In You
- Bachelor's degree or higher with 3+ years in product management. Hands-on experience building or evaluating AI agent systems (not just using AI tools) may substitute for part of the PM tenure requirement.
- Platform-building experience: You've built content platforms, testing platforms, or SaaS products — you know how to take a fuzzy internal workflow and turn it into a system others rely on. Experience with CMSes, testing infrastructure, or internal tooling counts.
- AI agent / evaluation: You've worked hands-on with AI agents, agent harnesses, and evals — especially for subjective, hard-to-measure quality problems (not just accuracy against a clean label). You can design eval harnesses and golden datasets, and read failure modes (hallucination, prompt drift, edge-case regressions) well enough to know whether a bad output is a prompt problem, a data problem, or a model problem.
- Multi-market shipping experience: You've shipped products across multiple markets or languages — you understand what breaks when a single-market assumption meets a global user base, and you're comfortable working cross-culturally with distributed teams.
- Excellent cross-functional leadership across engineering, AI teams, linguists, design, and external partners
Nice-To-Haves
- Localization/translation domain background — terminology & glossary management, TMS tools (Phrase, Smartling), internationalization (ICU/CLDR)
- Bilingual or multilingual (Chinese and/or European languages)
- Crypto, fintech, or other high-compliance / regulated domain experience
Perks & Benefits
- Competitive total compensation package
- L&D programs and education subsidy for employees' growth and development
- Various team building programs and company events
- Wellness and meal allowances
- Comprehensive healthcare schemes for employees and dependants
- More that we love to tell you along the process!
#LI-WWW#LI-ONSITE
Similar jobs
Junior Fpga Engineer
DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to ...
Manager Ii, Technical Enablement Management
We are Datadog's in-house product experts. The Technical Enablement Management (TEM) team empowers customers through technical training s...
Account Director
At Cision, we believe in empowering every individual to make an impact. Here, your voice is heard, your ideas are valued, and your unique...
Senior People Partner
Build the payment infrastructure powering global commerce.Boku connects billions of consumers to the payment methods they prefer, one mar...
Senior Account Executive, Strategic Pursuits - Banking / Social Services, Hong Kong (Based In Sin...
About UsAt Cloudflare, we are on a mission to help build a better Internet. Today the company runs one of the world’s largest networks th...
Senior Associate Account Executive, Financial Services
About AlphaSense: The world’s most sophisticated companies rely on AlphaSense to remove uncertainty from decision-making. With market int...