Quality Lead, YouTube Global Vendor Operations
Google · Posted Oct 5
Search engine, cloud computing, software, hardware, and artificial intelligence services
Get a personal compatibility score
Add a resume for personal matches
About the role
The YouTube Global Vendor Operations team develops practical ways to address some of the most complex business issues to keep YouTube growing, and leads the global vendor quality program that enables suppliers to deliver experiences for YouTube's consumers, creators and partners. The team works at the intersection of vendor operations and AI-driven quality assurance, using model metrics, golden labeling and human review. This role designs AI QA frameworks, sampling and arbitration protocols, and partners with training teams to remediate quality gaps.
What you will do
- Design Quality Lead (QA) Frameworks to ensure alignment with key business outcomes. Monitor AI QA model performance and accuracy using metrics such as Precision, Recall, and Human Inter-rater Agreement.
- Manage the ingestion of human-validated Golden Labeling from manual overrides to drive continuous, high-fidelity model retraining.
- Implement blind review protocols and complex multi-tiered human + AI arbitration to prevent Automation Bias.
- Design stratified sampling methodologies that prioritize high-risk or ambiguous interactions (e.g., cases where AI confidence is low) for human review.
- Partner with the training team to remediate quality gaps.
Skills used in this role
Preferred qualifications
- Experience with LLM-based evaluation frameworks and prompt engineering.
- Experience in stratified sampling to identify specific interactions for human validation, ensuring that human efforts are focused on high-signal or ambiguous cases.
- Familiarity with QA automation and sampling tools.
Benefits and support
- Compensation and benefits are detailed in the job posting
About Google
Google LLC is an American multinational technology company focusing on artificial intelligence, online advertising, search engine technology, cloud computing, computer software, quantum computing, e-commerce, and consumer electronics. As a subsidiary of Alphabet Inc., it is widely recognized for services like Google Search, YouTube, Android, and the Gemini AI ecosystem.
- Industry
- Internet
- Company size
- 180000+ employees
- Founded
- 1998-09-04
- Location
- Mountain View, California, USA
- Funding stage
- Public Company
Funding
Public Company · $25M raised
- 1999-06-07Series A$25M
Leadership
Chief Executive Officer
Senior Vice President and Chief Financial Officer
Senior Vice President and Chief Business Officer
Chief Executive Officer, Google Cloud
Recent coverage
Search Engine Journal
Google Rolls Out September 2026 Spam Update Globally2026-09-24
Android Headlines
Everything announced at Google's 2026 Made by Google event2026-08-24
TechRepublic
Google's Biggest Announcements of 2026: Gemini, Android 17, and More2026-08-03
Google Blog
At Google I/O 2026, we unveiled a vision for this era of the agentic web2026-05-19