QA Engineer
Chargebee · Posted Oct 16
Subscription management, recurring billing, and revenue growth infrastructure services
Get a personal compatibility score
Add a resume for personal matches
About the role
Chargebee provides billing and monetization solutions for businesses with recurring revenue models, and its AI Products team builds Chargebee Copilot and other product experiences that bring the company's AI Agents platform to real users, spanning backend services and APIs through to user-facing interfaces. The team focuses on conversational experiences and on giving users tools to configure and improve AI agent behavior. This role, an AI Quality Engineer, owns the quality of AI-powered products and agents across their lifecycle by designing evaluations, validating capabilities before release, analyzing production behavior, and driving improvements across the AI stack.
What you will do
- Design, review, and improve AI evaluation datasets, benchmarks, and regression suites, ensuring coverage reflects real customer scenarios, edge cases, and expected user behavior
- Validate new AI capabilities before they are released to customers, and continuously improve evaluations and release criteria
- Monitor AI quality in production using customer feedback, sampled conversations, telemetry, and quality metrics
- Perform root cause analysis to understand why AI interactions succeed or fail, classifying issues across prompts, retrieval, tool usage, model behavior, workflows, product knowledge, and application logic
- Identify recurring quality patterns and recommend engineering improvements
- Convert production learnings into evaluation cases to prevent future regressions
- Partner with Engineering, Product, and Support to improve prompts, agent workflows, retrieval systems, tools, and evaluation frameworks
- Define and track AI quality KPIs, and measure the impact of improvements using benchmarks and production metrics
Skills used in this role
What the employer is looking for
- Strong analytical and investigative mindset with excellent problem-solving skills
- 2+ years of experience
- Ability to identify root causes rather than symptoms
- Excellent written communication and documentation skills
- Comfort working with APIs, logs, structured data, and technical systems
- Ability to collaborate effectively with engineering teams
- Passion for building high-quality AI products
Preferred qualifications
- Experience in QA, SDET, Technical Support Engineering, Solutions Engineering, Product Analysis, or AI/ML applications
- Familiarity with LLMs, prompt engineering, RAG, MCP, AI agents, or AI evaluation frameworks
- Experience designing evaluation datasets, benchmarks, or testing strategies
- Basic programming or scripting skills (Python, SQL, JavaScript, etc.)
Benefits and support
- Health & Wellness: Robust coverage and access to programs to support your physical and mental well-being.
- Financial Security: Initiatives to help you build long-term financial stability.
- Work-Life Balance: Flexible options and resources to help you thrive both professionally and personally.
About Chargebee
Chargebee is a subscription management and recurring billing platform that automates revenue operations, invoicing, payments, and compliance for high-growth businesses. Its platform helps enterprises manage complex subscription lifecycles, usage-based pricing, and global tax configurations. Trusted by thousands of global innovators, Chargebee streamlines financial workflows from early-stage operations to multi-year enterprise contracts.
- Industry
- SaaS
- Company size
- 1000-5000 employees
- Founded
- 2011
- Location
- San Francisco, California, USA
- Funding stage
- Series H
Funding
Series H · $475M raised
- 2024-03-08Series H$5M
- 2022-02-01Series H$250M
- 2021-04-01Series G$125M
- 2020-10-01Series F$55M
- 2019-08-01Series D$14M
- 2018-03-01Series C$18M
Leadership
CEO & Co-Founder
CPO & Co-Founder
CTO & Co-Founder
Chief Financial Officer
Recent coverage
Chargebee Blog
Chargebee Named a Leader in 2026 Gartner® Magic Quadrant™ for Recurring Billing Applications for the Third Consecutive Year2026-08-18
Chargebee Blog
Chargebee and Twikey Partner to Connect Billing Directly With Bank Payments2026-07-15
Chargebee Blog
Agicap Selects Chargebee to Power Its Next Phase of Global Growth2026-03-10
Chargebee Blog
Chargebee Acquires Inai to Supercharge AI-Powered Payments Intelligence2025-09-15