Associate, AI Model Validation, RQA
BlackRock · Posted Oct 15
Investment management, asset management, risk analytics, and wealth services
Get a personal compatibility score
Add a resume for personal matches
About the role
The Risk & Quantitative Analysis (RQA) group provides independent oversight of BlackRock’s fiduciary and enterprise risks, and its Enterprise Risk team covers operational, technology, vendor, model and market risk. The AI Validation Associate performs independent reviews of AI tools and agents used across BlackRock, including agentic systems, RAG applications and other business workflows. The role challenges system design, data integrity, guardrails and performance monitoring, and works with AI Engineering, Data Science and Risk teams on validation strategies and benchmarks.
What you will do
- Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management and token usage.
- Design and implement comprehensive validation strategies and test plans tailored to each AI use case.
- Define clear success criteria and quality benchmarks to measure whether the AI solution meets all requirements and standards.
- Understand tool architecture, potential vulnerabilities and weaknesses.
- Assess conceptual soundness, methodology, and alignment with intended business use.
- Review data quality, preprocessing steps and engineering decisions for AI use cases.
- Execute functional and performance tests.
- Evaluate performance metrics, robustness, and limitations under various scenarios.
- Document validation findings, provide recommendations
- Stay current on emerging AI technologies, validation techniques, and regulatory developments.
- Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks or gaps to senior leadership and approval committees
- Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.
Skills used in this role
What the employer is looking for
- With 3-7 years’ works experience in the IT or financial industry.
- Degree or Advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
- Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
- Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).
- Excellent analytical, communication, and documentation skills.
- Ability to work independently and challenge stakeholders constructively.
- Experience with one or more of the following:
- Validating usage of large language models (LLMs) in business context
- Evaluation methodologies, benchmark development, ground-truth creation, and AI quality assessment
- AI observability, monitoring, performance measurement, and operational controls
- AI orchestration frameworks, agent architectures, prompt engineering, memory management, and context optimization
- Exposure to cloud-based AI platforms and scalable model deployment
- MCP, APIs, connectors and integrations of AI solutions with enterprise systems and data sources
- Responsible AI practices, including fairness, explainability, bias assessment, transparency, and model governance
- Fine-tuning or model customization techniques
- Security testing and risk assessment of AI systems, including prompt injection, jailbreak, and data leakage testing
Benefits and support
- strong retirement plan
- tuition reimbursement
- comprehensive healthcare
- support for working parents
- Flexible Time Off (FTO)
About BlackRock
BlackRock, Inc. is an American multinational investment management corporation and the world's largest asset manager. The firm provides institutional and individual clients with a broad range of investment, advisory, and risk management products, including its proprietary Aladdin financial technology platform. Headquartered in New York City, BlackRock manages trillions in assets across global equity, fixed income, alternative investments, and iShares exchange-traded funds (ETFs).
- Industry
- Investment Management
- Company size
- 24900 employees
- Founded
- 1988-03-09
- Location
- New York, New York, USA
- Funding stage
- Public Company
Leadership
Chairman and Chief Executive Officer
President and Director
Chief Financial Officer and Global Head of Corporate Strategy
Chief Operating Officer
Recent coverage
BeInSure
BlackRock Exceeds Q3 Expectations, Reaches $15.49 Trillion in AUM2026-10-14
BlackRock Newsroom
BlackRock Reports Second Quarter 2026 Financial Results2026-07-15
BlackRock Newsroom
BlackRock Launches $100 Million Philanthropic Initiative for Skilled Trades2026-06-15