Senior Applied AI Engineer
Microsoft · Posted Oct 20
Software, cloud computing, artificial intelligence, and hardware services
Get a personal compatibility score
Add a resume for personal matches
About the role
Engineering Operations (EngOps) drives operational excellence across the Microsoft Cloud, strengthening quality, reliability, security and customer trust by turning operational signals into AI-powered automation. Within EngOps ACES, the Agent Fleet Operations (AFO) team runs a fleet of autonomous agents that resolve customer support cases in the production customer path, backed by live-safety, evaluation/repair and control-and-learning planes. This role is a hands-on, DevOps-oriented AI engineering position building agentic resolution capabilities, evaluation frameworks, observability and safe-deployment practices so agents run reliably at cloud scale.
What you will do
- Build agentic resolution capabilities for AFO
- Build and improve agentic patterns that raise autonomous resolution rate, accuracy, and customer satisfaction, contributing to the team's technical execution.
- Deliver engineering for Agentic AI solutions
- Deliver end-to-end features across autonomous AI systems, including orchestration, reasoning, retrieval, action execution, and human escalation workflows. Take capabilities from prototype through production across support surfaces while partnering effectively across dependent engineering teams.
- Contribute to technical direction and engineering standards
- Help shape the technical foundation for agent development by implementing architectures, design patterns, evaluation frameworks, and reusable platform capabilities.
- Apply and help improve standards for prompts, tool interfaces, testing, versioning, code quality, CI/CD, and safe deployment practices (shadow, canary, progressive rollout, automated rollback) so agent behavior remains reliable, observable, and scalable.
- Build evals that improve customer confidence that our agents are trustworthy in understanding customer sentiment and intent and in validating agentic actions.
- Contribute to production health and reliability
- Help run the service with a DevOps/SRE operating model. Support availability, latency, cost per resolution, SLOs and error budgets, regression detection, incident response, and the on-call rotation for a fleet of agents acting autonomously in the customer path. Build the observability needed to explain why an agent did what it did, and drive durable fixes rather than repeated mitigations.
- Build and maintain evaluation systems
- Contribute to the evaluation framework that determines whether agents are safe and effective in production. Build and maintain golden datasets, rubrics, judge agents, frozen anchors, and offline/online evaluation pipelines. Use evaluation signals to gate releases and continuously improve quality through production feedback loops.
- Contribute to measurable business impact
- Help move key business outcomes, including support demand reduction, autonomous resolution rate, resolution quality, time-to-resolution, engineer productivity, and customer satisfaction. Ensure the capabilities you build are measurable and directed toward the highest-impact opportunities.
- Turn Customer Signals into Engineering Insight
- Build mechanisms that convert unresolved cases, customer feedback, and dissatisfaction signals into actionable engineering insights. Help ensure recurring failure patterns improve future agent capabilities and route product fixes back to the appropriate engineering teams.
- Grow your craft and share learnings
- Deepen your agentic AI and DevOps craft, and extend the team's impact by sharing proven patterns, tools, and operational learnings with your peers.
Skills used in this role
What the employer is looking for
- Bachelor's Degree in Computer Science, Engineering, or related field AND +8 years technical engineering experience in software engineering, applied AI/ML, or distributed systems, OR equivalent experience.
- 8+ years of IT experience and 3+ years of experience using Agentic AI
- Demonstrated hands-on experience building and shipping AI or machine learning systems in production.
- Demonstrated DevOps or SRE background: hands-on experience with CI/CD and release engineering, safe-deployment practices (e.g., blue/green, canary, progressive rollout, automated rollback), observability and monitoring, and participation in product...
Benefits and support
- Compensation and benefits are detailed in the job posting
About Microsoft
Microsoft Corporation is a multinational technology company that develops, licenses, and supports a wide range of software products, services, and hardware devices. Best known for its Windows operating systems, Microsoft 365 productivity suite, and Xbox gaming consoles, the company has expanded heavily into enterprise cloud computing via Azure and advanced artificial intelligence solutions. As one of the world's most valuable public enterprises, it empowers individuals and businesses globally through innovative digital transformation.
- Industry
- Software
- Company size
- 228,000+ employees
- Founded
- April 4, 1975
- Location
- Redmond, Washington, USA
- Funding stage
- Public Company
Leadership
Chairman and Chief Executive Officer
Chief Financial Officer
Vice Chair and President
Recent coverage
GeekWire
Microsoft makes 'biggest bet yet' on its own silicon, unveils new AI chip to reduce reliance on Nvidia2026-10-12
Krebs on Security
Microsoft Patches More Than 1,400 Security Flaws in Massive October Update2026-10-13
Microsoft News
Introducing the new Copilot with Home, Code and Autopilot2026-09-25
Microsoft Azure Blog
Microsoft expands Azure AI and HPC infrastructure with AMD2026-07-20