Senior Software Engineer, AI Infrastructure Solutions
NVIDIA · Posted Nov 5
Graphics processing units, artificial intelligence platforms, accelerated computing, and data center solutions
Get a personal compatibility score
Add a resume for personal matches
About the role
NVIDIA's AI Infrastructure Solutions team builds scalable AI platforms and autonomous agent solutions for enterprise use. The work spans LLM-based applications, agentic workflows, inference serving, and GPU-enabled Kubernetes deployments. This senior role owns designing and operating that infrastructure, taking applications to L3 support, and building backend services and CI/CD pipelines that expose AI capabilities internally.
What you will do
- Design, build, and operate AI infrastructure solutions that power LLM-based applications and autonomous agents across the enterprise.
- Develop agentic AI workflows and multi-agent systems, including tool calling, planning, memory, retrieval-augmented generation (RAG), and orchestration using frameworks such as LangChain, LangGraph, or similar.
- Integrate and serve LLMs through APIs and inference endpoints, optimizing for latency, throughput, cost, and reliability.
- Containerize applications with Docker and deploy and scale them on Kubernetes, including Helm charts, autoscaling, and GPU-enabled workloads.
- Take applications to L3 level: own production readiness, advanced troubleshooting, root-cause analysis, performance tuning, and escalation support for complex issues beyond L1/L2.
- Define and maintain observability for AI systems, including logging, metrics, tracing, and monitoring of agent behavior, LLM quality, and system health.
- Build and maintain CI/CD pipelines for automated build, test, and deployment of AI applications and infrastructure.
- Develop high-performance backend services and RESTful APIs (Python and/or .NET Core) that expose AI capabilities to internal applications.
Skills used in this role
What the employer is looking for
- Bachelor’s or equivalent experience in Computer Science, Engineering, or a related field.
- 5+ years of software engineering experience, with 2+ years working on AI/LLM-based solutions.
- Hands-on experience building agentic AI and autonomous agent systems, including tool use, multi-agent orchestration, and RAG pipelines.
- Strong experience with LLMs and their integration (OpenAI-compatible APIs, NVIDIA NIM, Hugging Face, or similar), prompt engineering, and evaluation.
- Strong proficiency in Python; experience with C#/.NET Core and ASP.NET Core for REST API development is a plus.
- Solid experience with Docker and Kubernetes, including deploying, scaling, and troubleshooting containerized workloads in production.
- Proven experience supporting applications at L3 level, including incident management, debugging, performance optimization, and root-cause analysis.
- Experience with CI/CD tools (GitHub Actions, Azure DevOps, GitLab CI, Jenkins) and Git-based workflows.
Preferred qualifications
- Experience with GPU-accelerated workloads, NVIDIA AI Enterprise, NVIDIA NIM, Triton Inference Server, or NeMo.
- Familiarity with MCP (Model Context Protocol) and agent-to-agent communication patterns.
- Experience with LLM evaluation, guardrails, and safety frameworks.
- Contributions to open-source AI or infrastructure projects.
Benefits and support
- Compensation and benefits are detailed in the job posting
About NVIDIA
NVIDIA Corporation is a multinational technology company that designs graphics processing units (GPUs) for the gaming and professional markets, as well as system on a chip units (SoCs) for the mobile computing and automotive market. Pioneering accelerated computing, the company has become a driving engine of modern artificial intelligence, deep learning, and data center infrastructure.
- Industry
- Semiconductors
- Company size
- 42000+ employees
- Founded
- April 5, 1993
- Location
- Santa Clara, California, USA
- Funding stage
- Public Company
Funding
Public Company · $5M raised
Leadership
Founder, President and Chief Executive Officer
Executive Vice President and Chief Financial Officer
Founder and NVIDIA Fellow
Executive Vice President, Worldwide Field Operations
Recent coverage
NVIDIA Newsroom
NVIDIA Sets Conference Call for Third-Quarter Financial Results2026-10-28
NVIDIA Newsroom
NVIDIA and Oracle Expand NVIDIA Vera Rubin to Multi-Gigawatt Scale2026-10-27
The Elec
Nvidia to Showcase Latest AI Infrastructure at AI Day Seoul 2026 Starting Nov. 92026-11-02