Senior System Software Engineer - Local AI
NVIDIA · Posted Oct 19
Graphics processing units, artificial intelligence platforms, accelerated computing, and data center solutions
Get a personal compatibility score
Add a resume for personal matches
About the role
The Local AI Inference team at NVIDIA works on making the shift from passive AI assistance to autonomous, always-on workflows seamless, efficient, and secure for millions of users globally. The team builds system-level software and infrastructure for deploying, benchmarking, and qualifying AI applications and models across Linux and Windows systems powered by RTX and DGX-class GPUs. This role leads development of that infrastructure — CI, model management services, backend data pipelines, dashboards, server infrastructure, and autonomous AI agents — in partnership with inference engineers, model developers, and AI application teams.
What you will do
- Partner across organizations with benchmarking teams, inference teams, and AI application developers to define requirements, design, and maintain shared infrastructure for deploying, benchmarking, and qualifying AI workloads across infernece backends such as llama.cpp, Ollama, PyTorch, vLLM, and WinML.
- Build Git-based CI infrastructure for automated builds, integration testing, and regression detection across hardware platforms, operating systems, and runtime configurations.
- Develop model management services to download, version, cache, and synchronize checkpoints and dependencies for reproducible execution.
- Design databases, backend services, and data pipelines to store and query model metadata, benchmark results, system telemetry, and execution artifacts.
- Create dashboards and reports that help engineers compare inference configurations, identify regressions, and investigate performance bottlenecks.
- Develop and operate server infrastructure, including environment provisioning, workload scheduling, resource management, and failure recovery.
- Develop autonomous AI agents for infrastructure management, workload orchestration, monitoring, and diagnostics, with appropriate access controls and execution safeguards.
- Maintain Git repositories and companion websites, keeping code, documentation, model recommendations, and evaluation results current.
- Collaborate with inference developers to implement instrumentation and debugging capabilities, resolve system issues, and improve infrastructure efficiency and scalability.
Skills used in this role
What the employer is looking for
- B.Tech or higher degree in Computer Science, Computer Engineering, Information Technology, Software Engineering, or a related field, or equivalent experience.
- 5+ years of relevant software development, with strong programming skills in Python, C/C++, C#, or another systems programming language.
- Experience developing system software, backend services, or infrastructure supporting engineering development.
- Understanding of system APIs, multithreading, process and resource management, and systems debugging.
- Hands-on with Git-based CI, databases, SQL, and data modeling.
- Strong analytical and problem-solving skills, with the ability to independently own projects and manage competing priorities.
- Excellent written and verbal communication skills for collaboration across engineering teams.
Preferred qualifications
- Track record of integrating or debugging AI inference frameworks and building model evaluation or benchmarking platforms
- Proficiency working with AI agents, including tool integration and evaluation of autonomous infrastructure workflows.
- Hands-on expertise in developing telemetry and observability systems using tools such as Grafana or Kibana.
- Demonstrated ability to publish technical documentation, model recommendations, or benchmark results through companion websites.
Benefits and support
- Compensation and benefits are detailed in the job posting
About NVIDIA
NVIDIA Corporation is a multinational technology company that designs graphics processing units (GPUs) for the gaming and professional markets, as well as system on a chip units (SoCs) for the mobile computing and automotive market. Pioneering accelerated computing, the company has become a driving engine of modern artificial intelligence, deep learning, and data center infrastructure.
- Industry
- Semiconductors
- Company size
- 42000+ employees
- Founded
- April 5, 1993
- Location
- Santa Clara, California, USA
- Funding stage
- Public Company
Funding
Public Company · $5M raised
Leadership
Founder, President and Chief Executive Officer
Executive Vice President and Chief Financial Officer
Founder and NVIDIA Fellow
Executive Vice President, Worldwide Field Operations
Recent coverage
NVIDIA Newsroom
NVIDIA Commits $1 Billion to Advance US Science Over the Next Five Years2026-10-08
NVIDIA Blog
Adobe and NVIDIA Expand Collaboration to Speed Agentic Creation2026-10-15
Cisco News
NVIDIA and Cisco Introduce New Pluggable Optics Form Factor to Scale AI Factories2026-10-12
Sustainability Magazine
The Earthshot Prize Welcomes NVIDIA as a Founding Partner2026-10-16