Senior System Software Engineer - Local AI Automation
NVIDIA · Posted Oct 12
Graphics processing units, artificial intelligence platforms, accelerated computing, and data center solutions
Get a personal compatibility score
Add a resume for personal matches
About the role
The Local AI Automation team builds and maintains robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models. The team works across inference backends including Llama.cpp, Ollama, PyTorch, vLLM, WinML, and TRT-RTX, along with model repository management, data processing, and system performance analysis. This role designs and maintains automation infrastructure for build, integration, deployment, and qualification of AI workloads, and collaborates with internal Local AI developers on instrumentation, diagnostics, and debugging.
What you will do
- Design, implement, and maintain robust system software and infrastructure to efficiently run various AI workloads through different inference backends such as Llama.cpp, Ollama, PyTorch, vLLM, WinML, TRT-RTX, and others.
- Scope out system requirements for deploying various AI applications, benchmarks, and models in automation, and develop solutions to measure correctness, functionality, reliability, and overall system performance.
- Develop infrastructure to download AI models from various sources, efficiently manage a local model repository, and create automated synchronization mechanisms to keep the repository up to date.
- Analyze large datasets and system workload metrics, implement data processing infrastructure to transform raw data into actionable information for developers, and design useful visualizations.
- Collaborate with internal Local AI developers to identify and implement system-level instrumentation, diagnostics, and debugging capabilities that help quickly debug and isolate issues.
- Design, implement, and maintain robust infrastructure for automated build, integration, deployment, and qualification across different inference frameworks and environments.
- Understand existing infrastructure and system software architecture, debug issues in current frameworks, identify performance bottlenecks, and devise solutions to enhance system efficiency and scalability.
Skills used in this role
What the employer is looking for
- 5+ years of experience with a B.Tech or higher degree in Computer Science, Information Technology, Software Engineering, or a related field.
- Strong analytical, systems debugging, and problem-solving abilities, with the capacity to multitask effectively in a dynamic environment.
- 5+ years of software development experience with strong programming skills in C/C++, C#, Java, or another systems programming language, along with exposure to at least one scripting language such as Python.
- Experience with system APIs, multithreaded software, process and resource management, debugging, or performance analysis.
- Familiarity with databases and SQL, source control systems, and CI/CD infrastructure such as Git, Perforce, and Jenkins.
- Outstanding written and oral communication skills, enabling effective collaboration with management and engineering teams.
Preferred qualifications
- Experience building robust system software, runtime infrastructure, or scalable backend components.
- Experience with system profiling, performance optimization, debugging, and telemetry/observability infrastructure, including analysis of Nsight Systems traces to understand system state and kernel performance.
- Hands-on background with Linux/Windows system programming, Git CI/CD, databases/SQL, Kubernetes, Docker, and observability tools such as Grafana or Kibana.
- Hands-on experience integrating or debugging inference frameworks such as Llama.cpp, Ollama, PyTorch, vLLM, WinML, or TRT-RTX.
- Experience understanding CPU/GPU resource utilization, memory management, concurrency, and performance characteristics of AI workloads.
Benefits and support
- Compensation and benefits are detailed in the job posting
About NVIDIA
NVIDIA Corporation is a multinational technology company that designs graphics processing units (GPUs) for the gaming and professional markets, as well as system on a chip units (SoCs) for the mobile computing and automotive market. Pioneering accelerated computing, the company has become a driving engine of modern artificial intelligence, deep learning, and data center infrastructure.
- Industry
- Semiconductors
- Company size
- 42000+ employees
- Founded
- April 5, 1993
- Location
- Santa Clara, California, USA
- Funding stage
- Public Company
Funding
Public Company · $5M raised
Leadership
Founder, President and Chief Executive Officer
Executive Vice President and Chief Financial Officer
Founder and NVIDIA Fellow
Executive Vice President, Worldwide Field Operations
Recent coverage
NVIDIA Newsroom
NVIDIA Announces $1 Billion Commitment to Advance U.S. Super Intelligence Research and Quantum Leadership2026-10-08
NVIDIA Newsroom
NVIDIA Board of Directors Authorizes an Additional $150 Billion for Share Repurchases2026-10-03
NVIDIA Newsroom
NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment2026-09-28
NVIDIA Newsroom
NVIDIA Reports Second Quarter Fiscal 2027 Financial Results with Record Revenue2026-08-26