Lead Principal Site Reliability Engineer
Oracle · Posted Oct 22
Cloud infrastructure, enterprise software, database systems, and AI-powered business applications
Get a personal compatibility score
Add a resume for personal matches
About the role
This Oracle team designs, architects, and operates highly reliable and scalable infrastructure and services, covering capacity forecasting and management, incident and service lifecycle management, and automation. The work spans monitoring and observability, alert correlation and enrichment, AI/LLM-assisted incident analysis, and automated remediation with audit and recovery safeguards. The Lead Principal Site Reliability Engineer leads architecture decisions, owns capacity and reliability strategy, mentors engineers, and drives automation and observability initiatives across service teams.
What you will do
- Serves as a consultant across teams and leads the design and architecture of infrastructure and/or service, ensuring alignment with reliability and functionality standards.
- Takes full ownership of forecasting infrastructure demands and strategically responds to capacity needs, ensuring systems have sufficient resources to handle current and future workloads, anticipating resource gaps.
- Owns collaborations and serves as a decision maker with the software development team in designing and developing infrastructures that are highly reliable and scalable, removing barriers to meet deployment requirements.
- Seeks opportunities for prototyping and encourages the implementation of prototyping initiatives (e.g., testing new applications or infrastructures, assisting in onboarding), driving new approaches.
- Identifies and projects resource gaps using cost calculators, identifying alternative solutions to lower costs, when necessary.
- Recommends methods for data collection, triage, technical analysis, and redirection to maintain and optimize operations and infrastructure reliability.
- Provides guidance to others when monitoring services, maintains up-to-date knowledge of their performance, and meticulously documents their condition.
- Oversees root cause analyses for incidents and/or maintenance on assigned services (e.g., software installs, version upgrades, security updates, backup and recovery), ensuring efficient execution and preventing incident reoccurrence.
- Provides strategic, future-oriented health and performance reporting, and anticipates actions needed based on trends in data.
- May guide provisioning to support infrastructure, applications, and services.
- May leverage non-conventional approaches and encourages experimentation in decommissioning approaches (e.g., shutting down servers, removing data from databases) to remove objects that are no longer needed.
- Contributes to strategies for automation, identifying opportunities and quantifying potential benefits for their line of business.
- Reviews the development and implementation of design, automation tools, or scripts to provide solutions, gather metrics, monitor, analyze, mitigate, or remediate issues/defects within infrastructures.
Skills used in this role
What the employer is looking for
- 8+ years of hands-on experience across software development, site reliability engineering, monitoring, and automation, with demonstrated ownership of production platforms.
- Strong programming skills in Python and experience developing backend services, REST APIs, integrations, and event-driven workflows.
- Proven experience building monitoring and observability solutions that collect and analyze alerts, metrics, logs, and service context.
- Hands-on experience integrating AI/LLM capabilities into applications for incident analysis, contextual recommendations, and operational decision support.
- Ability to develop reusable agents that combine live monitoring data, historical incidents, and previous remediation actions to recommend next steps.
- Experience implementing alert correlation, deduplication, enrichment, and prioritization to reduce noise and improve incident detection.
- Strong experience developing automated remediation workflows with approval controls, execution validation, audit trails, and recovery safeguards.
- Experience integrating monitoring platforms with incident management systems, knowledge repositories, and notification or escalation services.
- Strong understanding of distributed systems, Linux, networking, databases, and application performance, with the ability to translate failure patterns into platform capabilities.
- Experience applying software engineering practices, including automated testing, code reviews, CI/CD, version control, and secure development.
- Ability to evaluate platform reliability and AI output quality, measure operational outcomes, and improve capabilities using engineer feedback.
- Excellent communication and collaboration skills, with the ability to work with monitoring teams, service owners, and engineers to onboard services and define requirements.
Preferred qualifications
- Experience with agentic AI frameworks, tool calling, retrieval-augmented generation, and vector databases.
- Knowledge of anomaly detection, time-series analysis, and predictive techniques for identifying recurring alarm patterns.
- Experience building centralized notification services, automated escalation calls, and incident communication workflows.
- Familiarity with OCI, Kubernetes, and Infrastructure-as-Code tools such as Terraform or Ansible.
- Experience with observability platforms such as OCI Observability, Prometheus, Grafana or equivalent tools.
- Exposure to Oracle Database and Exadata monitoring, performance diagnostics, and remediation automation.
- Experience using AI coding assistants to improve operational scripts and validate proposed changes.
- Knowledge of SLOs, error budgets, and incident metrics such as detection time, recovery time, and recurrence rate.
Benefits and support
- Compensation and benefits are detailed in the job posting
About Oracle
Oracle Corporation is a multinational computer technology company specializing in cloud-engineered hardware, enterprise software, and database management systems. It provides comprehensive cloud solutions such as Oracle Cloud Infrastructure (OCI) and Oracle Cloud Applications (OCA) to power digital transformations globally. Headquartered in Austin, Texas, Oracle is a major player in enterprise AI integration and database solutions.
- Industry
- Cloud Computing
- Company size
- 141000 employees
- Founded
- 1977
- Location
- Austin, Texas, USA
- Funding stage
- Public Company
Funding
Public Company · $56B raised
- 2025-10-24Post IPO Debt$18B
- 2025-02-03Conventional Debt$18B
Leadership
Lawrence J. Ellison
Chairman of the Board and Chief Technology Officer
Clay Magouyrk
Chief Executive Officer
Mike Sicilia
Chief Executive Officer
Hilary Maxson
Chief Financial Officer
Jae Evans
Global Chief Information Officer and Executive Vice President
Recent coverage
PR Newswire
Oracle Introduces Claw-Powered Fusion Agentic Applications for HR2026-10-21
Morningstar
Oracle AI Agent Studio for Industry Applications Helps Accelerate Business Outcomes2026-10-21
Oracle News
Oracle to Advance Cancer Care with New Oncology EHR2026-10-15
BioSpace
AstraZeneca and Oracle Collaborate to Explore AI and Real-World Data for Clinical Trials2026-10-20