Staff Software Engineer- Search Quality
Databricks · Posted Nov 6
Data, analytics, machine learning, and generative AI intelligence platform services
Get a personal compatibility score
Add a resume for personal matches
About the role
Databricks is democratizing Data and AI for every enterprise user, building the retrieval backbone for AI agents and an intuitive search experience for people. The team works on hybrid retrieval, ranking models, and relevance scoring so LLMs are grounded in accurate context while employees can find assets and answers through high-recall interfaces. This role owns search quality across both the AI agent and human user experiences, from traditional keyword search to semantic vector search over structured, unstructured, and real-time data.
What you will do
- As a Search Quality Engineer, you sit at the heart of this transformation. You aren't just building a search engine; you are building the contextual backbone of the entire company.
- You will own the quality of results for two distinct but deeply connected "users":
- The AI Agent: Optimizing the retrieval layer that allows LLMs to reason over data they weren’t trained on, ensuring they have the "ground truth" needed to synthesize accurate, high-stakes business actions.
- The Human User: Improving the traditional search experience so employees can find assets and answers through intuitive, high-recall interfaces.
- Hybrid Retrieval: Balancing traditional keyword-based search (for exactness) with semantic vector search (for intent).
- Dual-Optimization: Fine-tuning ranking models that satisfy both human readability and LLM-ready context.
- High-Stakes Accuracy: In a world of Agents, poor search quality leads to hallucinations. You will build the guardrails and relevance scoring that ensure our AI stays grounded in reality.
- Data Heterogeneity: Connecting the dots across structured SQL tables, unstructured docs, and real-time business metrics.
Skills used in this role
What the employer is looking for
- The IR Specialist: You know your way around Lucene/Elasticsearch, embeddings, and ranking algorithms.
- The Quality and Metrics Obsessive: You understand that "relevance" is subjective and love building evaluation frameworks (human-in-the-loop and LLM-based) to measure it. You are obsessed with relevance metrics (nDCG, MRR, Precision@K).
- The Bridge Builder: You’re excited to apply traditional Information Retrieval (IR) wisdom to the cutting-edge world of Retrieval-Augmented Generation (RAG).
Benefits and support
- Compensation and benefits are detailed in the job posting
About Databricks
Databricks is a data and artificial intelligence company that provides a unified Data Intelligence Platform built on a lakehouse architecture. The platform combines data engineering, data warehousing, machine learning, and generative AI to help organizations manage and scale their data assets natively across major cloud providers. Founded by the original creators of Apache Spark, Databricks enables thousands of enterprises worldwide to power advanced analytics, business intelligence, and autonomous AI agents.
- Industry
- Data and AI Platform Software
- Company size
- 17300+ employees
- Founded
- 2013
- Location
- San Francisco, California, USA
- Funding stage
- Private Company
Funding
Private Company · $25.2B raised
- 2026-08-13Series L$5.0B
Leadership
Co-founder and Chief Executive Officer
Co-founder and Chief Technology Officer
Co-founder and Executive Chairman
Chief Financial Officer
Co-founder and Chief Architect
Recent coverage
Databricks Newsroom
Databricks Grows >80% YoY, Surpasses $7B Revenue Run-Rate, Scales Lakebase, Genie, and Unity AI Gateway2026-08-13
Forbes
Databricks Hits $190 Billion Valuation As CEO Ali Ghodsi Claims AGI Has Already Arrived2026-08-13
Databricks Newsroom
Databricks Launches Genie One: All-New Agentic Coworker for Every Team2026-06-16
Bloomberg
Context Needed to Reach AGI, Says Databricks CEO2026-06-16