Azure AI Search: Why Hybrid Search Beats Pure Vector

Everyone building RAG defaults to vector search.

You embed your documents, embed the query, find the nearest neighbors, pass to the LLM. Clean mental model.

The problem: vector search is great at semantic similarity but terrible at exact matches. And a significant portion of real user queries are exact.

Where Pure Vector Search Fails

Try vector search against these queries:

• "What is the error code 0xC0190034?"
• "Contract number INV-2024-00847"
• "What does the acronym SNAT stand for?"

Vector embeddings work by capturing semantic meaning. The embedding for "SNAT" is nearly identical to embeddings for "source NAT", "network address translation", and dozens of related concepts.

That's good for general understanding. It's bad when you need the document that contains the exact string "SNAT" — not a semantically similar document about NAT in general.

Keyword search catches this trivially. BM25 scores exact term matches heavily. A document containing "SNAT" repeatedly scores much higher than one that discusses NAT concepts without using that term.

How Hybrid Search Works (RRF)

Azure AI Search implements hybrid search using Reciprocal Rank Fusion (RRF).

Each search mode (vector, keyword) produces its own ranked list. RRF combines them:

RRF score = Σ 1 / (rank_i + k)

Where k is typically 60, and rank_i is the document's position in each result list.

Example:

Doc C wins because it ranks high in both modes. Doc B wins despite low vector rank because it's the top keyword match. Doc A loses despite excellent vector rank because keyword scoring pulls it down.

The result: documents that are relevant in multiple dimensions consistently beat documents that are only relevant in one.

Setting Up Hybrid Search on Azure AI Search

Step 1: Create an index with both vector and text fields

from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndex, SearchField, SearchFieldDataType,
    VectorSearch, HnswAlgorithmConfiguration,
    VectorSearchProfile, SemanticConfiguration,
    SemanticSearch, SemanticPrioritizedFields, SemanticField,
    SimpleField, SearchableField
)

fields = [
    SimpleField(name="id", type=SearchFieldDataType.String, key=True),
    SearchableField(name="content", type=SearchFieldDataType.String),
    SearchableField(name="title", type=SearchFieldDataType.String),
    SimpleField(name="source", type=SearchFieldDataType.String, filterable=True),
    SearchField(
        name="content_vector",
        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
        searchable=True,
        vector_search_dimensions=3072,   # text-embedding-3-large
        vector_search_profile_name="hnsw-profile"
    ),
]

vector_search = VectorSearch(
    algorithms=[HnswAlgorithmConfiguration(name="hnsw-algo")],
    profiles=[VectorSearchProfile(name="hnsw-profile", algorithm_configuration_name="hnsw-algo")]
)

semantic_config = SemanticConfiguration(
    name="semantic-config",
    prioritized_fields=SemanticPrioritizedFields(
        content_fields=[SemanticField(field_name="content")],
        title_field=SemanticField(field_name="title")
    )
)

index = SearchIndex(
    name="knowledge-base",
    fields=fields,
    vector_search=vector_search,
    semantic_search=SemanticSearch(configurations=[semantic_config])
)

client = SearchIndexClient(endpoint=endpoint, credential=credential)
client.create_or_update_index(index)

Step 2: Run hybrid search

from azure.search.documents import SearchClient
from azure.search.documents.models import VectorizedQuery

def hybrid_search(query: str, top_k: int = 5) -> list[dict]:
    search_client = SearchClient(endpoint=endpoint, index_name="knowledge-base", credential=credential)
    
    # Embed the query
    query_embedding = embed(query)  # your embedding function
    
    vector_query = VectorizedQuery(
        vector=query_embedding,
        k_nearest_neighbors=top_k * 2,   # over-fetch, RRF narrows down
        fields="content_vector"
    )
    
    results = search_client.search(
        search_text=query,               # keyword search
        vector_queries=[vector_query],   # vector search
        query_type="semantic",           # semantic re-ranking (optional but recommended)
        semantic_configuration_name="semantic-config",
        top=top_k,
        select=["id", "content", "title", "source"]
    )
    
    return [
        {
            "id": r["id"],
            "content": r["content"],
            "title": r["title"],
            "source": r["source"],
            "score": r["@search.score"],
            "reranker_score": r.get("@search.reranker_score"),
        }
        for r in results
    ]

The query_type="semantic" enables an additional re-ranking pass using Azure's semantic model. This costs extra (semantic queries are billed separately) but improves result quality for conversational queries.

When to Weight Vector vs Keyword More Heavily

The default RRF treats both signals equally. You can adjust the weighting:

# In your search config, set weights via the vector query weight
vector_query = VectorizedQuery(
    vector=query_embedding,
    k_nearest_neighbors=top_k,
    fields="content_vector",
    weight=0.3   # reduce vector influence; default is 1.0
)
# keyword search weight defaults to 1.0

Increase vector weight when:

• Users ask conceptual or conversational questions
• Your content uses varied terminology for the same concepts
• Multilingual search (embeddings bridge language gaps)

Increase keyword weight when:

• Users search for specific codes, IDs, or product names
• Content is highly technical with defined terminology
• Users expect exact match behavior (like a database query)

Practical Comparison

We ran the same query set against all three modes on an Azure documentation corpus (12,000 chunks):

Hybrid consistently beats either alone. Vector-only fails badly on exact lookups. Keyword-only fails on conceptual questions. Hybrid gets most of both right.

Cost Considerations

Hybrid search on Azure AI Search:

• Standard tier: ~$0.25/hour per search unit (starts ~$180/month)
• Semantic queries: $1 per 1,000 queries (on top of base cost)
• Vector storage: Standard tier supports up to 35GB of vectors per search unit

For most RAG systems with under 50K documents, one Standard search unit is sufficient. Add semantic queries for user-facing search where quality matters most. Batch/background processing can skip semantic re-ranking to save cost.

The Bottom Line

Pure vector search is a fine starting point. Switch to hybrid the moment you see users complaining about "search not finding the right thing" — it almost always means their exact-match queries are failing.

Azure AI Search makes hybrid trivially easy to add. It's one parameter change in your existing search call. The performance improvement is consistent and measurable.