Hybrid Search Strategy for Maximizing Llama 3 RAG Performance: The Synergy of Vector Search and Keyword Search

What is the best way to achieve optimal performance in a Llama 3-based RAG (Retrieval Augmented Generation) system? This article introduces a hybrid search strategy that combines vector search with traditional keyword search to retrieve highly relevant contexts and maximize the answer quality of Llama 3. Beyond mere theory, we guide you through practical, applicable code and configurations to upgrade your RAG system to the next level.

1. The Challenge / Context

RAG systems play a crucial role in improving the answer quality of LLMs (Large Language Models) by leveraging external knowledge sources. However, not all search methods yield the same results. Vector search excels at finding semantically similar documents but is vulnerable to specific keywords or exact phrase matching. Keyword search, on the other hand, is excellent for exact matching but can miss semantic similarities. This problem is even more pronounced in languages like Korean, where morphological analysis and inflectional changes are significant. To effectively utilize Llama 3, it's essential to understand the pros and cons of these search methods and find the optimal combination for the given situation. This goes beyond merely "searching" and becomes a core challenge for RAG systems to "understand and provide context."

2. Deep Dive: Hybrid Search

Hybrid search is a strategy that combines the strengths of vector search and keyword search to improve search accuracy. The key is not simply to combine the two search results, but to fuse them by considering the characteristics of each search method. It is generally implemented in the following ways:

  • Vector Search (Semantic Search): Searches for documents based on the semantic similarity of sentences. It uses embedding models (e.g., Sentence Transformers) to convert documents into vectors and finds the most similar documents to the query using cosine similarity, among other methods.
  • Keyword Search (Keyword Search): A traditional text-based search method that searches for documents containing specific keywords. Elasticsearch, BM25 등이 대표적인 알고리즘입니다. Morphological analyzers can be used to tokenize Korean text and remove stop words to improve search efficiency.
  • Result Fusion (Result Fusion): Combines the results of vector search and keyword search to generate the final result. At this point, weights can be assigned based on the reliability of each search method, or results can be fused in a complementary manner.

The success of hybrid search largely depends on the weight settings for each search method. The optimal weights vary depending on the characteristics of the dataset, the type of query, and the desired answer format. Therefore, it is necessary to find the optimal weights through various experiments. Additionally, the relevance of search results can be evaluated, and if necessary, additional filtering or ranking algorithms can be applied to improve the results.

3. Step-by-Step Guide / Implementation

Below is a step-by-step guide to implementing hybrid search in a Llama 3 RAG system. This example uses Langchain, Elasticsearch, and Sentence Transformers.

Step 1: 환경 설정 (Environment Setup)

Install the necessary libraries.


pip install langchain elasticsearch sentence-transformers konlpy

Step 2: Elasticsearch 설정 (Elasticsearch Setup)

Install and run Elasticsearch. If running in a local environment, using Docker is convenient.


docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.11.1

Define the schema for indexing data in Elasticsearch. We use the `nori` analyzer for Korean morphological analysis.


PUT /my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "nori_analyzer": {
          "tokenizer": "nori_tokenizer"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "text": {
        "type": "text",
        "analyzer": "nori_analyzer"
      },
      "embedding": {
        "type": "dense_vector",
        "dims": 768,
        "index": true,
        "similarity": "cosine"
      }
    }
  }
}

Step 3: 데이터 준비 (Data Preparation)

Prepare the data to be used in the RAG system. This example uses a simple list of text data.


data = [
    "Llama 3는 Meta에서 개발한 최신 LLM입니다.",
    "RAG 시스템은 LLM의 답변 품질을 향상시키는 데 사용됩니다.",
    "벡터 검색은 의미론적 유사성을 기반으로 문서를 검색합니다.",
    "Elasticsearch는 강력한 검색 엔진입니다.",
    "하이브리드 검색은 벡터 검색과 키워드 검색을 결합합니다."
]

Use Sentence Transformers to convert text data into vectors and index them in Elasticsearch.


from sentence_transformers import SentenceTransformer
from elasticsearch import Elasticsearch

model = SentenceTransformer('sentence-transformers/xlm-r-multilingual-base')
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])

index_name = "my_index"

for i, text in enumerate(data):
    embedding = model.encode(text)
    document = {
        "text": text,
        "embedding": embedding.tolist()
    }
    es.index(index=index_name, id=i, document=document)

es.indices.refresh(index=index_name)