Introduction

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating information retrieval. Traditional RAG systems often face challenges with query efficiency and accuracy. This article discusses two advanced RAG techniques: Adaptive-RAG and RQ-RAG, which address these limitations by classifying and refining queries.

Adaptive-RAG: Dynamic Query Complexity Management

Adaptive-RAG introduces a framework that adapts to the complexity of the query. The system employs classifiers to determine the appropriate strategy, ranging from simple to complex. Here’s how it works:

• Single-Step Approach:
• • Simple queries are answered directly after a single retrieval step.
  • Limitation: Inaccurate for complex queries.
• Multi-Step Approach:
  • Iterative document retrieval and generation steps for complex queries.
  • Limitation: Inefficient for simple queries.
• Adaptive Approach:
  • Uses classifiers to determine query complexity and select the best strategy.
  • Implemented using different tools for varying complexities.

Classifier Construction

The classifier’s construction is crucial for Adaptive-RAG. It involves:

• Dataset Construction:
  • Queries are labeled based on their complexity using public QA datasets.
  • Example: Label ‘A’ for non-retrieval, ‘B’ for single-step, and ‘C’ for multi-step.
• Training and Inference:
  • Classifier trained using cross-entropy loss.
  • During inference, the classifier determines the query complexity and selects the appropriate retrieval strategy.

RQ-RAG: Refining Queries for Enhanced Retrieval

RQ-RAG addresses the limitations of traditional RAG by refining queries before retrieval. It involves three main strategies:

• On-Demand Search: Directly responds to simple queries without unnecessary context.
• Query Decomposition: Breaks down complex queries into simpler subqueries for comprehensive retrieval.
• Query Disambiguation: Clarifies ambiguous queries to ensure accurate information retrieval.

Dataset and Training

The dataset construction for RQ-RAG involves:

• Task Pool Creation: Includes multi-turn dialogues, decomposable queries, and ambiguous queries.
• Query Refinement: Uses refined queries to retrieve information, creating a robust training dataset.
• Training Objective: Adjusts model parameters to maximize the probability of generating correct responses during iterations.

Selection Methods

RQ-RAG uses tree decoding and three selection methods to ensure optimal answers:

• PPL Based Selection: Selects answers with the lowest perplexity.
• Confidence Based Selection: Chooses results with the highest confidence.
• Ensemble Based Selection: Combines confidence scores for final selection.

Conclusion

Adaptive-RAG and RQ-RAG significantly improve the efficiency and accuracy of RAG systems by dynamically adapting retrieval strategies and refining queries. These advancements make RAG more suitable for complex and varied query scenarios, ensuring better utilization of computing resources and enhanced user experience. By implementing these advanced techniques, developers can significantly improve the performance of their RAG systems, making them more efficient and effective in handling a wide range of query complexities.

References

• https://ai.gopubby.com/advanced-rag-11-query-classification-and-refinement-2aec79f4140b
• https://blog.demir.io/advanced-rag-implementing-advanced-techniques-to-enhance-retrieval-augmented-generation-systems-0e07301e46f4