The Job Bot at Capria employs advanced Retrival-Augmented Generation (RAG) application to provide accurate and relevant information. However, challenges like “hallucination” (generating incorrect information) and citation accuracy needed to be addressed to improve the bot’s reliability and user experience. Here’s how these issues were tackled:

Solving the Hallucination Problem

To mitigate the risk of hallucination, the Job Bot utilizes a two-model LLM architecture. This architecture consists of:

• ChatGPT 4o: Acts as a filter.
• ChatGPT 4o-turbo: Generates the primary response based on context retrieved from the vector database.

The key to reducing hallucination lies in the role of ChatGPT 4o. Before any context information is passed to ChatGPT 4o-turbo, ChatGPT 4o is used to assess the relevance of the retrieved context data. The bot poses a query to ChatGPT 4o, asking it to determine if the context is directly relevant to the user’s question. Specifically, the query instructs ChatGPT 4o: “If you feel the context is relevant to the question, then only reply to this query. If not, then ignore it.” This filtering process allows ChatGPT 4o to condense and refine the context data, ensuring that only highly relevant information is passed to ChatGPT 4o-turbo for generating the main response. By narrowing down the context to the most pertinent information, the risk of ChatGPT 4o-turbo hallucinating or providing inaccurate responses is significantly reduced.

Solving Citation Problems

Enhancements were made to improve the accuracy and detail of citations provided by the Job Bot. Initially, when a user clicked on a citation, they were redirected to the entire gdrive PDF document where the referenced information was located. This was not ideal, as users had to manually search for the specific page. To improve this:

• Unique File IDs: Each document within the system is assigned a unique ID.
• Page Number Tracking: When a user clicks a citation, such as \[12\], the backend receives the unique ID of the PDF document along with the specific page number requested by the user.
• Metadata Verification: The backend checks the metadata to verify if the requested file ID exists. If the file is found, it is sent back to the user.
• PDF.js Integration: On the frontend, PDF.js is used to display the specific page of the PDF. PDF.js is a JavaScript library that renders PDFs in a web browser. By using PDF.js, the Job Bot can directly display the relevant page of the document instead of simply redirecting to the entire PDF.

Smart Caching Implementation

To further enhance the user experience and reduce latency, a smart caching system was implemented. The bot keeps track of the IDs of documents requested by users. If a user requests another page from the same document, the bot first checks if that document’s ID is already stored in the cache. If the ID is found in the cache, the bot retrieves the document from the cache instead of making a new request to the backend. This smart caching significantly decreases latency, providing faster access to subsequent pages within the same document. The user does not have to wait for the full process of finding the document to occur again. The bot simply loads the file from the cache.