In last week’s article, we explored how GAIN Bot leverages a structured flow and smart vectorization techniques to fetch accurate records. However, much of the groundwork and experimentation happened well before reaching that stage. In this article, we dive into the earlier iterations of the bot and how it evolved into the intelligent system it is today.

Early Days – Basic Query Search

Initially, when a user asked a question to GAIN Bot, the system would take the raw query, append “Capria VC” to it, and perform an organic Google search using the SERP API to extract the top 5 links. The intention behind this method was to filter and retrieve webpages specifically from Capria Venture’s ecosystem.

However, this simplistic approach did not access most of the web pages As a result, the bot’s response accuracy hovered around 60 percent, which was not ideal for internal use.

Shift to Entity-Based Search

To improve precision, the team introduced entity-based search.

In this approach:

• The bot first extracted key entities from the user’s query.
• It then constructed a more refined search by appending “Capria VC” to these entities.
• These entities were then split by white spaces to form smaller, broken components, which were used to locate sub-links or nested pages within broader sources.
• This enhanced the contextual relevance of the retrieved results.

This method improved the bot’s performance significantly, boosting answer accuracy to over 80 percent. Additionally, the bot started analyzing sub-links rather than relying solely on top-level URLs, enabling more precise content discovery.

The Latency Challenge

While accuracy improved, it introduced a new issue: latency. To gather correct context, the bot fetched a lot of links using SERP API, scraped the content of each link, and then, GPT processing big chunks of text used to take more time, cost and hallucinations. This increased both token consumption and response time. For end-users, this delay became about a minute or more in the worst cases, especially for more complex queries.

Addressing Authenticity and Response Time

Another critical question began to surface: how do we know if the answer provided is accurate?

This brought attention to two key concerns—authenticity and response verification. It became clear that simply fetching data was not enough; the system needed to validate the relevance and correctness of the answer before presenting it. Also, adding more links increased the data size, which led to many hallucinations, and increasing the number of links increased processing time.

To resolve both latency and authenticity challenges, the team introduced a redesigned architecture that uses parallel processing with threads. This allowed various components of the system to operate simultaneously, improving both performance, reliability and latency.

Enhanced Architecture – Parallel Answer Retrieval

When a user submits a query, the bot performs parallel searches:

• It retrieves internal content from GAIN Pages using vector search, which is then interpreted by a language model.
• In parallel, it scrapes external links and relevant sub-links using a threaded approach, allowing it to process multiple sources at once.

The responses from both internal and external sources are stored in a consolidated list. A final LLM call is then made to analyze this filtered list and generate a cohesive and accurate answer for the user.

If no relevant data is found in either source, the system falls back to performing an organic Google search.

This restructured approach successfully brought down the response time to under 10 seconds. It also improved answer authenticity, as multiple validation stages involving LLMs help ensure that the final response is based on relevant, verified content.

See you next time!