Part 2: Inside Atom Ai – How Retrieval Mechanisms Shape AI Responses

Unpacking the "R" in RAG: Retrieval

RAG refers to retrieval-augmented generation, a framework that combines both retrieval-based and generation-based approaches in relation to natural language processing (NLP) tasks.

In a RAG application, "retrieval" refers to the process of gathering relevant context from a selected set of data sources — context that is passed to a large language model (LLM) that then generates a response to a user prompt. 

LLMs operate by mimicking how humans would typically write a paper, respond to an email or give a presentation. As humans, part of our info-gathering process often involves performing online research, reviewing documents, or conversing with a colleague to get the right information. While RAG applications are evolving quickly and have surpassed the data-gathering capabilities of humans in many respects, LLMs are still inherently limited by their training data. To answer questions based on an organization's proprietary data, they must have some way of searching for this information or "doing research" like humans.

This is where the significance of the LLM's data sources comes into play.

Simply put, RAG applications are only as good as their data sources. Not only must RAG applications be equipped with the right data sources for the use case in question, but the data within each source must be well-suited for use by the LLM. 

What's more, each data source may require its own fine-tuned retrieval process to effectively fetch the appropriate data.

Applying this lesson to Atom Ai

For Atom Ai (formerly WWT GPT), our internally developed GenAI chatbot, we took care to select three data sources based on their importance to our business use case and based on the quality of the data. The data sources for Atom include:

• wwt.com: Every page from WWT's website plus video transcripts from all embedded video content.
• Advanced Technology Center (ATC) proof-of-concept documentation (POCDOC): Highly detailed and technical information about certain past engagements from WWT's ATC lab environment.
• ServiceNow records: A table containing metadata for every ATC project.

Each of these data sources required special considerations for preprocessing, storage, security and role-based access control (RBAC). For instance, the wwt.com and POCDOC sources are large sets of textual documents, whereas the ServiceNow data is a SQL database table. As such, the retrieval process is unique across the two formats. We will first discuss the document retrieval process for the textual sources, followed by the "text-to-SQL" retrieval for ServiceNow.