Introduction
The “R” in “RAG” stands for the Retrieval process, which is the step where the system fetches relevant information from external sources such as relational databases or PDF file to provide context for the AI model.
This makes it clear that Retrieval is the first stage in the pipeline, and those sources (query, database, memory) are examples of where the chatbot looks for information before moving on to Augmentation and Generation.
It is important to enumerate the types of retrieval sources because each source will trigger a different flow of processing. Structured data from a relational database may require one approach, while unstructured text from a PDF file demands another. Cached memory, like Redis, operates differently again, often optimized for speed and session context. Each of these sources is handled by different agents or modules within the chatbot, and they apply distinct strategies for chunking and embedding the information before it is passed along to the generation stage. By clearly identifying the retrieval sources, we can design retrieval flows that are tailored to the nature of the data, ensuring accuracy, efficiency, and relevance in the chatbot’s responses.
In this article, we are going to enumerate the sources of the Retrieval process which can be found inside a Chatbot.
Types of Retrieval Sources
1. User Query
The user query is the very first retrieval source of an AI chatbot because it sets the direction for everything that follows. It’s the customer’s request, expressed in natural language, that tells the AI chatbot what kind of information to look for.
For example, in an e‑shop chatbot, a shopper might ask:
Can you recommend a Christmas gift for my daughter?
That query immediately becomes the retrieval starting point. The chatbot interprets the request and searches through its available sources – such as product catalogs, seasonal promotions, or gift guides – to find items that match the intent.
In this case, the user query is not just a question; it’s the anchor that guides the chatbot toward relevant recommendations. By beginning with the query, the chatbot ensures that the retrieval process is aligned with the customer’s needs, leading to suggestions like toys, books, or personalized gifts that make sense for a daughter at Christmas.
2. Unstructured Document Source (Uploaded by the User)
When a user uploads a file directly into the chat window, that file becomes an Unstructured Document Source for the chatbot’s retrieval process. These sources are important because they contain free‑form information that must be interpreted differently from structured databases or cached memory. Uploaded via the chat bubble, they are treated as retrieval inputs that the chatbot can parse, analyze, and use to provide contextually relevant answers. Each file type requires a different handling flow since formatting and content vary widely. For Example:
.TXT
A .txt plain text file is the most common type of unstructured document source in a chatbot’s retrieval process. It contains simple, unformatted text with no typesetting, tables, or embedded images, making it straightforward for the chatbot to read and interpret.
.DOC (.DOCX)
A .docx File is an Unstructured Document Source in the chatbot’s retrieval process. Unlike plain text .txt File, it often contains typesetting and formatting such as bold, italics, and styles, as well as a hierarchical context window with headings (e.g., Heading 1, Heading 2) that provide structure to the content. It can also include tables, footnotes, and other embedded elements that add layers of meaning.
Spreadsheet
A spreadsheet file (such as .xls, .csv, or .tsv) is a common retrieval source for a chatbot because it organizes information into data tables. These files use delimiters (like commas in .csv) to separate values, and they are structured into columns and rows that make the data easy to reference. Spreadsheets can also contain multiple sheets, each with its own dataset, and rely on column names and row labels to provide context. When uploaded via the chat bubble, the chatbot treats the spreadsheet as a retrieval input, parsing the tabular data to answer queries.
For example, a user might upload a .csv file listing product types and then ask: “Are any of these suitable as a Christmas gift for my daughter?” The chatbot would scan the rows and columns, identify relevant categories such as toys or books, and generate recommendations based on the structured data.
This illustrates why spreadsheets are powerful retrieval sources: their tabular format allows the chatbot to quickly locate, filter, and interpret information in response to user queries.
A .pdf file is a versatile retrieval source because it can contain almost every type of content found in other document formats. It includes plain text, hierarchical structures like headings and sections, typesetting and styled formatting, as well as tables, images, and footnotes. In other words, a PDF can combine the features of a .txt file, a .docx document, and even a spreadsheet, all within one file.
3. Hot Chat Memory
Hot chat memory refers to the last 5 to 10 turns in the same chat session (under the same session ID) and acts as a retrieval source because it captures the immediate conversational context, allowing the chatbot to stay coherent and relevant without needing to re‑ask the user for details. It stores the short‑term dialogue history so the chatbot can recall what was just discussed.
For example, if a father is chatting with an e‑shop chatbot about buying a Christmas gift for his daughter, he might first mention her age and interests (such as liking books and puzzles) and later ask, “Can you recommend something suitable from the product list I mentioned before?” The chatbot uses hot chat memory to remember those earlier details and tailor its recommendations accordingly.
Without Hot Chat Memory, the AI chatbot would lose track of context and give generic suggestions, but with it, the system can connect the dots across multiple turns in the same session.
4. Cold Chat Memory
Cold chat memory refers to the dialogue history after the first 10 turns in the same chat session (under the same session ID). Unlike hot chat memory, which captures immediate context, cold chat memory provides a longer‑term record of the conversation, allowing the chatbot to recall details from earlier exchanges even after the dialogue has moved on. For example, if a father has been chatting with an e‑shop chatbot about buying a Christmas gift for his daughter, the chatbot may need to remember information mentioned much earlier in the session — such as her age, favorite hobbies, or budget preferences — to ensure that later recommendations remain consistent and personalized. Cold chat memory is especially useful when conversations are extended or complex, as it prevents the chatbot from losing track of important background details.
5. Cold Chat History
Cold chat history refers to any chat turns from previous sessions (i.e., with a different session ID), making it distinct from hot chat memory, which only covers the last 5–10 turns in the current session and disappears once the session ends. Cold chat history is stored permanently in a database, often a graph database, so it can be accessed across sessions to provide continuity and personalization over time. For example, if a father chats with an e‑shop chatbot about buying Christmas gifts for his daughter today and then returns the next day in a new session, cold chat history ensures the chatbot still remembers details like her age and hobbies. By storing this information in a relational or graph database, the system can model relationships between entities, intents, and past queries, enabling more dynamic and context‑aware retrieval that persists beyond a single session.
6. Relational Database
A relational database serves as a structured retrieval source, storing information in tables with rows and columns that can be queried using relationships between data. In the father‑buying‑Christmas‑gift example, the chatbot may need to access two relational databases:
- Product Database: This contains product categories and product details. For instance, the chatbot could query the database to find items in the “Books” or “Puzzles” category that match the daughter’s interests.
- CRM Database: This holds client information such as membership level, loyalty points, and order history. By retrieving this data, the chatbot can personalize recommendations — for example, suggesting products eligible for discounts based on the father’s membership tier or avoiding items he has already purchased.
Together, these relational databases allow the chatbot to combine structured product data with personalized client data, ensuring recommendations are both relevant and tailored.
7. Knowledge Base
A knowledge base acts as a structured retrieval source that contains domain‑specific or company‑specific information. Unlike chat memory or product databases, a knowledge base is designed to store reference material such as company history, business hours, FAQs, or domain expertise. This information can often be found on a company’s website or embedded in documents like PDFs hosted online. For example, if a father is chatting with an e‑shop chatbot about buying a Christmas gift for his daughter, the chatbot may need to pull product details from a relational database but also consult the company’s knowledge base to answer questions like: “What are your holiday delivery hours?” or “Does your company have a return policy for seasonal gifts?” Similarly, if the company has published a PDF guide on recommended gifts or product usage, the chatbot can retrieve that domain‑specific knowledge to enrich its response.
8. Human Plus Business Rules
Human + rules act as a deterministic retrieval source, meaning they are explicitly defined by business owners and avoid probabilistic reasoning or hallucination. These rules encode business know‑how and can be easily modified to reflect promotions, policies, or operational constraints. Unlike knowledge bases or chat memories, Human + rules are not about storing information but about enforcing clear, rule‑driven logic during retrieval.
For example, in the father‑buying‑Christmas‑gift scenario, the chatbot may retrieve product details from a relational database and client information from a CRM, but it also needs to apply business rules such as
“Buy one, get one free during the Christmas period.”
This ensures the chatbot’s recommendation is not only relevant to the daughter’s interests but also aligned with current promotions. Because these rules are deterministic, the chatbot will always apply them consistently, avoiding hallucinations or probabilistic guesses.
By combining structured data sources (like product and CRM databases) with Human + rules, the chatbot delivers accurate, compliant, and business‑aligned answers that reflect both factual information and company know‑how.
9. Website Scraping
Website scraping is a retrieval source that extracts structured or semi‑structured data directly from external websites. It complements internal sources like product databases, CRM systems, or knowledge bases by filling gaps in information.
In the father‑buying‑Christmas‑gift example, the chatbot may already know product categories and client details from relational databases, but it can deploy a scraper to compare prices with competitor websites. This ensures the father gets the best deal and sees promotions beyond the company’s own catalog.
Scraping can also retrieve domain‑specific details missed in other sources — for instance, competitor delivery times, seasonal offers, or product availability.
10. API Store
An API store functions as a retrieval source similar to website scraping, but it is far more efficient and reliable because it fetches structured data directly from official APIs rather than parsing raw web pages. This reduces errors, avoids inconsistencies, and ensures faster access to up‑to‑date information.
In the father‑buying‑Christmas‑gift example, the chatbot may already use relational databases for product details and CRM data for client information, but if the father wants to know the latest NBA match results to decide which NBA team T‑shirt to buy for his daughter, the chatbot can query a sports API. This guarantees accurate, real‑time results without the overhead of scraping competitor sites.
API stores also serve as a complementary retrieval source, filling gaps left by other sources. For instance, if the company’s knowledge base or product database doesn’t include live sports scores, weather updates, or external promotions, APIs can provide that missing context seamlessly.
Conclusion
When an AI chatbot integrates all these retrieval sources — unstructured data, user queries, relational databases, hot and cold chat memory, cold chat history, CRM, product detail databases, company and domain‑specific knowledge bases, business rules, web scrapers, and API stores — it essentially builds a Human‑Plus intelligence layer. This means the chatbot can operate at a level comparable to a very high IQ human assistant: it remembers context across sessions, applies deterministic business rules without error, pulls structured and unstructured knowledge seamlessly, and augments its reasoning with real‑time external data.
In practice, such a chatbot can act like a Sales specialist consultant — whether as a shopping assistant who knows your preferences and promotions, a customer service agent who recalls your history, or a domain expert who can combine company policies with external intelligence. The “IQ level” here is not about raw problem‑solving alone, but about practical intelligence: the ability to connect multiple knowledge sources, avoid hallucinations, and deliver precise, personalized, and context‑aware answers that feel as sharp and reliable as a well‑trained human professional.
Leave a Reply