How Does an AI Agent Know What It May Answer?

The most common question in a first meeting is not what an AI agent costs. It is: what happens if it tells a customer something wrong? The concern is justified and it has a technical core. A language model knows nothing by itself about your prices, your delivery times or your opening hours. It produces text that sounds plausible. Whether it is true is decided by the architecture around it.

Key takeaway: An AI agent in business use does not answer from model memory but from a connected knowledge base. The method is called retrieval augmented generation: for every request the relevant company documents are retrieved first, then the model formulates the answer exclusively from them. This reduces false statements considerably but does not eliminate them. Four things matter: clean data sources, an enforced source binding, a defined route for unanswerable questions, and a test before go-live.

Where does an AI agent get its knowledge?

From two sources that must be kept strictly apart.

The first is the model itself. A language model such as GPT or Claude was trained on large volumes of text and learned language from it, plus world knowledge up to a cut-off date. About your company it knows nothing, unless you happen to be prominent enough to appear in the training material. If someone asks about your opening hours, the model has no information, only a linguistic sense of how opening hours usually look.

The second source is the knowledge base you provide. It consists of your documents: service descriptions, price lists, FAQs, manuals, data sheets, contracts. For every request the system first searches this collection, extracts the matching passages and presents them to the model together with the question. Only then does the model formulate the answer.

This method is called retrieval augmented generation, or RAG. The difference is fundamental: without RAG the model guesses. With RAG it reads and phrases.

What is a hallucination and why does it occur?

A hallucination is an answer that is linguistically flawless and factually invented. The model is not lying in the human sense. It does what it was built for: it places the most probable next word. When information is missing, it fills the gap with what seems statistically plausible.

That is precisely what makes hallucinations dangerous. A person who does not know something sounds uncertain. A language model sounds just as confident about an invented delivery time as about a correct one.

Three situations provoke false statements particularly often:

• The knowledge base does not contain the answer. The system finds nothing suitable, the model answers anyway.
• The knowledge base contradicts itself. Two documents state different deadlines, the model picks one.
• The question sits beside the documentation. The customer asks about an edge case, the material describes only the standard case.

How often does this happen in practice?

More often than many assume, but in clearly distinguishable orders of magnitude. The 2026 figures paint a clear picture:

The jump from the second-to-last to the last row is the actual lesson: RAG reduces false statements drastically as long as retrieval works. When it fails because the question finds nothing suitable, the system falls back on model guessing. The bottleneck is rarely the model, usually the knowledge base.

The business side is documented as well: according to a survey by Arthur AI, 34 percent of the companies asked had a customer-facing incident caused by a false statement from their system within one year, with substantial remediation costs in regulated industries.

How are false statements prevented?

Not by a better model, but by rules that constrain the model. Four work best in practice:

1. Enforce source binding. The agent may only state what appears in the retrieved documents. If it finds nothing, it must not phrase freely. That is a configuration decision, not a question of the model.
2. Require evidence for answers. Every statement gets an internal source reference. If it is missing, the answer is not delivered. This stays invisible to the customer and is the single most important lever for quality assurance.
3. Escalation instead of improvisation. For anything outside the knowledge base there is a defined route to a human, with the full conversation history. An agent that does not know and says so beats one that improvises.
4. Set topic boundaries. An agent for appointment booking need not answer legal questions. The narrower the remit, the less often it strays into areas without a data basis.

Which data sources work as a knowledge base?

In principle anything that exists in writing and is maintained. What matters is less the format than the reliability:

A common misconception: more material is better. The opposite holds. A small, maintained knowledge base produces better answers than a large archive with conflicting versions. Contradictions are worse than gaps, because a gap leads to escalation while a contradiction leads to a wrong answer delivered with full confidence.

Who maintains the knowledge base after go-live?

This is the question on which projects fail after succeeding technically. An agent is not a device you install and forget. If prices, deadlines or responsibilities change and the knowledge base stays put, the agent gives outdated answers for months without anyone noticing.

Three models are common: the provider maintains it as part of operations, the company maintains it through an interface, or the knowledge base is fed automatically from leading systems such as the ERP or booking system. The third is the most robust because it requires no additional discipline.

Above all, responsibility must be settled contractually. The question "who updates the knowledge base, and is that included in the price?" belongs in every first meeting, as we set out in our checklist for choosing an AI agency.

What happens when the agent does not know something?

Ideally it says so and hands over. That sounds banal but is the single most important decision in the whole project.

A well-configured agent has three exits for questions beyond its knowledge: it points to the responsible department, it records the request as a call-back, or it hands over directly to a member of staff. Which route is right depends on the channel. On the phone direct handover is usually better; in chat outside business hours, recording with a call-back.

What an agent must not do: guess, deflect or change the subject. All three destroy trust faster than an honest "I cannot tell you that, let me connect you".

How do you test this before go-live?

With a question list fixed before launch. A three-part collection has proven effective:

1. The 30 most common real customer questions. From the email inbox, phone notes or the previous contact form. These the agent must master.
2. Ten questions it must not be able to answer. Legal advice, edge cases, topics outside the remit. Here you check whether it escalates cleanly instead of improvising.
3. Five trick questions with false premises. For instance asking about a product that does not exist. A good agent contradicts the assumption, a poor one invents a description.

The third part is the most revealing and the most frequently skipped. It shows whether the agent is bound to its sources or whether, in doubt, it phrases freely after all.

How ArkeonTech handles this

All agents we build work source-bound: they answer exclusively what is held in the knowledge base and hand everything else to the team, with the full conversation history. The knowledge base is built together with the client, and maintaining it is part of ongoing operations, not an extra item.

Before every go-live the test described above runs with real customer questions, including the trick questions. The results go to the client before the agent goes live. Where the knowledge base shows gaps, they are closed or the remit is narrowed.

Frequently asked questions about AI agent knowledge bases

Can an AI agent access our internal systems? Yes, that is the standard case for custom-built agents. Common connections are CRM, ERP, calendar and ticketing. The advantage over an uploaded document collection is currency: the agent reads the state as of now, not as of the last export. It requires an interface and a settled permission model so the agent only sees what it is allowed to see.

How long does building a knowledge base take? For a defined use case usually one to two weeks, provided the material exists. The effort rarely lies in the technology but in gathering and cleaning: discarding outdated price lists, resolving contradictions, clarifying ownership. Companies that already maintain their FAQs are considerably faster.

What happens to our documents, are foreign models trained on them? With a properly set up solution, no. The documents sit in your knowledge base and are only read for each request, not fed back into the model. That no training use takes place should be contractually assured; for processing in the EU, the Article 28 GDPR processing agreement applies as well.

Do I have to prepare every document or is a folder enough? A folder works as a starting point but rarely produces good answers. A pre-selection is sensible: which materials are current, which still apply, which contradict each other? The provider handles the review, but only the company can decide what is valid.

How do I notice if the agent gives false information? Through logging. Every request and answer is recorded, and someone should spot-check in the first weeks. It is also worth watching the escalation rate: if it suddenly rises, knowledge is missing; if it drops to almost zero, the agent may be answering questions it should not.

Does the agent get better over time? Not on its own. An agent does not learn from conversations unless someone feeds the findings back into the knowledge base. That is precisely the core of ongoing maintenance: questions that led to escalation are reviewed and added as new knowledge. The escalation rate falls over the months as a result.

Conclusion

Whether an AI agent talks nonsense is not a question of trust in artificial intelligence. It is a question of architecture. An agent without a connected knowledge base will inevitably guess, because it can do nothing else. A source-bound agent with a defined escalation route and a test before go-live operates in a range that is workable for customer contact.

Anyone evaluating a quote should therefore ask less about the model in use than about where the answers come from, what happens on a knowledge gap, and who keeps the base current. Those three answers say more about quality than any benchmark.

Sources

• SQ Magazine (2026): LLM Hallucination Statistics, review of more than 40 surveys
• Arthur AI (2026): Survey on customer-facing incidents caused by false statements
• K2view (2026): RAG Hallucination, causes and countermeasures
• MDPI Electronics (2025): Evaluating Retrieval-Augmented Generation Variants, Hallucination Mitigation