An AI agent can retrieve the right document and still produce the wrong business outcome.
Imagine a service company with three sources of information: a public policy, an internal exceptions guide, and a customer's account record.
A customer asks whether a fee can be waived.
The agent may find relevant information in all three sources. But those sources do not carry the same authority. Some information may be outdated. Some may be restricted. And reading a policy is not the same as being authorized to change an account.
This is the problem that gets lost when teams treat business context as a collection of documents to upload.
The question is not simply whether an AI model can find relevant information.
It is whether the system can provide the right information, for the current task, within the permissions that apply.
Context is not just information
Retrieval-augmented generation, commonly called RAG, allows an AI system to retrieve information from external knowledge sources and use it when generating a response.
That helps address a genuine limitation.
A general-purpose model cannot be expected to know every company's policies, internal procedures, customer records, or current operational state.
But retrieval introduces a separate responsibility.
The system must determine which information the agent is entitled to use.
A passage can be highly relevant to a question while still being restricted to another department, employee, or customer.
A document can contain a correct answer but no longer reflect the company's current policy.
And an agent may understand a procedure without having permission to execute it.
These are not necessarily failures of language understanding.
They are failures in the system surrounding the model.
One request, three different boundaries
Consider a hypothetical AI support agent helping a customer change a subscription.
The request appears straightforward. The customer wants to understand the available options and make a change.
But the agent needs to operate within three boundaries.
1. Business identity
The system must establish which business and customer the request belongs to, and what identity or role is making the request.
Information from another business should not become available simply because it matches the search terms.
Identity must remain part of the retrieval process, not disappear once the request reaches the AI system. Okta's guidance on securing RAG workflows emphasizes carrying identity and authorization context into retrieval so that access decisions reflect the requesting user.
2. Knowledge access
The agent may need an approved subscription policy and a customer-support procedure.
It may not need access to confidential finance instructions or internal management documents.
The retrieval system should respect those distinctions before information reaches the model.
3. Authority to act
Explaining a subscription policy and modifying a billing record are different capabilities.
The agent might be permitted to explain the policy while requiring additional authorization to make a financial change.
That authorization should be enforced by the application and its tools, not left to the model to decide.
These boundaries allow an AI agent to remain useful without granting unrestricted access to business information or operations.
Better retrieval does not automatically mean better governance
Retrieval quality still matters.
Hybrid search can combine keyword and semantic signals. Reranking can help identify which passages are most relevant. Better document structure and chunking can preserve information that might otherwise be lost during retrieval.
These techniques help answer an important question:
Did the system find the information needed to complete the task?
But they do not independently answer another question:
Was the system authorized to retrieve and use that information?
A similarity score is not a permission check.
This is where permission-aware retrieval matters. The system should enforce access rules associated with the requesting identity, business and knowledge source before restricted passages can reach the model. Depending on the architecture, these controls may operate through tenant isolation, retrieval filters and authorization checks, with the application preserving those boundaries throughout the request.
A document that falls outside those permissions should not become available to the model simply because it is semantically relevant to the question.
AWS demonstrates how fine-grained document authorization can be applied during retrieval in multi-tenant RAG architectures, filtering information according to access permissions before it reaches the model.
Those permissions also need to remain effective when documents are updated, access is revoked or responsibilities change.
Reading information and taking action are separate permissions
The distinction becomes more important when an AI agent can interact with business systems. As we explored in From AI Demo to Production: What Changes When AI Starts Taking Actions?, generating a convincing response and safely executing a business operation are two different engineering challenges.
An agent that drafts an answer has one kind of responsibility.
An agent that can update customer records, send messages, approve requests or change account information has another.
Access to knowledge should not automatically grant authority to perform an action.
For example, an agent may be allowed to read a cancellation policy and explain it to a customer.
That does not necessarily mean it should be allowed to cancel an account, issue a refund or override a contractual condition.
OWASP identifies excessive agency as a security risk when AI systems have more functionality, permissions or autonomy than a task requires. Its guidance reinforces an important design principle: authorization for sensitive operations should be enforced by the underlying systems, rather than relying on the model to decide what it is allowed to do.
Some actions may be safe to automate. Others may require confirmation or human approval.
The appropriate level of oversight depends on the task's risk, reversibility and operational requirements.
The important point is that the boundary should be explicit.
What this looks like in system architecture
A practical design separates the request, identity, retrieval and action layers. This separation is central to how businesses should approach AI and automation engineering when moving from isolated AI capabilities to systems that operate within real business workflows.
An incoming request is associated with the correct business and actor.
The system establishes the relevant business, user and agent scopes.
The retrieval layer applies those access rules when selecting eligible knowledge sources and passages. Restricted content is excluded before the retrieved context is provided to the model.
The agent then uses the authorized information to prepare a response or propose an action.
If the workflow requires an external action, the application or receiving service independently verifies that the requesting identity and agent have permission to perform that operation on the specific resource before executing it.
An unauthorized operation is denied. A request requiring additional approval follows the relevant confirmation or escalation process.
If the system lacks sufficient information to answer reliably, it should clarify the request or follow an appropriate fallback path.
This architecture separates three decisions that should not be confused:
- Who is making the request?
- Which information may be used?
- Which actions may be performed?
It does not guarantee that every generated response will be correct.
It establishes controls around the information an agent may access and the operations it may perform.
The tests that matter
When evaluating this architecture, successful responses are only part of the evidence.
The system should also be tested against situations where access or authority is unavailable.
Can an agent retrieve a document belonging to another business?
What happens when an employee loses access to a knowledge collection?
Can an agent use previously retrieved information after the requesting user's access has been revoked?
Does a restricted document appear in retrieval results because it happens to match the question?
Can an agent invoke an action that its role has not been granted?
What happens when the available documents do not contain a reliable answer?
These scenarios help determine whether the system behaves correctly when it encounters a boundary.
A trustworthy AI workflow is not one that completes every request.
It is one that completes appropriate requests while respecting the limits of its knowledge and authority.
What business leaders should take away
When evaluating an AI agent, it is reasonable to ask how accurately it answers questions and how efficiently it completes tasks.
But those measures are incomplete without understanding how the system handles business context.
Who determines which information the agent can access?
How are permissions enforced during retrieval?
What can the agent do with the information it receives?
And what happens when the request exceeds its authority?
These questions may appear technical, but they determine whether an AI system can be trusted with real business information and operational responsibilities.
The goal is not to give an AI agent access to everything a company knows.
It is to give the agent the right context, under the right permissions, when it needs to do the work.