Klyro
Back to blog
November 16, 2026

Knowledge base for AI: how to keep it current

Explains knowledge base structure for AI agents, provides guidance on document format, granularity of information, routine updating after a change of policy or product, response validation process, strategy of
A dedicated athlete competes in a marathon using a racing wheelchair on city streets.
Photo by Yusuf Miah on Pexels

Why keeping the knowledge base up to date is the most important ongoing work in AI

Companies that implement AI agents in service usually concentrate effort, time, and budget on choosing the model, configuring the platform, and initial tests. After the go-live, the operation starts working and the team returns to taking care of other priorities. Weeks later, signs begin to appear that something isn't working: customers complain about wrong answers, conversations are frequently transferred to human attendants, and satisfaction drops for no apparent reason.

The problem isn't in the AI model. It's in the outdated, incomplete, or poorly structured knowledge base that fuels the agent's responses. Unlike a chatbot based on fixed flows, where each response is programmed manually, generative AI seeks information in documents, pages, and records provided by the company. If those materials are outdated, contradictory, or missing, the AI responds based on what it finds—and that could mean guiding the customer with an old policy, a price that changed, or a process that no longer exists.

Keeping the knowledge base up to date is not a one-off implementation task. It is a continuous operational routine, as important as reviewing service queues, monitoring satisfaction metrics, or adjusting campaigns. This article guides how to structure documents, define those responsible for updating, identify content gaps, test responses before releasing to clients, and measure the quality of automation over time.

How to structure documents so that AI can use

Generative AI is unable to infer missing context or fill information gaps. It searches for a match between the customer's question and the content available in the database. If the document is generic, incomplete, or written for internal use, the response will be inaccurate or incomplete.

The recommended structure for each knowledge base document includes:

  • Descriptive title that reflects the customer's actual question, not the internal name of the process. Example: “How to request a refund for a faulty product” instead of “Business Policy 2.3 - returns”.
  • Objective response in the first lines, followed by detailing only when necessary. AI tends to prioritize the first sentences found in the content.
  • Explicit eligibility criteria when the answer depends on conditions. Example: “This procedure only applies to purchases made within the last 30 days.”
  • Numbered step by step for processes that involve multiple steps, avoiding single-paragraph descriptions that make it difficult to extract clear instructions.
  • Practical examples that illustrate common situations, especially with regard to exception policies, deadlines, or amounts.
  • Date of the last updated and responsible for the review, to facilitate periodic audits and prevent outdated content from remaining active.

Documents written for domestic consumption—with acronyms, references to legacy systems, or technical language—must be rewritten or supplemented with customer-facing versions before feeding into the AI base. The BRANDplatform [allows training AI agents with the company's documentation, website, and knowledge base] (URL0), but the quality of the response depends directly on the clarity and completeness of the content provided.

When to update: signs that the base needs revision

The knowledge base must be reviewed whenever there is a change in product, commercial policy, operational process, price, delivery time, integration with partners, or any information that affects the response to the customer. Waiting for the problem to appear in conversations and then fix it means exposing customers to wrong information during the outdated period.

Practical signs that indicate a need for updating include:

  • Increase in transfers to human attendant in demands that should be resolved by AI, especially when the registered reason is “incorrect information” or “incomplete response”.
  • Decrease in satisfaction (CSAT) in conversations answered by IA without corresponding increase in volume or change of customer profile.
  • Customers who return for the same reason within a short period of time, suggesting that the first response didn't solve the problem or was out of date.
  • Internal Audit Reports pointing out divergences between what AI reported and what is in effect in the current policy.
  • Direct feedback from human attendants reporting that they need to correct information provided by the AI before transferring the conversation.

In addition to these reactive signs, it is essential to establish a proactive review routine. Any product change, seasonal campaign, regulatory change, or system update affecting the customer must trigger a knowledge base update checklist prior to the go-live of the change.

Update routine: who does it, when and how to test

Updating the knowledge base requires shared responsibility across areas. The service team identifies gaps and inconsistencies from the conversations; the product or sales team provides the changes; and the person responsible for curating the content validates, structures, and publishes the updates.

The recommended routine includes:

  1. Weekly review of sample conversations attended by AI, selecting cases transferred to human, negative reviews, and recontacts for the same reason. The objective is to identify recurring failure patterns.
  2. Record of identified gaps in a centralized document, with a description of the problem, an example of an affected conversation, responsible for providing the correct information and a deadline for resolution.
  3. Update or document creation in the knowledge base, following the previously guided structure, with title, objective answer, criteria, step by step, and examples.
  4. Response test before release to customers, simulating real questions in an approval environment to validate that the AI finds the updated content and answers correctly.
  5. Versioning and change record, keeping history of changes to track when certain information was updated and by whom.
  6. Internal communication for human attendants, ensuring that the team knows what has changed and is prepared to take on conversations that the AI transfers on the topic.

Modern service platforms with integrated AI facilitate this process by allowing [AI agents to be trained with multiple sources] (URL1) and provide handoff to the human attendant with full context of the conversation. BRAND, for example, combines [service automations with the possibility of human participation in the operation] (URL2), preserving continuity between channels and between AI and person.

How to identify content gaps from transferred conversations

Conversations transferred from AI to human attendant are the primary source of intelligence for identifying what is missing in the knowledge base. Every transfer should carry not only the conversation history, but also the reason why the AI decided to transfer.

The main transfer reasons that indicate a content gap are:

  • Question outside the mapped scope, when the customer requests information about a product, service, or process that is not documented in the database.
  • Information present but out of dated, causing the AI to provide a correct answer according to the old documentation, but incorrect according to the current policy.
  • Question that requires decision or interpretation, such as exception requests, deadline negotiation, or analysis of a specific case that AI does not have the autonomy to resolve.
  • Contradictory content between documents, leading AI to provide inconsistent answers according to the source consulted.
  • Generically described process, without sufficient detail to guide the client in specific situations.

For each recurring reason for transfer, it must be evaluated whether the solution is to add new content, update existing content, improve the document structure, add practical examples, or redefine the limits of AI action - making it clear when it should transfer because demand requires human intervention by nature, not due to lack of information.

[BRANDoffers a unified inbox] (URL3) with full customer context, making it easier for agents to review the conversation history with the AI and identify patterns of failure. When [several attendants use the same WhatsApp Business number] (URL4), for example, all interactions are recorded in a single history, allowing a longitudinal analysis of the quality of automated service.

Granularity of information: when to divide or unify documents

Documents that are too long make it difficult for AI to locate specific information; documents that are too fragmented increase the risk of incomplete answers. Optimal granularity balances depth and ease of searching.

The practical orientation is:

  • One document per process or policy, covering all variations and exceptions within the same text, with clearly identified sections.
  • Division into multiple documents when processes are independent, apply to different audiences, or follow different deadlines and conditions. Example: exchange policy and cancellation policy must be in separate documents, even if both are business policies.
  • Sections within the same document for variations of the same process, such as reimbursement for in-person versus online purchase, provided that the eligibility criteria are explicit at the beginning of each section.
  • Internal links between related documents, allowing the AI to complement an initial response by suggesting that the client consult another topic if the need evolves during the conversation.

Avoid duplication of content. If the same information appears in multiple places, any update will need to be replicated manually, increasing the risk of inconsistency. Prefer to centralize common information in a main document and reference that document from other contexts.

Response validation process before releasing

Updating a document and publishing it directly to the knowledge base without testing the AI response is risky practice. The AI may not locate the new content, it may mix information from multiple sources, or it may misinterpret the structure of the text.

The recommended validation process includes:

  1. Publication in an approval environment or marking the document as “under review” until the tests are completed.
  2. Simulation of real questions submitted by clients on the updated topic, using multiple language variations to test the robustness of the search.
  3. Verification of complete and correct answer, confirming that the AI found the updated content, didn't mix it with old documents, and presented the information clearly.
  4. Transfer testing when applicable, validating that AI recognizes when demand requires human intervention and transfers with preserved context.
  5. Review by a human attendant or specialist on the subject, ensuring that the response generated by the AI adheres to the current policy and contains no ambiguities.
  6. Release for production only after approval of all previous points, with internal communication about what changed.

Platforms that offer [AI agents with handoff for human attendant] (URL5) facilitate this process by allowing the transfer to take place in a transparent manner when the AI identifies that it is unable to respond securely. Setting clear boundaries for when to transfer is just as important as the quality of the knowledge base.

Versioning strategy and change history

Maintaining a history of knowledge base changes is essential for audits, root cause analysis of problems, and reversal of changes that hampered the quality of responses.

The versioning strategy must include:

  • Date and time of each update, recording when the document was modified.
  • Responsible for the change, facilitating contact in case of doubt or need to be reversed.
  • Brief description of the change, explaining what was updated and why. Example: “Updated delivery time from 7 to 10 business days according to the new commercial policy.”
  • Previous version of the document preserved, allowing side-by-side comparison and reversion if necessary.
  • Reason that triggered the update, such as product change, customer feedback, internal audit, or identification of a gap in a transferred conversation.

Content management tools aimed at knowledge bases often offer native version control. When the database is fed by external documents (Google Docs, Notion, Confluence), it is important to establish a manual process for recording changes in a centralized spreadsheet or content management tool.

Knowledge Base Quality Indicators

Monitoring the quality of the knowledge base requires metrics that reveal not only the volume of content, but its effectiveness in solving customer demands without the need for human transfer.

Recommended indicators include:

  • Resolution rate without transfer, measuring how many conversations initiated by the AI were finalized without passing to a human attendant, segmented by reason of contact.
  • **Satisfaction of conversations answered by AI (CSAT) **, collected via automatic survey at the end of the conversation, compared to the satisfaction of human conversations.
  • Average resolution time of conversations answered by AI, measuring whether the response was objective or if the customer had to ask multiple times to obtain complete information.
  • Recontact rate within 24 hours, identifying customers who returned for the same reason, suggesting that the first response was incomplete or incorrect.
  • Volume of transfers by reason, sorting by content gap, outdated content, contradictory information, or demand outside the expected scope.
  • Average age of the database documents, monitoring how much time elapses between updates and flagging content that hasn't been reviewed for more than 90 days.
  • Coverage of reasons for contact, comparing the list of actual reasons for contact (obtained from conversations) with the topics documented in the knowledge base.

These indicators must be reviewed weekly in active operations and monthly in smaller operations, with actions defined for each identified deviation. [BRANDoffers analytics with real-time dashboards] (URL6), making it easy to monitor service metrics by channel and by type of demand.

Integration with CRM and systems for personalized answers

Generic knowledge bases answer common questions, but many customer requests require specific registration information, purchase history, order or contract status. For AI to provide personalized answers, it needs to consult external systems in real time.

Recommended integration includes:

  • AI connection with CRM via API, allowing it to search for contact information, history of previous conversations, segmentation tags, and customized attributes.
  • Consultation with order, delivery, and billing systems, providing updated status of ongoing transactions without the need for the customer to provide an order number or protocol.
  • Controlled access to sensitive information, ensuring that AI only exposes personal data after customer authentication or identity confirmation.
  • Record of all inquiries made by IA, maintaining an audit of access to data and facilitating the investigation of problems or undue exposures.

Platforms such as BRAND[ connect to CRMs and automation tools via REST API and webhooks] (URL7), allowing AI agents to consult information from external systems before responding. This capability transforms AI from a generic assistant into a contextualized assistant capable of solving demands that require specific customer data.

Common mistakes that compromise the knowledge base

Even with a structured process, recurring errors compromise the effectiveness of the knowledge base:

  • Documents written in internal language, with acronyms, jargon, and references to systems that the client is not aware of, making it difficult to interpret by AI and making answers incomprehensible.
  • Duplicate content in multiple places, requiring manual updating at every point and increasing the risk of inconsistency.
  • Lack of practical examples, leaving abstractly described processes that do not guide the client in real situations.
  • Absence of explicit eligibility criteria, leading AI to provide generic guidelines that do not apply to the client's case.
  • Outdated documents not removed from the base, allowing the AI to consult old information and provide incorrect answers.
  • Slow approval process that delays the publication of updates, keeping erroneous information available for days or weeks after the change.
  • Lack of tests before releasing new content, trusting that the AI will correctly interpret the text without practical validation.

Avoiding these errors requires clear governance, defined accountability, and periodic review routines. The knowledge base is not an implementation project; it's an operational asset that requires ongoing maintenance.

Frequently Asked Questions

**How often should I review the knowledge base? ** Reactive reviews should occur whenever a product, policy, or process changes. Proactive reviews should include weekly review of transferred conversations and monthly auditing of documents that have not been updated recently. Critical documents (business policies, deadlines, pricing) must be reviewed before each seasonal campaign or launch.

**Who should be responsible for keeping the database up to date? ** Responsibility is shared. Attendants identify gaps and inconsistencies; product, commercial, or operations teams provide updated information; and the content curator structures, validates, and publishes. In smaller operations, the service manager may accumulate curatorship, but the process should not depend on a single person.

**How to test if the AI is responding correctly after updating the database? ** Simulate real questions in an approval environment, using multiple language variations. Validate that the AI finds the updated content, that the answer is complete and correct, and that there is no mix with old documents. Also test exceptional cases and situations that require human transfer. Only release for production after approval of all points.

**What to do when AI mixes information from different documents? ** Review the structure of the documents to eliminate content overlap. Centralize common information in a main document and reference it from other contexts. Use clear titles and sections to make it easier to search for AI. If necessary, state explicitly in the document when each information applies.

**How to prevent documents from becoming out of date without anyone noticing? ** Record the date of the last revision on each document and keep track of the average age of the base. Set up automatic alerts for documents that have not been updated for more than 90 days. Every product or process change must trigger a database review checklist before go-live. Include content review in regular operational routines.

**Can AI invent information that's not in the knowledge base? ** Generative language models can create plausible answers that don't match real facts, a phenomenon known as hallucination. To reduce this risk, configure the AI to respond only based on provided documents and transfer it to human when it doesn't find matching information. Test regularly and review conversations with low satisfaction.

**How to identify which topics are missing in the database? ** Analyze conversations transferred to the human attendant and classify the reasons. Recurrent topics that generate transfer due to lack of information indicate gaps to be filled. Compare the list of real contact reasons (obtained from conversations) with the documents available in the database. Prioritize topics with a higher volume and impact on satisfaction.

**Can I use the company's website as a knowledge base? ** Yes, but be careful. Websites often mix institutional content, marketing, and operational information. Guide the AI on which pages to consult and test whether the responses generated reflect current policies. Many companies prefer to structure a dedicated knowledge base, with clear, service-oriented language, for greater quality control.

References

  • [Chatbot with AI or chatbot with rules: which to choose for service? | Octadesk] (URL8)
  • [Service with AI: from the response to the execution of tasks in the systems | Octadesk] (URL9)
  • [Service flow: how to draw yours from scratch (with template) | Octadesk] (URL10)

Share

Open Instagram and paste the link in your post/story.