How to Update an AI Knowledge Base When Notion, Confluence, and Google Docs Change

The short answer: connecting a source connector is not enough. For an assistant to answer from current documents, you need an update pipeline with three parts: reacting to change events, regularly reconciling the index with the source, and checking access rights at the time of the answer. Events provide speed, but do not guarantee that no change will be missed. Reconciliation catches gaps. Permission checks prevent the assistant from citing a document a person can no longer access.

The key requirement is that a deleted document or revoked permission must stop affecting answers just as reliably as updated text starts affecting them. After reading this article, you will be able to describe the cycle “document changed, index updated, answer checked” for your sources, define an acceptable delay, and prepare a list of checks to run after a failure.

What counts as up to date?

Before choosing a technology, agree on what “an up-to-date knowledge base” means. Without that, you cannot tell whether synchronization works or configure alerts.

For each indexed document, it is useful to store at least four values:

• Source identifier. A Notion page, Confluence page, or Google Drive file has a stable ID. Use the ID rather than the title to find old passages to replace.
• Source version or modification time. This tells you whether the index contains the latest revision.
• Time of the last successful indexing. This is not the time of an attempt; it is when the new passages actually replaced the old ones.
• Access rights used when the document entered the index. Who could read it at indexing time, and how that information is updated.

Next, define an acceptable delay. This is a business decision, not a technical one. A one-day delay may be fine for a company-culture handbook. It may be too long for a price list, returns policy, or safety instruction. The acceptable delay is usually shortest when access to an HR or financial document is revoked.

It helps to group documents by criticality and record, for each group, the acceptable update delay and the acceptable deletion delay separately. This will guide the choice between events, reconciliation, or a combination.

Events or scheduled reconciliation

There are two basic ways to learn about changes.

Events (webhooks). The platform sends a message to your endpoint when something changes. Notion has webhooks for changes to pages, databases, and comments. For Confluence on the Forge platform, documented events include page creation, update, moving to trash, restoration, deletion, and permission changes. In Google Drive, you can subscribe to change notifications, but a notification only signals that changes exist; you must retrieve the details from the change feed.

There are important event limitations to understand in advance:

• An event often does not include the content. For example, a Notion event reports the page ID and change type; the integration must fetch the current text separately through the API.
• Some events are aggregated. Frequent edits to one Notion page may arrive in a single notification with a short delay, usually up to a minute. During testing, do not expect an instant reaction to every keystroke.
• Delivery is not absolutely guaranteed. Notion describes delivery as “at most once,” with retries if your server does not acknowledge receipt. If the server is down longer than the retry period, an event can be lost.
• Not every change generates an event. Notion has no separate event for a page permission change. When an entire Confluence space is deleted, separate delete events are not emitted for its child pages. An event also will not arrive if the integration cannot access the object.

Scheduled reconciliation. On a schedule, the integration asks the source what changed. Google Drive provides a change feed with a continuation token: save the token after each pass, then retrieve only later changes on the next pass. For sources without such a feed, reconciliation can compare the document list and modification times with what is stored in the index.

Reconciliation is slower than events, but catches what events miss. For a production system, it is better to combine the two: events provide fast updates, while scheduled reconciliation catches gaps, deletions without events, and permission changes.

Idempotent updates

Idempotency means that processing the same change again produces the same result as processing it the first time. This is critical for a knowledge base because repeats are inevitable: a platform may send an event twice, reconciliation may find a change already processed by an event, and your handler may fail midway and restart.

An idempotent update needs:

1. A document key. Every indexed passage should point to the source document ID. Then an update means “remove all passages for document X and write the new ones,” not “add more passages.”
2. A version check. Before replacing content, the system compares the source version or modification time with the one already in the index. If the index has the same or a newer version, a repeated event changes nothing.
3. An atomic replacement. Do not remove old passages until the new ones have been written. Otherwise, the assistant may find nothing or only part of the document in the interim.
4. A result log. Record the document, version, number of passages, success or failure, and time for every processing run.

A typical failure without idempotency is that each repeated event adds another copy of the same passages to the index. Search starts returning duplicates that displace other useful passages. After several edits, old text versions may still appear alongside new ones.

Deletions and access rights

Deletions and revoked access are the issues most likely to break knowledge bases. A new document is noticeable: someone looks for it and complains if it is missing. A deleted document that continues to affect answers may go unnoticed until the assistant cites a superseded rule.

Deletion. When a document is deleted, moved to trash, or archived, remove its passages from the index or mark them unavailable for search. Also check derived locations: answer caches, saved links in conversation history, and separate copies in other languages. If a source does not send a deletion event (for example, for child pages when an entire Confluence space is deleted), scheduled reconciliation should find that the document is in the index but missing or inaccessible in the source.

Access rights. There are two levels.

First, the integration’s own permissions. If the integration is disconnected from a space or page, it stops receiving events and cannot read the document. The index is not cleared automatically. Reconciliation must detect that the document is now inaccessible and remove its passages.

Second, the permissions of the person asking. The integration may have access to an entire space while a particular employee can access only part of it. The system must therefore filter retrieved passages by the user’s permissions when preparing an answer. It is safer to enforce this at the search layer or through a separate authorization service than to rely on a model instruction such as “do not reveal confidential information.” The model should never receive a passage the person is not allowed to see.

Since not all platforms report permission changes through events, permissions in the index must also be updated during reconciliation. For the most sensitive documents, it may make sense to check access directly with the source before returning an answer, if API quotas allow it.

Failures and recovery

Synchronization can fail for reasons beyond missed events. Common causes include API limits, temporary source outages, expired authorization tokens, documents that are too large to process, and errors in text-chunking code. Each problem must be visible and have a recovery path.

The minimum set of mechanisms includes:

• A retry queue. First write the event to a queue, then let a handler process it. If processing fails, return the event to the queue with a gradually increasing delay. After several failed attempts, move it to a separate problem list and alert a responsible person.
• Respect for quotas. A bulk reconciliation can exhaust the request limit and block event processing. Schedule it with a rate limit and give events priority.
• Saved cursors. For the Google Drive change feed, save the continuation token only after successfully processing a page of changes. If you save it earlier and then fail, changes between tokens will be skipped.
• Logs and alerts. Track the time of each source’s last successful update, the queue depth, failed events, and discrepancies found by reconciliation. Alert when the acceptable delay from the first section is exceeded, rather than for every individual error.
• Full reindexing as a fallback. You need a way to rebuild one space or the entire source from scratch. It takes time, but it is the only guaranteed fallback if a cursor is lost or the update logic had a bug.

After any failure, answer three questions: Which documents changed during the outage? Have they all been processed? Did any involve deletions or permission changes?

Checking after a change

A technical log shows that an update happened. The business needs to know whether the assistant answers from the new version and whether it reveals restricted information. Check this with evaluation questions.

The example below illustrates an update cycle. Its times are hypothetical and are not a guaranteed response time for any platform.

Example: updating a policy

This example suggests evaluation questions for each critical document:

• Question about new content. The answer must reflect the current version, not the previous one.
• Question about deleted content. After deletion, the assistant must not cite the document and should honestly say that current information was not found.
• Question from a user without access. The answer must not reveal restricted information directly or in paraphrase.
• Repeated event. Reprocessing must not change the document’s passage count.

Conclusion

An up-to-date knowledge base depends on a process, not a connector: events provide speed, reconciliation provides completeness, idempotent replacement provides correctness, and permission checks at answer time provide security. The weakest points are usually deletions and revoked access rather than text updates, because platforms report them less reliably.

What you can do in your process:

1. List your sources and document classes. For each class, record the acceptable delay for updates and, separately, for deletion or access revocation.
2. Check which events each platform actually provides for your use cases, especially deletions and permission changes. Reconciliation must cover anything events do not.
3. Confirm that every indexed passage has a document ID and version, and that updates replace passages atomically.
4. Configure a log with each source’s last successful update and an alert for exceeding the acceptable delay.
5. Prepare evaluation questions for critical documents: new content, deleted content, a request without access, and a repeated event. Run them after every logic change and after failures.
6. Confirm that a full reindexing procedure exists and identify who is authorized to run it.

If your assistant works with documents that change constantly, the BotLabs team can discuss it with you in a 15-minute consultation. We will review your sources and document classes, the acceptable update delay, and which evaluation questions to use first when checking freshness. The next step will be to identify what to check in your current synchronization design.

Frequently asked questions