AI knowledge base security checklist: leave with a decision
Contents
Ask for the evidence
Ten checks before your pilot
The vendor demonstrates a convincing answer and says your data is secure. Your procurement meeting still has a job to do: decide which information, users and actions the evidence actually supports. Use these ten questions to leave with a bounded pilot decision, not a list of reassuring feature names.
Write the proposed use in one sentence
For example: “Three operations staff will search five fictional support documents in the vendor’s web application; no customer records, external agents or source changes are included.” That is a deliberately small pilot, not a company-wide security approval.
Bring the proposed data, identities, client and enabled actions to the meeting. Ask the vendor to identify each data destination, including imported copies, indexes, model processing and logs. The diagram sets the scope for the questions below. A product-wide answer that ignores your configuration is not specific enough.
Keep evidence type separate from the finding
Use two fields in your notes. Evidence type tells you what you have: vendor-documented commitment, observed behavior in a controlled check, independent assurance, or no evidence supplied. Finding tells you whether that evidence supports the proposed use, contradicts a requirement, or leaves a question open.
A no-training contract can establish a commitment; a successful browser query cannot verify it. An observed permission denial establishes what happened in that check; a policy saying “permissions are respected” does not establish the same thing. An independent report has a scope, period and exclusions that a qualified reviewer must assess.
“Not applicable” is a scope decision, not an evidence type. An excluded write tool may be irrelevant to a read-only pilot, but becomes relevant if someone later enables it. Avoid a single green “verified” label that hides these distinctions.
Ten questions for the meeting
This is an editorial buying aid. The questions are not a certification standard, and a completed table does not establish compliance. Attach evidence references and the configuration reviewed; assign unresolved questions to a named owner.
| Check | Ask | Decision it informs |
|---|---|---|
| 1. Data destinations | Where do imported content, derived indexes, questions and logs go? Name relevant processors and regions. | Whether this data may enter the proposed route. |
| 2. User identity | Which identity and grants govern retrieval, including source titles and full-document access? | Whether the required separation between users is supported. |
| 3. Source updates | How are edited documents and changed source permissions reflected in imported copies? | Whether stale content or access delays fit the workflow. |
| 4. Training and retention | What do current terms say about training use and retention at each processor? | Which commitments must cover the exact account and service. |
| 5. Model account | If a customer supplies a model key, who stores and uses it, and which provider agreement applies? | Who is responsible for provider configuration, costs and data handling. |
| 6. Actions | Which tools can read, write, delete or send, and which are enabled? | Whether the pilot can remain within its intended action scope. |
| 7. Revocation | What happens to new requests from fresh and already-open sessions after access is removed? | Whether offboarding behavior meets the stated requirement. |
| 8. Records | Which events are recorded, who can inspect them, and how long are they retained? | Whether a reviewer can investigate the incidents that matter. |
| 9. Deletion | What is deleted from active systems, backups and downstream providers, and when? | Whether exit and retention requirements can be met. |
| 10. Assurance and commitments | Which reports, contracts and support terms cover this service and configuration? | Whether the purchase has sufficient independent and contractual support. |
A completed meeting record
The following team, documents and meeting observations are fictional. They illustrate how to make a decision; they are not vendor or HeyBrain test results. The meeting takes place on 20 October 2026.
Operations proposes the three-person, five-document web pilot described above. Lina owns operations, Sam reviews security, and Jo reviews supplier terms. Their rule before the meeting is that restricted data and external agents remain excluded unless their access lifecycle is established.
| Question | Evidence type and actual note | Finding for this proposal |
|---|---|---|
| Data route and retention | Vendor-documented: diagram D1 names the application store, search index, model processor and request log. Terms T1 describe retention for each. | Jo accepts the documented route for invented test documents only. No restricted-data approval. |
| Training commitment | Vendor-documented: T1 states customer content is excluded from model training for this service. | Commitment recorded. The meeting did not inspect provider operations. |
| Web identity and revocation | Observed in the fictional check: two test users have different document grants; denied text and titles stay hidden. After removing a grant, fresh and already-open web sessions cannot retrieve the document. | Supports these actors and this web configuration. Does not establish every identity path. |
| Actions | Observed configuration: no write, delete or send tools are enabled. The pilot uses web search over imported documents only. | Read-only scope retained. Enabling a tool requires another decision. |
| External-agent revocation | No evidence supplied: the vendor cannot demonstrate the external-agent session behavior during the meeting. | Unresolved. External agents are excluded, not marked passed. |
| Independent assurance | No current independent report covering the proposed service has been supplied. | No independent assurance claimed. Sam must resolve this before considering restricted data. |
The decision: a small pilot, with two explicit exclusions
Lina approves a five-working-day pilot from 21 to 27 October 2026 for the three named staff and five invented documents. Its purpose is to check whether search is useful enough to justify further evaluation. It is not approval to import customer or employee records.
The approved route is the vendor web application only. No external agents and no write actions are permitted. Lina owns the participant and document list; Jo retains the reviewed terms. Sam asks for the missing external-agent evidence and assurance scope by the 27 October review.
At that review, the team can stop, continue the same fictional-data scope, or propose an expansion supported by new evidence. Missing evidence does not silently become approval when a date arrives. If the tool cannot remain within the agreed route or data scope, the pilot stops.
The unresolved agent question matters, but it does not prevent every possible use of the tool. The team has excluded that route from this experiment. For a purchase that requires agents and restricted records from day one, the same missing evidence would block the required use. Scope explains the different decisions.
Use demonstrations for the questions they can answer
For a practical permission exercise, take the five-check company knowledge access test to the technical reviewer. It supplies fictional documents, exact questions and completed results. Keep this buying meeting focused on which findings affect the decision.
OWASP’s excessive-agency guidance identifies excessive functions, permissions and autonomy as distinct risks. For a reading task, an unnecessary send or delete tool increases what can go wrong. Ask about the enabled operations, not just whether the product calls itself an assistant.
Apply these questions to HeyBrain as well. The published privacy policy is a policy source, not a substitute for observations of your required workflow or an independent assurance report. Keep training, retrieval, retention and source write-back separate in the decision.
Leave with one sentence that names what is approved, what is excluded, who owns it, and what evidence could change the decision. If the meeting produces only “security looks good,” the most useful work remains unfinished.
Prepared by HeyBrain Editorial Team with AI assistance. The checklist and completed meeting record are original editorial examples; the meeting and all observations are fictional. The October 7, 2026 correction replacing unsupported product assurances and fixed timing claims is retained, along with the original publication date. Primary sources were checked on 13 October 2026.
