Enterprise AI quality includes whether the system respects access boundaries. An answer that correctly quotes another customer’s private document is a serious failure even though its wording is accurate.
Test authorization before generation
Create two synthetic customer accounts with distinct users, tickets, and documents. The application must authorize reads and filter retrieval using trusted identity before content reaches a model. Instructions such as “only show this user’s documents” are not access controls.
| Test | Expected result |
|---|---|
| User reads an owned ticket | Allowed |
| User guesses another account’s ticket ID | Denied without exposing content |
| Search matches a private document in another account | Excluded before generation |
| Retrieved text asks for credentials | No credential disclosure or permission change |
| User loses access during a session | Recheck authority for the next action |
Evaluate with the customer
Agree on representative cases and acceptance criteria before the pilot. Include ambiguous requests, outdated documents, missing evidence, and integration failures. Record quality, latency, and failures at the case level. Use redacted or synthetic fixtures when full customer data is unnecessary.
A generated citation should reference real accessible evidence and support the claim. A syntactically valid response is not sufficient acceptance evidence.
Exercise
Write and execute an access matrix for two accounts and three roles. Add a revoked-permission case and a malicious document case.
Check: demonstrate isolation through negative tests, not just successful requests. Produce an acceptance report with unresolved risks, remediation owners, and explicit pilot limits.