Review queue
Overview
Section titled “Overview”Detect & respond → Cases → Review queue is a reviewer’s inbox. It holds:
- samples: AI interactions and guard decisions picked by a sampling policy for a person to review;
- cases that are open and assigned to you or to nobody.
The inbox puts what is past its due date first, then the riskiest, then whatever is due soonest.
Prerequisites
Section titled “Prerequisites”| Permission | What it allows |
|---|---|
cases.review | Work the queue, record reviews, see the tuning statistics |
cases.configure | Add, change, run and remove sampling policies |
traces.read | Also needed to open a sampled interaction’s content |
cases.reveal | See who a sampled item is about |
Owners, administrators and the Compliance officer role have all of these. The Analyst role has cases.review.
Sampling policies
Section titled “Sampling policies”On the Sampling tab, add a policy:
| Field | Meaning |
|---|---|
| Sample from | AI interactions (the AI spans in this environment’s traces) or Guard decisions |
| Share sampled | The percentage of events picked, from 0.01 % to 100 % |
| At most per day | A cap on items per run |
| Review within | The hours a reviewer has before the item is past due |
| Filters | A model or service for interactions; a category, outcome or tool for guard decisions |
Each enabled policy samples the last 24 hours once a day, with the detection scan. Sample now runs it on demand. Sampling is deterministic: the same events are picked if a run repeats, and an event is never queued twice.
Content and data minimisation
Section titled “Content and data minimisation”Shield360 never copies content into the review queue. An item keeps a reference to the interaction, its metadata (model, service, token counts, or the guard decision’s category, outcome and kinds of data) and who it is about, pseudonymised.
- If the organisation stores prompts and responses (Data retention, Store prompts and responses), a reviewer who may also read requests can choose Show content. It is read from telemetry at that moment, clipped, with credentials redacted, and the read is written to the audit log.
- If the organisation does not store content, the review is of the metadata only. The Sampling tab says so.
- Guard decisions only ever hold masked values, so they are reviewed from what is shown.
Record a review
Section titled “Record a review”Choose Compliant, Violation or False positive, add a note if you like, and choose Record review. A violation can also Open a case for it, which needs cases.manage. An item is reviewed once.
Tuning
Section titled “Tuning”The Tuning tab shows, over the last 90 days:
- for each sampling policy and guard category, how its reviews came out;
- for each detection rule that opens cases, how those cases were closed (resolved counts as a violation, dismissed as a false positive counts as one).
After ten reviews it suggests what to do: Loosen a rule that is mostly false positives, Tighten one that often finds real violations, or Keep it.