AI analyst stories and behaviour drift

Overview

Some risks only show when you put signals side by side. A new MCP tool is ordinary. So is a busy afternoon of file reads, or a call to a domain nobody has used before. All three from the same person, in that order, within an hour, is worth a look. The AI analyst looks for these combinations every day and writes them up as stories.

It also compares each agent, person and MCP server’s week with their own previous weeks. A subject that suddenly writes files instead of reading them, reaches three new destinations or retries four times as often gets a drift finding that lists what changed.

Both run with the analyst’s daily checks, on the same schedule and lease, and add findings to Detect & respond → Detections → Analyst findings. Filter by Story or Drift.

Finding a story or drift is deterministic. It uses SQL over your data and fixed rules, and works with the analyst off and with no model configured. When the analyst is on and a model is configured, the model writes each new story’s account and explains up to three new drift findings. It is sent only the aggregates described below, pseudonymised like every other analyst run (see Privacy). These runs count against the daily budget. When the analyst is off or the budget is spent, the finding is still raised with its timeline and score, without the account.

Stories

What the story pass reads

Once a day, the story pass reads the last 24 hours of each environment’s risk signals. You can change this to between 1 and 168 hours in Settings → AI analyst → Stories. It groups the signals by subject: a person, a machine, a coding agent or an MCP server.

Signal	Source
Detection rule matches, other alerts, collector tampering	Alerts for the environment
Sensitive data blocked, sent anyway after a warning, unapproved model used	Guard decisions
Prompt-injection detections, policy violations	The collector’s security events
New MCP tool	An MCP tool first called in the window, not seen in the previous 30 days
Spike in file reads	At least 40 reads, and at least three times the subject’s rate over the previous 7 days
Outbound calls to a new domain	A web or shell tool reaching a domain first seen in the window, not in the previous 30 days
Agent permission prompts bypassed	Tool calls made in a bypass or full-auto permission mode
Collector silent	A machine’s collector quiet for an hour or more, between heartbeats or since the last one
Key used from a new machine	A key seen on a machine for the first time, that was already in use elsewhere
Usage spike	At least 200,000 tokens, and at least three times the subject’s rate over the previous 7 days
AI inventory change	An agent or MCP server registered or decided on in the inventory
Agent gateway grant	A grant made to an agent, or for an MCP server

Only counts leave the database: how many events, the first and last time, and how many distinct destinations. No prompt, file path, argument, domain name or guarded value is read. Each subject’s timeline keeps at most 12 kinds of signal, and a pass keeps at most 500 subjects: those with the most kinds of signal.

Each source is read separately. If one fails, the story pass carries on without it, and each story it raises says which sources could not be read. The pass fails only when every source fails.

Scoring

Each subject is scored:

Kinds of signal. Each kind of signal the subject has counts once, for 1 to 3 points, however many events it had. Twenty blocks are one signal. Prompt injection and collector tampering are worth 3, routine alerts, blocks, inventory changes and grants 1, and the rest 2.
Closeness. The points are multiplied by 1.5 if every kind of signal falls within two hours of the others, by 1.25 within six hours, and by 1 otherwise.
Known sequences. A bonus is added for each known risky combination the timeline completes, in the order that makes it risky:

Pattern	What it needs	Bonus
New MCP tool, file reads and outbound calls	A new MCP tool, a spike in file reads and outbound calls to a new domain, the tool first seen no later than the last outbound call	6
Collector silent while agents ran unchecked	The collector silent or tampered with, and permission prompts bypassed within two hours of it	5
A key on a new machine, and a usage spike	A key new to a machine, and a usage spike that goes on after the key appeared	5
Sensitive data sent anyway, and an unapproved model	Sent anyway after a warning at least twice, and an unapproved model used	4

Feedback. The score is multiplied by the weight of the story’s pattern in this environment (see Feedback).

A subject needs at least two kinds of signal. Stories at or above the floor become findings, highest score first, up to the stories-per-pass setting (5 by default). The floor is 8 by default, so any complete known sequence qualifies, while three unrelated signals need to be close together. If the same events form a story about several subjects, such as one person on one machine with one agent, it is told once, about the person.

A story’s severity is critical at twice the floor and warning otherwise. Confidence is high for a known pattern at one and a half times the floor, medium for any other known pattern or a score at one and a quarter times the floor, and low otherwise.

The story

The finding shows the timeline, oldest first. Each kind of signal links to the page where its events are, with its count, first and last time, and number of distinct destinations. The finding also shows the score, the floor and the pattern. When the analyst is on, The analyst’s account follows. It covers what happened in order, why the combination matters, how confident the analyst is, what would confirm or rule it out, and the next step.

The model is sent the subject’s alias (or the agent’s or MCP server’s name), the score, the pattern and the timeline lines, and nothing else.

Feedback learning

Marking a story Useful or Noise adds a vote to its pattern in that environment. A pattern is one of the named sequences above, or several signals at once for a story without one. The pattern’s weight is worked out from its votes:

weight = clamp(1.2 ^ useful × 0.7 ^ noise, 0.25, 2)

Three noise votes bring a pattern’s stories to about a third of their score. One useful vote raises them by a fifth. A pattern never falls below a quarter or rises above double, so feedback can quieten a pattern but not silence it. Changing or withdrawing a vote undoes exactly what it did. Weights belong to the environment, so one team’s noise does not quieten another’s. The story shows the weight it was scored with.

How story findings behave

One finding per subject, story and day. A story is identified by its subject, its pattern and kinds of signal, and the UTC day of its latest event. Running the pass again, or tomorrow’s pass reading the same events, refreshes the same finding and keeps its status and feedback.
A new kind of signal, or a new day of activity, is a new story. The earlier story stays as it was.
Stories are not resolved automatically. They record something that happened. Resolve or dismiss them yourself.
The account is kept while the story is unchanged. It is written again only when the story changes.

Behaviour drift

The fingerprint

Drift compares each agent, person and MCP server’s coding-agent activity this week with the four weeks before. This week is the seven whole UTC days before the check runs. The fingerprint is read from the runtime traces:

Dimension	Measured as
Tool-call mix	The share of tool calls that are file reads, file writes, commands, web calls, MCP tools and other tools
Destinations	Distinct MCP servers and outbound domains, and how many this week were not seen in the baseline
Models	Distinct models, and how many are new
Error rate	Tool calls that ended in an error
Retry rate	Tool calls repeating an earlier call in the same session with the same arguments
Permission prompts bypassed	The share of tool calls in a bypass or full-auto mode
Volume	Tool calls this week, against the baseline’s weekly average
Active hours	The hours of the day with activity, and how many are new

Destinations, models and hours are compared inside ClickHouse. Only the counts come back, never a domain or a model name.

Distance and score

Each dimension’s change is scored from 0 (no change) to 1:

Tool-call mix: the share of calls that moved between categories.
Destinations and models: new ones divided by this week’s count, or 3 if that is larger.
Error, retry and volume: the size of the ratio between this week and the baseline. An eightfold change either way scores 1.
Permission prompts bypassed: twice the change in share.
Active hours: the share of active hours that are not shared between the two periods.

The drift score is the weighted root mean square of the dimensions. Tool-call mix, destinations and permission bypass count double, models one and a half times, and active hours half. One dimension that changed completely stands out, while several small changes do not add up to much. A dimension at 0.4 or more is listed as changed, and a subject scoring 0.3 or more has drifted.

A subject is compared only with enough history: at least two active weeks in the baseline, at least 50 tool calls in it, and at least 20 this week. Up to 10 subjects with the highest scores become findings. Severity is warning from 0.5 and info below. Confidence depends on how much baseline there is.

The drift finding

The finding lists what changed, for example: new destinations: 3; tool-call mix shifted toward file writes (20% → 70% of calls); retries 4× (2% → 8% of calls). The table shows each dimension’s baseline and this week, with the changed dimensions first. When the analyst is on, a short explanation follows. The model is sent the subject’s alias or name and these figures only.

Drift findings are keyed by subject and ISO week, so a second run in the same week refreshes the finding. They are not resolved automatically.

Why drift is not a detection rule

A detection rule asks one question of one signal per group, such as a threshold, a rise against the norm, a sequence or a first sighting. Drift compares a whole distribution across eight dimensions with the subject’s own history, so it runs as an analyst check. To alert on a rise in one metric, use a baseline detection rule.

Detection templates

Two of the story pass’s sequences are also detection templates, under Rule engine → Detections → Templates:

MCP tool call, then an outbound call: in the same session, within 30 minutes. A rule cannot tell whether the tool or domain is new, so it fires more often than the story.
Sent anyway, then an unapproved model: the same person, within an hour.

The other sequences need something a rule cannot express. “A key new to this machine” and “the collector silent” are not events a sequence step can match, so those stay in the story pass. The existing template Collector tampered with, then permissions bypassed covers the tampering half of the unchecked-agents pattern.

Schedule and running now

The story pass runs with the daily checks. Drift runs on the first scheduled scan at least a week after its last run. Run checks now in Findings runs both, regardless of the weekly schedule. With the inbox filtered to Story or Drift, the button runs only that kind: Look for stories now or Check for drift now. A run by hand is limited to once every ten minutes per environment, as for the other checks, and is recorded in the audit log.

Permissions

These findings use the same permissions as the rest of the analyst:

Permission	What it allows
View AI analyst findings	Read stories and drift findings, and mark them useful or noise, which adjusts the pattern weights
Triage AI analyst findings	Change their status, and run the checks now
Change AI analyst settings	Change the story window, stories per pass and score floor for the organisation

Limits

Stories see what Shield360 records. A signal from an unmonitored machine, or a source that could not be read, is missing. The story says which sources were missing.
First sightings look back 30 days. Something last seen 31 days ago counts as new.
Spikes compare with the subject’s own previous 7 days. A subject with no history spikes as soon as it passes the minimum.
Agent and MCP server names come from telemetry, and gateway agents use their agent key. The same agent under two names is two subjects.
Drift needs coding-agent traces. Subjects without them are not compared.
The account and the explanation are the model’s reading of the figures. They add no evidence.