Activity provenance: what every AI agent did

Overview

Detect & respond → Agent activity answers, for every action an AI agent took: who did what, through which agent and model, on what system, under whose authority, at what cost, with what result. An action is a shell command, a file read or edit, an MCP tool call, a database query, a cloud or Kubernetes command, a git push, a request through the AI proxy — one record each.

Records come from:

• Coding agents with the Shield360 hooks (Claude Code, Codex, Cursor): tool calls, edits, commits and pull requests, sub-agents, model turns, guard decisions and permission prompts.
• The agent gateway (MCP): every tool call an agent makes through it, with the grant it acted under.
• The AI proxy: each request it blocked, redacted or warned about.
• Apps using the SDK: spans that execute a tool (gen_ai.operation.name = execute_tool) or call an MCP tool.

Records are assembled on the server as telemetry arrives, after Data retention has run, so a record never holds more than your organisation stores.

Enabling recording

Recording is an organisation setting, changed on Detect & respond → Agent activity by an administrator with the Change what is stored permission. Each change is recorded in the audit log.

Organisation	Default
Created after recording became an organisation setting	On
Existed before	Unchanged: it records only if the deployment had PROVENANCE_ENABLED=true. Administrators see a prompt to turn it on.

While it is off for an organisation, nothing is assembled at ingest for it, nothing is chained or written, the page shows Provenance recording is disabled above any records written earlier, Verify integrity reports disabled when there is no chain (an earlier chain is still verified), and the provenance.recorded control reports not applicable rather than a gap.

What a record holds is already minimised: the agent, model, action type, target, result, cost and who it acted for. The command itself is kept only under Data retention’s Actions (tokenized) category (see below), with credentials as tokens and personal data masked. Prompts, responses and tool results are never part of a record.

Deployment kill switch

PROVENANCE_ENABLED is now a deployment-level kill switch. Set it to false (or 0, off, no) on every app replica to stop recording for every organisation, whatever their setting:

PROVENANCE_ENABLED=false

Unset, organisation (what the Kubernetes manifests in deploy/k8s/shield360-*/app.yaml set) or any other value, each organisation’s setting decides. true also keeps older organisations recording until an administrator chooses.

Recording never slows ingest. Each replica hands records to a bounded in-memory queue and the ingest request returns at once; the queue writes in the background, one write at a time per environment (records that arrive meanwhile are coalesced into the next write) and at most four at a time per replica, so provenance holds at most four database connections per replica, each only briefly. If more than 20,000 records are waiting, the oldest are dropped, counted and logged (activity provenance queue full: dropped …). A dropped record never got a sequence number, so the chain stays gapless and verifiable; the action is still in the environment’s traces, it is just not in the chain.

How secrets are handled

Credentials are never kept in any form except their type and a keyed fingerprint:

curl -H "Authorization: Bearer [token:fp_3f9a0c51]" https://api.example.com/v1/items
PGPASSWORD=[password:fp_84eaeb79] psql -h db.internal -d ledger

• A token is [<type>:fp_<12 hex>]: the type (api_key, token, password, cloud_credential, private_key, secret) and the first 12 hex characters of HMAC-SHA256 over the type and value, keyed with your organisation’s fingerprint key. It contains no character of the credential, so nothing of it can be read back, but the same credential used twice gives the same token.
• There are no partial masks: not a prefix, not the last four characters.
• Where a vault reference is known (a gateway grant, a vault key name), it is recorded in place of the value.
• Credentials are recognised in URLs (postgres://user:…@host), headers (Authorization, X-Api-Key), assignments and environment variables (PGPASSWORD=…, "password": "…", ?token=…), command-line flags (--token …, mysql -p…), provider key formats (OpenAI, Anthropic, GitHub, Slack, Google, Stripe, AWS, npm, Hugging Face, JWTs, private keys) and long high-entropy strings.
• Tokenizing happens on the machine (the coding-agent hooks) and again on the server whatever a client sent. Tokenizing already tokenized text changes nothing.
• Personal data in the action (e-mail addresses, phone numbers, payment cards, IBANs, US social security numbers) is always masked, and the text is cut to 1,024 characters after tokenizing, so a cut never leaves part of a credential.
• Targets are reduced further: a URL to its host, a file to its name and a category (secrets, ci, infrastructure, source, …), a git remote to owner/name.

Data retention: “Actions (tokenized)”

The tokenized action is stored under its own Data retention category, Store actions (tokenized), which is on by default. Prompts and responses stay off unless you turn them on.

Actions (tokenized)	What a record keeps
On (default)	Everything below, including the tokenized action and the targets and privilege read from it.
Off	The kind of action (cloud_cli, file_edit, …) and the fields that are not the action: agent, model, person, computer, environment, result, policy decision, approval, delegation and cost. No action text, no target read from it, no privilege read from the command.

An environment can turn it off for itself; it cannot turn it on when the organisation has it off. The device policy tells machines whether to send it (dataRetention.storeActions).

The record

Group	Fields
Agent and model	agent, agent_version, agent_id, model, provider, source (coding_agent, sdk, gateway, proxy)
Action	action_type (shell, file_read, file_write, file_edit, http, db_query, mcp_tool, cloud_cli, k8s, iac, git, package, search, web_fetch, subagent, model_call, ai_request, tool), action_tool, action_text (tokenized)
Target	target_kind, target_host, target_resource, target_database, target_cloud_provider / account / region / service, target_k8s_context / namespace, target_git_remote, target_path_category, cloud_resource_key
Where it ran	host_name, host_id (the collector id), container_id, os_type, os_arch, shell, runtime, exec_environment (ci, container, remote, workstation), ci_provider
Security context	person, os_user, service_account, cloud_identity (AWS profile/role, GCP account, Azure subscription — names only), privilege_level (standard, elevated, admin, root), privilege_reasons, assumed_role, permission_mode
Result	occurred_at, ended_at, duration_ms, result (success, failure, denied, unknown), exit_code, error_type
Policy and approval	policy_decision (allow, warn, redact, block), policy_source (guard, sequence, gateway_grant, injection, proxy), policy_name, approval_decision (allow_once, allow_always, deny, auto_allowed), approval_by, approval_at, approval_source, override_marker
Under whose authority	delegation (the chain, earliest first: person → agent → sub-agents → MCP server, each with its session and the grant or permission mode it acted under), initiating_human, acting_principal
Correlation	session_id, parent_session_id, run_id, trace_id, span_id, parent_span_id, tool_call_id
Cost	input_tokens, output_tokens, cost_usd, tool_cost_usd, session_outcome
Integrity	record_id, seq, record_hash, prev_chain_hash, chain_hash

Privilege

Privilege is read from the action and from the machine: sudo, doas, runas and PowerShell -Verb RunAs (elevated); a root shell or running as root (root); Kubernetes contexts and impersonation that read as administrators, AWS roles and profiles named for administrators, assumed roles, and changes to access rights such as aws iam attach-user-policy or az role assignment create (admin); an agent bypassing its permission prompts (elevated). Auto-accepting edits is recorded as a reason but does not raise the level.

Approvals

When a person answers a coding agent’s permission prompt, the record says what they chose, who they were and when. Where the agent cannot tell a person’s answer from its own mode allowing the action, the record says auto_allowed with the mode (for example bypassPermissions), never a guess:

• Claude Code: permission prompts are seen through the PermissionRequest hook; an allowed call is allow_once (Claude Code does not tell hooks whether “always” was chosen).
• Codex: approvals from its PermissionRequest hook and approval mode.
• Cursor: Cursor does not report the developer’s answer, so its calls are auto_allowed with the composer mode.
• Guard overrides: a guard block is deny from hook_policy; sending a warned prompt anyway sets override_marker (the reason itself follows Data retention).
• The agent gateway has no approval step: calls are allowed or refused by grants.

Cost

Each record carries the AI cost of its step where the step is a model call (token usage priced at ingest from your model prices), and tool cost where a tool reports one. The page header and the export add up the cost of autonomy for the period:

• Per successful action: AI and tool cost divided by the actions that succeeded.
• Failed or denied: each session’s cost shared evenly across its actions, the failed and denied shares added up. An allocation, not a measurement.
• Per session outcome and per merged pull request: sessions whose outcome is merged, and the pull requests they opened.
• Per resolved ticket: not available — sessions are not linked to tickets.

A session’s cost counts once: the sum of its model calls, or the session’s own total when its calls were not each reported.

Cloud cost attribution. When an action targets a cloud resource, the record carries cloud_resource_key (provider:account:region:resource, for example aws:123456789012:eu-west-1:s3://acme-logs). Join it to a cloud cost connector’s data later; Shield360 does not fetch cloud bills when an action is recorded.

Using the page

• Filters: agent, model, person, computer, action type, privilege, result, policy decision, source, a target search (host, resource, database, account, Kubernetes context or repository) and a minimum cost. The window is the page’s time range.
• A record opens every field above, the chain of delegation, and links to its trace, policy violations, the AI inventory, the risk register and the analyst’s findings.
• A session opens its steps in order — its sub-agents’ too — with the approvals given, the chain of delegation and the session’s cost.

API and export

All routes read the caller’s current project database, or a project API key’s own; a database named in the query is ignored.

Route	Permission	Returns
GET /api/provenance?start&end&<filters>&limit&before&summary=1	provenance.read	A page of records, newest first; before is the previous page’s next.
GET /api/provenance/record?id=<record id>	provenance.read	One record and the approvals given for it.
GET /api/provenance/session?id=<session id>	provenance.read	A session’s steps, approvals, totals and delegation.
GET /api/provenance/cost?start&end	provenance.read	Cost per outcome.
GET /api/provenance/export?format=csv|json&start&end&<filters>	provenance.export	Up to 50,000 records with their chain hashes and covering checkpoints; JSON adds the cost per outcome and the public keys (see Integrity).
GET /api/provenance/export?format=csv|json&from&to	provenance.export	Records from–to by sequence number, unfiltered: an export with no gaps in the chain.
GET /api/provenance/verify?from&to or ?start&end	provenance.read	Whether the records are intact: see Verify on the server.

curl -s -H "Authorization: Bearer $SHIELD360_API_KEY" \
  "$SHIELD360_URL/api/provenance/export?format=json&start=2026-09-01&end=2026-10-01&privilege=admin" \
  -o provenance.json

Every read and export is recorded in the audit log. The Compliance officer role has both permissions; Analyst can read.

Retention

Records are kept like the other compliance evidence (guard events, prompt-injection detections, policy violations): seven years (2,557 days), in shield360_activity_provenance in the environment’s own database.

SIEM, controls and detections

• SIEM export: the Agent activity category sends each record as an OCSF API Activity event (class 6003), with the agent as actor, the computer as source, the target as destination and resources, and the tokenized action, privilege, approval and cost under unmapped.shield360. See SIEM export.
• Controls evidence: AI actions have provenance is met when agents acted in the period and at least 95% of their actions name the agent and the person or principal they acted for. The check also verifies the period’s records (see Integrity) and reports Integrity verified with how many records and checkpoints it checked; a broken chain or a checkpoint that does not verify makes it a gap, however well attributed the actions are. It backs EU AI Act Art. 12, NIST AI RMF MEASURE-2.4, MEASURE-2.8 and MANAGE-4.1, and OWASP ASI03.
• Detections: the Agent actions signal offers action type, tool, agent, model, target, privilege, result, policy decision, approval, environment, exit code, duration and cost (the action text can be filtered on and is never shown). Templates: Privileged cloud command by an agent, Agent touched secret files, Many denied agent actions in one session and Agent session cost runaway.

Integrity

Activity provenance is tamper-evident: changing, removing, inserting or reordering a record after it was written is detectable, on the server and offline.

The hash chain

Each environment’s records form a hash chain:

• record_hash is the SHA-256 of the record’s canonical serialisation: its fields in a fixed order, nested keys sorted, numbers rounded to nine decimal places, in the form the record is stored in (timestamps as YYYY-MM-DD HH:MM:SS.mmm, counts as whole numbers).
• seq numbers the environment’s records 1, 2, 3 … with no gaps.
• chain_hash is SHA-256(prev_chain_hash + record_hash), where + joins the two lowercase hex strings and prev_chain_hash is the chain_hash of the record numbered one below (64 zeros before record 1).

Ordering guarantee. Records are numbered in the order their batch reserved its sequence numbers, not by occurred_at. Each batch is hashed first; then one short PostgreSQL transaction locks the environment’s chain head (SELECT … FOR UPDATE, with a two-second lock and statement timeout), claims the batch’s record ids, gives the newly claimed records the next sequence numbers, chains them onto the head and moves the head. Only after that commits is the batch written to ClickHouse, with retries. Concurrent writers on any number of replicas therefore produce one gapless, correctly linked chain, and the head is never locked while ClickHouse is written. Records written before the chain existed have empty chain hashes and are not verified.

Duplicates. A span is chained once, however many times and to however many replicas it is delivered. Records already stored in ClickHouse are dropped before the head is locked. The exact check is a claim: one PostgreSQL row per environment and record id (provenance_record_claim, the record id only, no content), inserted in the same short transaction with ON CONFLICT DO NOTHING. Of two replicas reserving the same record at the same instant, only the first to claim it chains it; the other drops it as a duplicate. Claims are deleted after 24 hours by the checkpoint job, in bounded batches; by then the record is in ClickHouse and a resend is dropped there. When a write fails (below), its claims are released, so a resend is chained again. Claims go with the environment when it is deleted; an erasure leaves them to age out, since they hold nothing about the person.

Writes that fail. Hashes are fixed when the sequence numbers are reserved, so the chain stays verifiable whether or not the rows land. If the ClickHouse write still fails after its retries, each reserved seq gets an append-only tombstone with reason write_failed (its hashes, no content) and a provenance.write_failed audit event. Verification reads such a seq as lost on write: a data-loss warning, not tampering, and the chain links through the tombstone’s hashes. It does not fail verification or the provenance.recorded control; the count is shown with the result. Only if the tombstone could not be written either does the seq read as missing.

Signed checkpoints

Every five minutes (PROVENANCE_CHECKPOINT_SCHEDULE), and as soon as 10,000 records are unsigned, Shield360 signs a checkpoint for each environment with new records: {type, tenant, fromSeq, toSeq, count, prevChainHash, lastChainHash, issuedAt, keyId} as canonical JSON (keys sorted, no spaces), signed with the environment’s Ed25519 attestation key (the one AI code attestations use). keyId is the first 16 hex characters of the SHA-256 of the public key’s PEM.

Checkpoints are stored in PostgreSQL and are append-only: there is no route that changes or deletes one, a database trigger refuses updates and deletes (except when the environment itself is deleted), and each one is recorded in the audit log (provenance.checkpoint). When the signing key is rotated, the old public key is kept, so older checkpoints still verify; each checkpoint names the key that signed it.

A record that was changed together with every chain hash after it still fails: the signed checkpoint’s lastChainHash no longer matches. Records after the latest checkpoint are protected by the chain head until the next checkpoint signs them.

Lawful removal: erasure and retention

Two things remove records lawfully, and verification tells them apart from tampering:

• Erasure. When a person’s data is erased (Offboarding → person data), each of their chained records first leaves a tombstone in PostgreSQL: its seq, record_hash, prev_chain_hash and chain_hash, the reason (erasure) and the id of an audit event (provenance.erasure) recording the erasure — no content and nothing about the person. Tombstones are append-only, like checkpoints. If a tombstone cannot be written, the record is not deleted. Verification reads a tombstoned seq as erased (lawful), links the chain through the tombstone’s hashes and lists the audit events; a tombstone whose audit event does not exist is reported as tampering.
• Retention. Records are kept for seven years. The retention horizon is the last checkpoint issued more than the retention period ago: every record it covers was written before then, so the table’s TTL may have removed it. A missing seq at or below the horizon is expired by policy; checkpoints over expired ranges still have their signatures checked, and the chain resumes from the horizon checkpoint’s lastChainHash.

Verification sorts what it finds into five categories: ok, erased (lawful), expired by policy, missing or tampered, and signature failure. Only the last two make ok false or fail the AI actions have provenance control.

Verify on the server

Detect & respond → Agent activity → Verify integrity checks the records in the page’s time range. The API takes a range of sequence numbers or a time window (at most 100,000 records); with neither, it checks the latest 100,000:

curl -s -H "Authorization: Bearer $SHIELD360_API_KEY" \
  "$SHIELD360_URL/api/provenance/verify?from=1&to=50000"

{
  "data": {
    "ok": false,
    "from": "1",
    "to": "50000",
    "checked": 49999,
    "firstBrokenSeq": "1734",
    "categories": { "ok": 49997, "erased": 2, "expired": 0, "writeFailed": 0, "tampered": 1, "signatureFailures": 0 },
    "counts": { "missing": 1, "duplicate": 0, "record_hash": 0, "link": 0, "chain_hash": 0, "tombstone_unaudited": 0 },
    "issues": [{ "kind": "missing", "seq": "1734" }],
    "erased": { "count": 2, "auditEventIds": ["cm1x…"] },
    "expired": { "count": 0, "throughSeq": null },
    "checkpoints": { "checked": 12, "valid": 12, "failures": [] },
    "uncheckpointed": 0,
    "headMismatch": false,
    "beyondHead": 0
  }
}

Field	Meaning
categories	Records that verified, were lawfully erased, expired by policy, were lost on write (writeFailed) or were tampered with, and checkpoints whose signature failed. Only tampering and signature failures make ok false.
firstBrokenSeq	The first record where something is wrong.
counts.missing	Sequence numbers with no record (removed).
counts.duplicate	Sequence numbers with more than one record (inserted).
counts.record_hash	Records whose content no longer hashes to their record_hash (changed).
counts.link	Records whose prev_chain_hash is not the previous record’s chain_hash (reordered or replaced).
counts.chain_hash	Records (or tombstones) whose chain_hash is wrong.
counts.tombstone_unaudited	Tombstones with no matching audit event (provenance.erasure for an erasure, provenance.write_failed for a failed write).
erased, expired	Lawful removals: the erasures’ audit event ids, and the retention horizon.
checkpoints.failures	Checkpoints that failed: unknown_key, signature, payload_mismatch, count, chain_mismatch, record_missing or not_contiguous.
uncheckpointed	Records chained but not yet in a signed checkpoint.
headMismatch, beyondHead	The latest record disagrees with the chain head, or records claim a place after it.

Malformed ranges are refused with 400. With recording switched off the result is { "ok": true, "empty": true, "disabled": true, … }.

Verify an export offline

Every export carries seq, record_hash, prev_chain_hash and chain_hash for each record. The JSON export adds an integrity section with the environment id, the checkpoints covering the records, the public keys that signed them, the tombstones in the range and the retention horizon; each CSV row carries its covering checkpoint’s payload, signature and public key.

tools/provenance/verify-export.mjs checks an export without Shield360 or a network connection (Node.js 18 or later, no dependencies):

# Export by sequence range, so the chain has no gaps
curl -s -H "Authorization: Bearer $SHIELD360_API_KEY" \
  "$SHIELD360_URL/api/provenance/export?format=json&from=1&to=50000" -o provenance.json

node tools/provenance/verify-export.mjs provenance.json --trusted-key shield360-provenance.pem

VERIFIED: 50000 chained records (json)
  record hashes recomputed: yes
  checkpoints: 12 of 12 valid; 0 records not covered by one

The script recomputes every record hash (JSON only; a CSV leaves out hashed fields, so for a CSV it checks the chain and checkpoints), every chain link between consecutive records, and every checkpoint’s signature and ends. Pass --trusted-key with a public key you obtained separately — from your Shield360 administrator (the environment’s attestation key, also returned as publicKeyPem by POST /api/ai-code/attestations), or an earlier export you kept — so a file that brings its own key is not trusted on its word; repeat it for rotated keys. --json prints the result as JSON. It exits 0 when everything verifies, 1 when a check fails and 2 when the file cannot be read. It reports the same categories as the server: erased records link through their tombstones (the erasures’ audit event ids are listed; whether those events exist can only be checked on the server), and records at or below the retention horizon are expired by policy. In an export by sequence range any other missing record fails; an export by time window can have gaps, because records are numbered in the order they arrived, and the script reports them without failing.

What is not recorded

• Prompts, responses, tool results and file contents (unless Data retention stores them, and then in traces, not here).
• Requests the AI proxy let through without acting on them, and SDK apps’ model calls.
• Exit codes where the agent does not report one (Claude Code’s Bash tool usually does not).

Related

Data retention

What Shield360 stores, including tokenized actions.

Coding agents

Install the hooks that report agents’ actions.