Behavior engine
The behavior engine watches every app, service and agent in each environment and compares what it does now with what it normally does. It needs no setup: it reads the spans, logs and runtime protection events Shield360 already ingests. Open it at Telemetry → Behavior. Active findings also appear on the command view.
What it watches
Section titled “What it watches”Every 10 minutes, per app and environment:
| Signal | Source |
|---|---|
| Request rate, error rate, median and p95 latency | Root and server spans |
| AI tokens and estimated spend | gen_ai.usage.* on spans |
| Log volume and error logs | Log records (ERROR and above) |
| New error log patterns | A sample of error logs, normalized into templates |
| Runtime protection events by attack type, new IAST findings | RASP and IAST reports (Security module) |
| Feature flag changes, new releases, settings changes | Flag history, service.version, the audit log |
Runtime protection and IAST signals need the Security module. Everything else works on every plan.
How normal is learned
Section titled “How normal is learned”The engine keeps one number per app, signal and 10 minutes in a small table in each environment’s own database (shield360_behavior_series, kept 35 days). It never recomputes from raw spans.
- Baseline. The median and the median absolute deviation (MAD) of the same hour on the same weekday over the last four weeks. With too little of that, the same hour on any day, then the last 24 hours. Robust statistics mean a past incident does not shift what counts as normal.
- Warm-up. An app is not judged until it has a day of history. A new app shows nothing at first.
- Flat series. When every value is the same, the spread is floored at 10% of the median and at a per-signal minimum, so a small change is not treated as infinitely unusual.
What it finds
Section titled “What it finds”| Finding | Meaning |
|---|---|
| Spike / drop | The latest 10 minutes are at least 4 robust deviations from normal (6 on low sensitivity, 3 on high), and by more than the signal’s smallest meaningful change. Drops are reported only for traffic and log volume. |
| Drift | Latency or error rate has been moderately above normal for 30 minutes. |
| Prediction | A line fitted through the last two hours (R² ≥ 0.6) will cross the signal’s limit within the horizon (60 minutes by default). The limit is 5% for error rate, otherwise the top of the normal band. |
| New pattern | An error log template the app has never produced before, after the app’s first day of logs. |
Ratios and latency are judged only in buckets with at least 20 requests.
Each finding is linked to the nearest change before it, within an hour: a feature flag change for the same app, a new release of the app, or a settings change in the environment. The link is a lead, not a verdict.
Findings and alerts
Section titled “Findings and alerts”Findings are kept per environment as open, acknowledged or resolved. A finding is resolved automatically when its app and signal were measured and stayed normal for 30 minutes (six hours for a new log pattern), or when it has not been seen at all for a day. A missing measurement never counts as recovery. A finding that comes back is reopened, however long ago it was resolved.
Findings raise alerts of kind behavior.finding through the alert pipeline, so routing rules, channels, quiet hours and maintenance windows apply as usual. By default only critical findings alert. A finding alerts again only after the cooldown (60 minutes by default) or when its severity rises. At most 10 alerts per environment per run.
A finding’s details
Section titled “A finding’s details”Select a finding to open its details:
- What was observed against the app’s usual value and spread, how far off it was, and which history the usual range came from.
- The recent 10-minute buckets drawn against that range.
- For a prediction, the level it is heading for and when. For a new log pattern, the masked template.
- The change just before it (a lead, not a verdict).
From there, open the app’s logs or traces for the window from an hour before the finding began to 15 minutes after it was last seen, or ask Rex to investigate. Acknowledge and Resolve work in the panel too. A link to the page with ?finding=<id> (alerts carry one) opens the panel directly.
Settings
Section titled “Settings”The engine is on for every organization by default. People who manage alerts can change these settings for the whole organization on the Behavior tab:
- Turn the engine off.
- Choose which severity alerts, or none.
- Set the sensitivity (low, normal or high).
- Set the alert cooldown.
Forecast
Section titled “Forecast”Telemetry → Behavior → Forecast shows where one app’s signal is heading. Name the app (its service.name), pick a signal (requests, error rate, latency p50 or p95, log volume, error logs, AI tokens or AI spend) and how far ahead to look (up to 7 days).
- The chart shows the signal per hour for the last two weeks and the forecast, with a band that holds the actual value about 80% of the time.
- The forecast is Holt-Winters (triple exponential smoothing) with a daily season and a damped trend, fitted to the engine’s own hourly series. With less than two days of history it is a damped trend line, and with almost none, the average so far. The method is named under the chart.
- Threshold: the first hour the forecast passes it. Use it for capacity (requests an hour your service can take), cost (AI spend an hour) or latency.
- Error-rate target (error rate only, for example 99.9): the share of the 30-day error budget already used, how fast the last hour burned it (1× is exactly the rate the target allows), and in how many hours the forecast uses it up. Each hour counts the same, whatever its traffic, so a quiet hour with a few errors weighs as much as a busy one.
- AI spend: the forecast total for the next 24 hours and for the whole window. For the month and its budget, see budgets, which forecast spend to the end of the period.
The forecast reads only the engine’s series (one small read of the environment’s database); it never reads raw spans or logs. To be alerted before a breach, add a forecast rule.
Access
Section titled “Access”- Reading findings needs the permission to view telemetry.
- Acknowledging, resolving and changing settings needs the permission to manage alerts.
- Runtime protection findings also need the permission to view runtime protection.
Limits and safety
Section titled “Limits and safety”- Each environment’s run makes at most 12 queries. Each query is read-only, stops after 15 seconds and is refused above 50 million rows read. Raw reads cover at most the last hour and the busiest 200 apps per 10 minutes. When that cap is reached, or a raw table is missing, apps that were not read count as unknown for that interval, not as silent.
- The engine uses the environment’s background database login where it has one, so it never competes with ingest or dashboards. An overloaded environment is skipped for that run and logged.
- A run stops starting environments after four minutes. The next run starts where it left off.
- No raw log body is stored. Log templates have numbers, ids, addresses, quoted values, emails and other personal data replaced before they are saved. Findings carry numbers, a masked template and the linked change.