Logs

Navigate to Logs (/logs), the one home for logging: a searchable, filterable explorer of ingested log records with a detail view per log entry. Traces (/logs/traces) sits beside it. Older links to /telemetry?tab=logs and /telemetry?tab=traces redirect here and keep their filters.

Both pages are included with Security or Observability. With Security alone you can read logs and traces. Saving views, defining log metrics, Explain with AI, background queries, the app picker and the full-page log and trace views need Observability.

Shield360 ingests OTel logs the same way it ingests traces and metrics: through the built-in OpenTelemetry Collector’s otlp receiver (gRPC :4317, HTTP :4318), which exports them into the otel_logs table in ClickHouse. Point your application’s log exporter at the same OTLP endpoint you use for traces and metrics to have logs show up here automatically - no separate pipeline is required.

An application that uses the Shield360 SDK with an API key sends its logs to the same account as its traces, once logs are turned on: see Application logs. They are off until you turn them on. Any OpenTelemetry logs exporter works too, pointed at <SHIELD360_URL>/api/ingest/otlp/v1/logs with the API key as a bearer token.

The logs list shows, newest first, per entry: Time, Severity, Service, Body, Trace ID, and Span ID. Each row is marked by its severity: red for error and fatal, amber for warn, blue for info, grey for debug and trace. The level is the record’s OpenTelemetry severity number when it has one, and otherwise its severity text.

Filter the list by:

• Time range, as on every Telemetry page.
• Services (the application’s service.name), Severities and Environments (deployment.environment), several at once.
• Search log messages: text anywhere in the message, ignoring case.
• Custom attributes: Log Attributes, Resource Attributes, Scope Attributes, or a top-level Field, with the keys seen in the time range offered as you type.

With no logs in the range, the page says how to send them, with the Python, TypeScript and OpenTelemetry settings.

Around the list

The Logs page and the Logs tab of an application’s page (Protect > Agents & access > Applications & agents, then the app) hold the same explorer. The tab is scoped to that app, as is the page once you pick an app under App. Around the list are:

• Facets on the left: the commonest values of Level, Source, Host and Environment in the time range and filters, with a count each, then the log fields the app’s records carry most (such as requestId or orderId). Select a value to show only those lines; select it again to clear it. Picking another value of the same field replaces it. Filter facets finds a facet by name.
• Log volume: lines per time bucket, stacked by level, so a burst of errors stands out. Table shows the same numbers as a list.
• Patterns: lines with the same shape grouped into one row. Numbers, IDs, UUIDs, hex strings, emails, quoted text and query strings are taken out, so order 42 synced and order 77 synced are one pattern, order <n> synced. Each row shows how many lines it covers, its share, the levels seen, when it was last seen and the latest example. Show these lines searches the list for the pattern’s fixed text.
• Live tail: the window follows the last 15 minutes and the list refreshes every few seconds, newest first. The facets and chart refresh every 30 seconds while it runs. Turn it off to go back to the range you had.

The facets, chart and patterns read the same filters as the list, so their counts agree with the rows. They need View logs, as the list does. A connected data source that cannot count values shows the list without them.

Explain with AI

Explain with AI in the explorer’s toolbar asks the AI analyst what the current selection (filters, search and range) means. It answers in three parts: What’s happening, Likely causes (each tied to the patterns that show it) and Check next.

The analyst reads only the number of lines at each level and the most frequent patterns, with values masked. It never reads a raw line. Names are replaced with aliases for the model and put back for you. Each explanation counts against the analyst’s budget and hourly limit. When the analyst is off, has no model, or has used its budget, the panel says so.

Saved views

Views in the explorer’s toolbar keeps a search, its filters and the range under a name. Save current view names what you are looking at now. Tick Share to let everyone who can view logs in this environment use it; otherwise only you see it. Picking a view puts its filters and range back. On an app’s Logs tab the view stays scoped to that app, so one view works on every app and on the Telemetry hub.

Only the person who saved a view can delete it. Each person can keep up to 50 views per environment. Views belong to the environment they were saved in, and are removed with it.

Logs and traces

A log written while a request is being handled carries that request’s trace and span ids. From a log, Open the span opens the trace view on the span it was written in, and This trace’s logs lists every log of the trace. In the trace view, the Logs tab lists the logs of the whole trace or of the open span, oldest first, with Open in Logs for the full list.

Log pipelines

Settings → Collection & deployment → Log pipelines holds the environment’s own steps for log lines. They run on every line as it arrives, in order, before anything is stored, on lines from SDK API keys and from enrolled collectors alike. Lines a self-hosted deployment sends straight to the bundled collector’s OTLP ports (:4317, :4318) skip them.

Step	What it does
Parse JSON	A line that is a JSON object becomes attributes, with nested keys joined by dots (three levels, 64 fields). Field that holds the message moves that field into the line’s text. Lines that are not JSON are left alone.
Parse with a regular expression	Each named group, (?P<name>…), becomes an attribute.
Parse with grok	%{PATTERN:attribute} pieces, for example %{IP:client.ip} %{METHOD:http.method} %{URIPATHPARAM:http.target} %{STATUS:http.status_code}. Patterns: WORD, NOTSPACE, SPACE, DATA, GREEDYDATA, INT, NUMBER, BASE16NUM, IP, IPV4, IPV6, HOSTNAME, IPORHOST, USER, EMAILADDRESS, UUID, URIPATH, URIPARAM, URIPATHPARAM, HTTPDATE, TIMESTAMP_ISO8601, LOGLEVEL, QUOTEDSTRING, METHOD, STATUS.
Rename attributes	Moves an attribute to a new name, or copies it with Keep the original.
Drop lines	Lines that match every condition are not stored. A drop step must have a condition.
Redact	Replaces email addresses, IPv4 addresses, payment card numbers, bearer tokens, JSON web tokens, UUIDs or your own pattern, in the text, the string attributes or both.
Set level	Sets the level to a fixed value, or reads it from an attribute (warning, err, 17 and other common spellings are understood).

Any step can have conditions (Only when): the text, level, service or an attribute contains, is, is not, starts with, is present, or matches a pattern. Conditions are combined with AND.

Patterns use RE2 syntax. Matching always takes time in proportion to the line, so no pattern can stall ingest; lookaround and backreferences are not supported. Attributes whose names start shield360. are Shield360’s own and cannot be written by a step.

Preview runs the steps on up to 50 lines you paste, without saving or storing anything. Save pipeline stores a new version; ingest uses it within 30 seconds. Versions lists the last 20 saved, and Load into editor puts one back to save again. Saving needs the permission to change what is collected, and each save is recorded in the audit log. If someone else saved since you opened the editor, the save is refused and you are asked to reload.

Nothing is lost silently. A line is only left out when a drop step matches it. The settings page counts, for the version running now, the lines seen, dropped, with a failed step and with a skipped step, and names the last step that failed.

• A step that fails on a line is skipped for that line, and the line is stored marked shield360.pipeline.failed with the step ids. A redaction step that fails replaces the text it could not check.
• Each parsing step and matches condition reads at most 16 KB of a field. A longer field skips that parser, and the line is marked shield360.pipeline.skipped = size.
• Each request has a time budget (250 ms unless SHIELD360_LOG_PIPELINE_BUDGET_MS is set). Lines left when it is spent are stored with only their redaction steps run, marked shield360.pipeline.skipped = budget.

Search the explorer for those attributes to find the lines concerned. Secret scrubbing and Data retention still apply after the pipeline; redaction steps add to them and never replace them.

Log-based metrics

Metrics in the explorer’s toolbar defines numbers computed from this environment’s logs every minute or five minutes:

• Measure: Count of lines, or the sum, average, lowest, highest, median, 95th or 99th percentile of a numeric log attribute (such as duration_ms). Lines where the attribute is missing or not a number are left out of an attribute measure.
• Filters: the same fields and comparisons as log health alert rules: service, severity (including is at least), environment, scope, text in the message, or an attribute.
• Split by: up to three of service, severity, environment, scope or an attribute. The 100 largest groups of each bucket are kept.
• Metric key: the name dashboards use, such as checkout.payment_errors. It cannot change once the metric exists.

New metric from this search starts with the explorer’s search text as a message filter. A new metric fills in the last hour on its first run. Changing what a metric measures (its measure, attribute, filters, groups or interval) starts its series again from the last hour. Each run also computes the two newest buckets again, so logs that arrive a little late are counted. After a long outage a metric resumes from the last hour rather than filling the gap.

Points are kept 90 days in the environment’s ClickHouse table shield360_log_metric_points (metric, bucket_start, labels, value, samples), holding numbers and group values, never log text. Dashboards can query it, for example:

SELECT bucket_start, labels['service'] AS service, value
FROM shield360_log_metric_points FINAL
WHERE metric = 'checkout.payment_errors' AND bucket_start >= now() - INTERVAL 1 DAY
ORDER BY bucket_start

To alert on a metric, add an alert rule of type Log-based metric. Each metric shows its last day as a sparkline, how far it is computed, and why its last run failed if it did (too many logs to read in time, the table not yet created, or a timeout); the next run tries the same buckets again. Viewing metrics needs the permission to view logs; defining, changing or deleting them needs the permission to manage alerts and is recorded in the audit log. An environment holds up to 50 metrics. The job runs on the platform’s scheduled-job worker (LOG_METRICS_SCHEDULE, every minute by default).

What is stored

Secrets in log messages and attributes (API keys, tokens, passwords, connection strings) are removed before anything is stored. An application’s own log message is not treated as prompt or response content, so it is kept whatever the content switches in Data retention say, but personal data in it (email addresses, phone numbers, card numbers) is masked unless Data retention keeps personal data. AI events sent as log records follow the content switches, as on traces.

To alert on logs — a service’s error log rate, or a message that contains a piece of text — see Log health. Settings → Usage shows the size of the logs sent this month.

Log detail view

Opening a log entry shows a severity badge, meta tiles (Time, Service, Event, Cost, Tokens), a log summary grid (body, duration, input/output/cache tokens, request ID, model, source, session, prompt ID, terminal, user - with links out to the parent trace and span where available), and a tabbed attributes panel for Log Attributes, Resource Attributes, and Scope Attributes.

Surrounding lines

Below the summary, Surrounding lines shows what the same service wrote just before and after the entry, oldest first, with the entry highlighted, like grep -C. When the log names a pod, container or host, switch to This pod (or This container, This host) to see only that instance’s lines. When the entry’s collector recorded which stream it was written to (log.iostream, stdout or stderr, as container log collectors do), that choice also keeps to the same stream. Each other line opens its own detail, and Show more widens both sides, up to 100 lines each way. Lines are looked for within an hour either side.

---

Traces & Exceptions

Jump from a log’s linked trace/span IDs into the full trace detail view