Infrastructure: servers, containers and processes

Infrastructure (/telemetry/infrastructure, a tab of Telemetry) shows the machines under your app, from OpenTelemetry metrics sent to Shield360 with your API key:

• Hosts with their latest CPU, memory, fullest disk and network in and out.
• Containers (Docker or Kubernetes) with their CPU and memory.
• Processes that run the SDK with system metrics on: CPU, memory and, for Node.js, event-loop delay.
• The top consumers: busiest hosts, fullest hosts and largest containers.
• Over time: any of these metrics for the busiest hosts, with your app’s error rate and p95 latency on the same time axis, so a CPU spike and a slow checkout line up.

Pick a service and a range (last hour to last 7 days). Error rate and latency over more than a day need a service chosen.

Send the metrics

There are two ways, and you can use both.

From the SDK

The SDKs can send the host’s and their own process’s CPU and memory with your traces. Nothing else to run.

import shield360

shield360.init(application_name="my-app", collect_system_metrics=True)

import Shield360 from "shield360";

Shield360.init({ applicationName: "my-app", collectSystemMetrics: true });

Or set SHIELD360_COLLECT_SYSTEM_METRICS=true. The TypeScript SDK sends system.cpu.utilization, system.memory.utilization, system.memory.usage, process.cpu.utilization, process.memory.usage and nodejs.eventloop.delay.p99, each with host.name, every 60 seconds (OTEL_EXPORTER_OTLP_METRICS_EXPORT_INTERVAL). Python uses the OpenTelemetry system metrics instrumentation (pip install opentelemetry-instrumentation-system-metrics).

From the OpenTelemetry Collector

For the whole server, its disks and network, containers, and log files, run the OpenTelemetry Collector Contrib (otelcol-contrib) beside your app. This config sends the host’s metrics and your app’s log files:

receivers:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      disk: {}
      network: {}
      load: {}

  filelog/app:
    include: [/var/log/my-app/*.log]
    start_at: end
    multiline:
      line_start_pattern: '^\S'

processors:
  memory_limiter:
    check_interval: 5s
    limit_percentage: 80
    spike_limit_percentage: 25
  resourcedetection:
    detectors: [system, env]
  resource:
    attributes:
      - key: service.name
        value: ${env:SERVICE_NAME}
        action: upsert
      - key: deployment.environment
        value: ${env:DEPLOYMENT_ENVIRONMENT}
        action: upsert
  batch:
    timeout: 10s

exporters:
  otlphttp/shield360:
    endpoint: ${env:SHIELD360_URL}/api/ingest/otlp
    headers:
      Authorization: Bearer ${env:SHIELD360_API_KEY}
    compression: gzip

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, resource, batch]
      exporters: [otlphttp/shield360]
    logs:
      receivers: [filelog/app]
      processors: [memory_limiter, resourcedetection, resource, batch]
      exporters: [otlphttp/shield360]

export SHIELD360_URL=https://app.shield360.ai   # your Shield360 address
export SHIELD360_API_KEY=...                       # Settings → API keys
export SERVICE_NAME=my-app                         # your app's service.name
export DEPLOYMENT_ENVIRONMENT=production
otelcol-contrib --config host.yaml

The full versions (with JSON log parsing and filesystem exclusions), and versions for Docker (docker_stats plus every container’s stdout/stderr) and systemd (journald), are in examples/app-monitoring in the Shield360 repository. Each passes otelcol validate with Collector Contrib v0.142.0.

Note

service.name decides which app the metrics belong to. Set it to the same name your SDK uses (application_name / applicationName) so Infrastructure, traces, logs and alerts line up.

Kubernetes

Run one collector per node as a DaemonSet. It sends each node’s metrics (hostmetrics), every pod’s and container’s CPU and memory (kubeletstats), and every container’s logs from /var/log/pods, with the pod, namespace and deployment added by the k8sattributes processor. service.name comes from the pod’s app.kubernetes.io/name label, else its deployment, else the container’s name.

kubectl create namespace shield360-monitoring
kubectl -n shield360-monitoring create secret generic shield360 \
  --from-literal=url=https://app.shield360.ai --from-literal=api-key=<your API key>
kubectl -n shield360-monitoring create configmap otel-collector --from-file=config.yaml=collector.yaml
kubectl apply -f daemonset.yaml

collector.yaml and daemonset.yaml (service account, the read-only cluster role kubeletstats and k8sattributes need, and the DaemonSet) are in examples/app-monitoring/kubernetes.

What is read

Shown as	Metric
Host CPU	system.cpu.utilization (from the collector: 1 − the average idle share across cores)
Host memory	system.memory.utilization, the used state
Disk	system.filesystem.utilization, the fullest mount
Network	system.network.io, bytes per second over the range
Container CPU	container.cpu.utilization (docker_stats)
Container memory	container.memory.usage.total (docker_stats) or container.memory.working_set (kubeletstats)
Process CPU and memory	process.cpu.utilization, process.memory.usage (SDKs)
Event-loop delay	nodejs.eventloop.delay.p99 (TypeScript SDK)

Metrics are kept as long as your other telemetry. Reads are bounded to the range you pick (at most 7 days) and stop after 15 seconds.

Alert on them

Add an alert rule of type Server, container or process metric. Templates start you at:

Template	Fires when
High CPU	a host’s CPU stays above 90% for five minutes
High memory	a host’s memory stays above 85% for five minutes
Disk almost full	a host’s fullest disk stays above 90% for ten minutes

“Every reading crosses it” (the default) fires only when every reading in the window is past the threshold, so a short spike does not page anyone; “the average crosses it” uses the window’s average. Rules are read every 15 seconds, raise one infra.metric alert per host or container (carrying its service), and resolve when the condition clears. Add infra.metric to an escalation policy to open incidents from them.