Oodle
To send OpenTelemetry metrics generated by Shield360 from your AI Application to Oodle, follow the below steps.
1. Get your Oodle Credentials
- Sign in to your Oodle account
- Get your Oodle credentials:
- OODLE_ENDPOINT: Your Oodle metrics ingestion endpoint
- INSTANCE_ID: Your Oodle instance identifier
- API_KEY: Your Oodle API key for authentication
2. Configure OpenTelemetry Collector
Install OpenTelemetry Collector (if not already running): For detailed installation instructions, refer to the OpenTelemetry Collector Documentation.
Configure the Collector for Oodle integration:
Complete Collector Configuration
receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318
processors: batch:
exporters: otlphttp/oodle: metrics_endpoint: "https://OODLE_ENDPOINT/v1/otlp/metrics/INSTANCE_ID" headers: X-API-KEY: "API_KEY" debug: verbosity: detailed sampling_initial: 5 sampling_thereafter: 200
service: pipelines: metrics: receivers: [otlp] processors: [batch] exporters: [otlphttp/oodle] traces: receivers: [otlp] processors: [batch] exporters: [debug]Replace:
OODLE_ENDPOINTwith your Oodle endpointINSTANCE_IDwith your Oodle instance IDAPI_KEYwith your Oodle API key
3. Instrument your application
For direct integration into your Python applications:
import shield360
shield360.init( otlp_endpoint="YOUR_OTELCOL_URL:4318")Replace:
YOUR_OTELCOL_URL:4318with your OpenTelemetry Collector HTTP endpoint.- Example (Local):
http://127.0.0.1:4318 - Example (Remote):
http://otel-collector.company.com:4318
- Example (Local):
import shield360
shield360.init()Set these environment variables:
export OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTELCOL_URL:4318"export OTEL_SERVICE_NAME="my-ai-service"export OTEL_DEPLOYMENT_ENVIRONMENT="production"Replace:
YOUR_OTELCOL_URL:4318with your OpenTelemetry Collector HTTP endpoint.- Example (Local):
http://127.0.0.1:4318 - Example (Remote):
http://otel-collector.company.com:4318
- Example (Local):
Refer to the Shield360 SDK configuration reference for more advanced configurations and use cases.
For zero-code auto-instrumentation via command line:
# Using CLI argumentsshield360-instrument \ --otlp-endpoint "YOUR_OTELCOL_URL:4318" \ --service-name "my-ai-service" \ --deployment-environment "production" \ python app.pyReplace:
YOUR_OTELCOL_URL:4318with your OpenTelemetry Collector HTTP endpoint.- Example (Local):
http://127.0.0.1:4318 - Example (Remote):
http://otel-collector.company.com:4318
- Example (Local):
# Set environment variables (takes precedence over CLI args)export OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTELCOL_URL:4318"export OTEL_SERVICE_NAME="my-ai-service"export OTEL_DEPLOYMENT_ENVIRONMENT="production"
# Run your applicationshield360-instrument python app.pyReplace:
YOUR_OTELCOL_URL:4318with your OpenTelemetry Collector HTTP endpoint.- Example (Local):
http://127.0.0.1:4318 - Example (Remote):
http://otel-collector.company.com:4318
- Example (Local):
Refer to the Shield360 SDK configuration reference for more advanced configurations and use cases.
4. Import Oodle Dashboard
Once your LLM application is instrumented, you can visualize the metrics in Oodle using our pre-built dashboard:
- Log into your Oodle Instance
- Navigate to Dashboards: Click Dashboards in the primary menu
- Import Dashboard: Click New → Import in the drop-down menu
- Copy the Dashboard JSON: Use the JSON provided in the accordion below
- Paste and Import: Paste the JSON directly into the text area and click Import
- Save the Dashboard: Save the imported dashboard
GenAI Observability Dashboard JSON
{ "annotations": { "list": [ { "builtIn": 1, "datasource": { "type": "grafana", "uid": "-- Grafana --" }, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "type": "dashboard" } ] }, "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 1, "id": 7995, "links": [], "panels": [ { "fieldConfig": { "defaults": {}, "overrides": [] }, "gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 }, "id": 61, "options": { "code": { "language": "plaintext", "showLineNumbers": false, "showMiniMap": false }, "content": "<img\n src=\"https://www.oodle.ai/img/logo.svg\"\n width=\"100%\"\n height=\"100%\">", "mode": "html" }, "pluginVersion": "12.3.1", "title": "", "transparent": true, "type": "text" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "blue", "mode": "palette-classic-by-name" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "red", "value": 0 }, { "color": "#EAB839", "value": 10 }, { "color": "#6ED0E0", "value": 100 } ] }, "unit": "none" }, "overrides": [] }, "gridPos": { "h": 4, "w": 5, "x": 4, "y": 0 }, "id": 22, "options": { "colorMode": "background", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "value_and_name", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))", "legendFormat": "Total Requests", "range": true, "refId": "A" } ], "title": "", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "purple", "mode": "shades" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "short" }, "overrides": [] }, "gridPos": { "h": 4, "w": 5, "x": 9, "y": 0 }, "id": 3, "options": { "colorMode": "background", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(increase(gen_ai_client_token_usage_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))", "legendFormat": "Total Usage Tokens", "range": true, "refId": "A" } ], "title": "", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "blue", "mode": "shades" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "#EAB839", "value": 0.5 }, { "color": "red", "value": 1 } ] }, "unit": "currencyUSD" }, "overrides": [] }, "gridPos": { "h": 4, "w": 5, "x": 14, "y": 0 }, "id": 5, "options": { "colorMode": "background", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "inverted", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval])) \n/ \n(sum(last_over_time(gen_ai_usage_cost_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval])))", "legendFormat": "Avg Usage Cost", "range": true, "refId": "A" } ], "title": "", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "blue", "mode": "shades" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [] }, "gridPos": { "h": 4, "w": 5, "x": 19, "y": 0 }, "id": 2, "options": { "colorMode": "background", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval]))", "legendFormat": "Total Usage Cost", "range": true, "refId": "A" } ], "title": "", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 2, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [] }, "gridPos": { "h": 7, "w": 12, "x": 0, "y": 4 }, "id": 55, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": true, "mode": "multi", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "increase(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[1h]) / 1000", "legendFormat": "{{gen_ai_provider_name}} / {{gen_ai_request_model}}", "range": true, "refId": "A" } ], "title": "Cost Trend Analysis", "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 2, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "percent" }, "overrides": [] }, "gridPos": { "h": 4, "w": 12, "x": 12, "y": 4 }, "id": 56, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "multi", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(rate(gen_ai_client_operation_duration_count{http_status_code!~\"4..|5..\", telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) / sum(rate(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) * 100", "hide": true, "legendFormat": "Total Requests", "range": true, "refId": "A" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "( \n sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) \n - \n (sum(increase(gen_ai_client_operation_duration_count{error_type=~\".+\", telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) or vector(0)) \n) \n/ \nsum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) \n* 100 ", "hide": true, "legendFormat": "__auto", "range": true, "refId": "B" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", error_type=~\".+\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) \n/ \nsum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) \n* 100 ", "legendFormat": "__auto", "range": true, "refId": "C" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum by(le, gen_ai_provider_name) (rate(gen_ai_client_operation_duration_bucket{service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval]))) ", "hide": true, "legendFormat": "{{gen_ai_provider_name}}", "range": true, "refId": "D" } ], "title": "Error Rate %", "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "align": "left", "cellOptions": { "mode": "basic", "type": "gauge" }, "footer": { "reducers": [] }, "inspect": false }, "decimals": 0, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "short" }, "overrides": [ { "matcher": { "id": "byName", "options": "Metric" }, "properties": [ { "id": "displayName", "value": "Model" }, { "id": "custom.width", "value": 150 }, { "id": "custom.cellOptions", "value": { "type": "auto" } } ] } ] }, "gridPos": { "h": 7, "w": 12, "x": 12, "y": 8 }, "id": 65, "options": { "cellHeight": "md", "frameIndex": 0, "showHeader": true, "sortBy": [ { "desc": true, "displayName": "Model" } ] }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "topk(5, sum by(gen_ai_request_model) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))", "instant": true, "legendFormat": "{{gen_ai_request_model}}", "refId": "A" } ], "title": "Top 5 Models by Requests", "transformations": [ { "id": "seriesToRows", "options": {} }, { "id": "organize", "options": { "excludeByName": { "Time": true }, "renameByName": { "Metric": "Model", "Value": "Requests" } } } ], "type": "table" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "yellow", "value": 2 }, { "color": "red", "value": 5 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 4, "w": 12, "x": 0, "y": 11 }, "id": 58, "options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum by (le, gen_ai_provider_name) (rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])))", "legendFormat": "{{gen_ai_provider_name}} P95", "range": true, "refId": "A" } ], "title": "Time to First Token", "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "smooth", "lineWidth": 2, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": true, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "reqps" }, "overrides": [] }, "gridPos": { "h": 10, "w": 24, "x": 0, "y": 15 }, "id": 64, "options": { "legend": { "calcs": [ "mean", "max", "lastNotNull" ], "displayMode": "table", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "multi", "sort": "desc" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum by(gen_ai_provider_name, gen_ai_request_model) (rate(gen_ai_client_operation_duration_count[$__rate_interval]))", "legendFormat": "{{gen_ai_provider_name}} / {{gen_ai_request_model}}", "range": true, "refId": "A" } ], "title": "Request Rate by Platform & Model", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "bars", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "smooth", "lineWidth": 2, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "always", "showValues": false, "spanNulls": true, "stacking": { "group": "A", "mode": "normal" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "none" }, "overrides": [] }, "gridPos": { "h": 6, "w": 24, "x": 0, "y": 25 }, "id": 6, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "avg(gen_ai_client_token_usage_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\", gen_ai_token_type=\"input\"})", "legendFormat": "Prompt Tokens", "range": true, "refId": "A" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "avg(gen_ai_client_token_usage_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\", gen_ai_token_type=\"output\"})", "legendFormat": "Completion Tokens", "range": true, "refId": "B" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "avg(gen_ai_usage_cost_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"})", "legendFormat": "Usage Cost", "range": true, "refId": "C" } ], "title": "Average Token Consumption vs. Average Usage Cost Comparison", "transparent": true, "type": "timeseries" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 31 }, "id": 31, "panels": [], "title": "Cost", "type": "row" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "align": "left", "cellOptions": { "mode": "basic", "type": "gauge" }, "footer": { "reducers": [] }, "inspect": false }, "decimals": 4, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [ { "matcher": { "id": "byName", "options": "Metric" }, "properties": [ { "id": "displayName", "value": "Model" }, { "id": "custom.width", "value": 150 }, { "id": "custom.cellOptions", "value": { "type": "auto" } } ] } ] }, "gridPos": { "h": 8, "w": 12, "x": 0, "y": 32 }, "id": 67, "options": { "cellHeight": "md", "frameIndex": 0, "showHeader": true, "sortBy": [ { "desc": true, "displayName": "Last *" } ] }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "topk(5, sum by(gen_ai_provider_name) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))", "instant": true, "legendFormat": "{{gen_ai_provider_name}}", "refId": "A" } ], "title": "Top 5 Providers by Cost", "transformations": [ { "id": "seriesToRows", "options": {} }, { "id": "organize", "options": { "excludeByName": { "Time": true }, "renameByName": { "Metric": "Model", "Value": "Cost (USD)" } } } ], "type": "table" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "align": "left", "cellOptions": { "mode": "basic", "type": "gauge" }, "footer": { "reducers": [] }, "inspect": false }, "decimals": 4, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [ { "matcher": { "id": "byName", "options": "Metric" }, "properties": [ { "id": "displayName", "value": "Model" }, { "id": "custom.width", "value": 150 }, { "id": "custom.cellOptions", "value": { "type": "auto" } } ] } ] }, "gridPos": { "h": 8, "w": 12, "x": 12, "y": 32 }, "id": 66, "options": { "cellHeight": "md", "frameIndex": 0, "showHeader": true, "sortBy": [ { "desc": true, "displayName": "Last *" } ] }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "topk(5, sum by(gen_ai_request_model) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))", "instant": true, "legendFormat": "{{gen_ai_request_model}}", "refId": "A" } ], "title": "Top 5 Models by Cost", "transformations": [ { "id": "seriesToRows", "options": {} }, { "id": "organize", "options": { "excludeByName": { "Time": true }, "renameByName": { "Metric": "Model", "Value": "Cost (USD)" } } } ], "type": "table" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "decimals": 4, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 0, "y": 40 }, "id": 29, "options": { "legend": { "calcs": [ "median" ], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "multi", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum by(gen_ai_provider_name) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval]))", "legendFormat": "{{gen_ai_provider_name}}", "range": true, "refId": "A" } ], "title": "Total Cost By System", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 100, "gradientMode": "opacity", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "decimals": 2, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "currencyUSD" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 12, "y": 40 }, "id": 32, "options": { "legend": { "calcs": [ "median" ], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "multi", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum(increase(gen_ai_usage_cost_sum{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by(gen_ai_request_model) / 1000", "legendFormat": "__auto", "range": true, "refId": "A" } ], "title": "Total Cost By Model", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "decimals": 0, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "blue", "value": 80 } ] }, "unit": "short" }, "overrides": [] }, "gridPos": { "h": 6, "w": 8, "x": 0, "y": 48 }, "id": 70, "options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum by(deployment_environment) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))", "instant": true, "legendFormat": "{{deployment_environment}}", "refId": "A" } ], "title": "Requests by Environment", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "purple", "mode": "shades" }, "decimals": 0, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 } ] }, "unit": "short" }, "overrides": [] }, "gridPos": { "h": 6, "w": 8, "x": 8, "y": 48 }, "id": 69, "options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum by(gen_ai_provider_name) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))", "instant": true, "legendFormat": "{{gen_ai_provider_name}}", "refId": "A" } ], "title": "Requests by Platform", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "fixedColor": "green", "mode": "fixed" }, "decimals": 0, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "short" }, "overrides": [] }, "gridPos": { "h": 6, "w": 8, "x": 16, "y": 48 }, "id": 68, "options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "percentChangeColorMode": "standard", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showPercentChange": false, "textMode": "auto", "wideLayout": true }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "sum by(gen_ai_operation_name) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))", "instant": true, "legendFormat": "{{gen_ai_operation_name}}", "refId": "A" } ], "title": "Requests by Type", "transparent": true, "type": "stat" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "continuous-GrYlRd" }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 54 }, "id": 57, "options": { "displayMode": "lcd", "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": false }, "maxVizHeight": 300, "minVizHeight": 16, "minVizWidth": 8, "namePlacement": "auto", "orientation": "horizontal", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showUnfilled": true, "sizing": "auto", "valueMode": "color" }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (le, gen_ai_provider_name))", "legendFormat": "{{gen_ai_provider_name}}", "range": true, "refId": "A" } ], "title": "Latency Distribution", "transparent": true, "type": "bargauge" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 62 }, "id": 46, "panels": [], "title": "Latency", "type": "row" }, { "fieldConfig": { "defaults": {}, "overrides": [] }, "gridPos": { "h": 2, "w": 12, "x": 0, "y": 63 }, "id": 59, "options": { "code": { "language": "plaintext", "showLineNumbers": false, "showMiniMap": false }, "content": "### By System", "mode": "markdown" }, "pluginVersion": "12.3.1", "title": "", "type": "text" }, { "fieldConfig": { "defaults": {}, "overrides": [] }, "gridPos": { "h": 2, "w": 12, "x": 12, "y": 63 }, "id": 60, "options": { "code": { "language": "plaintext", "showLineNumbers": false, "showMiniMap": false }, "content": "### By Model", "mode": "markdown" }, "pluginVersion": "12.3.1", "title": "", "type": "text" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "continuous-GrYlRd" }, "decimals": 2, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 0, "y": 65 }, "id": 38, "options": { "displayMode": "gradient", "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": false }, "maxVizHeight": 300, "minVizHeight": 16, "minVizWidth": 8, "namePlacement": "auto", "orientation": "auto", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showUnfilled": true, "sizing": "auto", "valueMode": "color" }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, \n sum by(le, gen_ai_provider_name) (\n increase(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])\n )\n)", "instant": true, "legendFormat": "__auto", "refId": "A" } ], "title": "Tokens Generation Duration (P95)", "transformations": [ { "id": "seriesToRows", "options": {} }, { "id": "sortBy", "options": { "sort": [ { "desc": true, "field": "Value" } ] } } ], "transparent": true, "type": "bargauge" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "continuous-GrYlRd" }, "decimals": 2, "mappings": [], "min": 0, "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 12, "y": 65 }, "id": 37, "options": { "displayMode": "gradient", "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": false }, "maxVizHeight": 300, "minVizHeight": 16, "minVizWidth": 8, "namePlacement": "auto", "orientation": "auto", "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showUnfilled": true, "sizing": "auto", "valueMode": "color" }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, \n sum by(le, gen_ai_request_model) (\n increase(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])\n )\n)", "instant": true, "legendFormat": "__auto", "refId": "A" } ], "title": "Tokens Generation Duration (P95)", "transformations": [ { "id": "seriesToRows", "options": {} }, { "id": "sortBy", "options": { "sort": [ { "desc": true, "field": "Value" } ] } } ], "transparent": true, "type": "bargauge" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 0, "y": 73 }, "id": 41, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum(rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_provider_name, le))", "legendFormat": "__auto", "range": true, "refId": "A" } ], "title": "Time to first token", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 8, "w": 12, "x": 12, "y": 73 }, "id": 42, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum(rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_request_model, le))", "legendFormat": "__auto", "range": true, "refId": "A" } ], "title": "Time to first token", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 9, "w": 12, "x": 0, "y": 81 }, "id": 39, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_provider_name, le))", "legendFormat": "__auto", "range": true, "refId": "A" } ], "title": "Tokens Generation Duration", "transparent": true, "type": "timeseries" }, { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 9, "w": 12, "x": 12, "y": 81 }, "id": 43, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.1", "targets": [ { "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "editorMode": "code", "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"shield360\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_request_model, le))", "legendFormat": "__auto", "range": true, "refId": "A" } ], "title": "Tokens Generation Duration", "transparent": true, "type": "timeseries" } ], "preload": false, "refresh": "30s", "schemaVersion": 42, "tags": [ "ai-observability-integration", "ai-generated" ], "templating": { "list": [ { "current": { "text": "oodle", "value": "P4033CA274A2C2DFE" }, "includeAll": false, "label": "Metrics data source", "name": "metrics_datasource", "options": [], "query": "prometheus", "refresh": 1, "regex": "(?!grafanacloud-usage|grafanacloud-ml-metrics).+", "type": "datasource" }, { "current": { "text": "All", "value": [ "$__all" ] }, "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "definition": "label_values(service_name)", "includeAll": true, "label": "service_name", "multi": true, "name": "service_name", "options": [], "query": { "qryType": 1, "query": "label_values(service_name)", "refId": "PrometheusVariableQueryEditor-VariableQuery" }, "refresh": 2, "type": "query" }, { "current": { "text": "All", "value": [ "$__all" ] }, "datasource": { "type": "prometheus", "uid": "${metrics_datasource}" }, "definition": "label_values(deployment_environment)", "includeAll": true, "label": "deployment_environment", "multi": true, "name": "deployment_environment", "options": [], "query": { "qryType": 1, "query": "label_values(deployment_environment)", "refId": "PrometheusVariableQueryEditor-VariableQuery" }, "refresh": 2, "sort": 1, "type": "query" } ] }, "time": { "from": "now-24h", "to": "now" }, "timepicker": {}, "timezone": "browser", "title": "GenAI Observability", "uid": "cdiz9piuoa3ggc", "version": 1}Your Shield360-instrumented AI applications will appear in Oodle with comprehensive metrics including:
- LLM Request Rates: Monitor request volume and patterns
- Usage Costs: Track AI model costs and budget allocation
- Token Consumption: Analyze input/output token usage
- Model Performance: Compare performance across different models
- VectorDB Operations: Monitor database requests and operations
- Application Breakdown: View metrics by application and environment