Skip to content
Shield360 Shield360 Shield360

Elastic

To directly send OpenTelemetry metrics and traces generated by Shield360 from your AI Application to Elastic, follow the below steps.

1. Get your Credentials

  1. Sign in to your Elastc account.
  2. Open the side navigation and click on APM under Observability.
  3. Make sure the APM Server is running
  4. In the APM Agents section, Select OpenTelemetry and directly jump to Step 5 (Configure OpenTelemetry in your application):
  5. Copy and save the configuration value for OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS

2. Instrument your application

For direct integration into your Python applications:

import shield360


shield360.init(
  otlp_endpoint="YOUR_ELASTIC_OTEL_URL",
  otlp_headers="YOUR_ELASTIC_OTEL_AUTH"
)

Replace:

  1. YOUR_ELASTIC_OTEL_URL with the OTEL_EXPORTER_OTLP_ENDPOINT value you copied in Step 1.
    • Example: https://38xxx.apm.us-central1.gcp.cloud.es.io:443
  2. YOUR_ELASTIC_OTEL_AUTH with the OTEL_EXPORTER_OTLP_HEADERS value you copied in Step 1.
    • Example: Authorization=Bearer%20<APIKEY>

Refer to the Shield360 SDK configuration reference for more advanced configurations and use cases.

3. Import the pre-built Dashboard

  1. Copy the dashboard NDJSON provided in the accordion named Dashboard below and save it in a file (.ndjson).
  2. Log into your Elastic Instance.
  3. Go to Stack Management > Saved Objects.
  4. Click Import and upload your file containing the dashboard NDJSON.
  5. Click Import and you should have the dashboard available.
Dashboard
{"attributes":{"allowHidden":false,"allowNoIndex":true,"fieldAttrs":"{}","fieldFormatMap":"{\"trace.id\":{\"id\":\"url\",\"params\":{\"urlTemplate\":\"apm/link-to/trace/{{value}}\",\"labelTemplate\":\"{{value}}\"}},\"transaction.id\":{\"id\":\"url\",\"params\":{\"urlTemplate\":\"apm/link-to/transaction/{{value}}\",\"labelTemplate\":\"{{value}}\"}},\"transaction.duration.us\":{\"id\":\"duration\",\"params\":{\"inputFormat\":\"microseconds\",\"outputFormat\":\"asMilliseconds\",\"showSuffix\":true,\"useShortSuffix\":true,\"outputPrecision\":2,\"includeSpaceWithSuffix\":true}}}","fields":"[]","name":"APM","runtimeFieldMap":"{}","sourceFilters":"[]","timeFieldName":"@timestamp","title":"traces-apm*,apm-*,logs-apm*,apm-*,metrics-apm*,apm-*"},"coreMigrationVersion":"8.8.0","created_at":"2024-06-02T10:27:43.786Z","id":"apm_static_data_view_id_default","managed":false,"references":[],"type":"index-pattern","typeMigrationVersion":"8.0.0","updated_at":"2024-06-02T10:27:43.786Z","version":"WzUsMV0="}
{"attributes":{"color":"#a6a6fb","description":"","name":"genai"},"coreMigrationVersion":"8.8.0","created_at":"2024-06-02T11:00:42.608Z","id":"dd5a07f8-eb90-41fa-92e1-7e0535e92881","managed":false,"references":[],"type":"tag","typeMigrationVersion":"8.0.0","updated_at":"2024-06-02T11:00:42.608Z","version":"WzU5MiwxXQ=="}
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"},"optionsJSON":"{\"useMargins\":true,\"syncColors\":false,\"syncCursor\":true,\"syncTooltips\":false,\"hidePanelTitles\":false}","panelsJSON":"[{\"type\":\"visualization\",\"gridData\":{\"x\":0,\"y\":0,\"w\":48,\"h\":9,\"i\":\"0ab1b11b-153c-4a27-a342-940e2253dacb\"},\"panelIndex\":\"0ab1b11b-153c-4a27-a342-940e2253dacb\",\"embeddableConfig\":{\"savedVis\":{\"id\":\"\",\"title\":\"\",\"description\":\"\",\"type\":\"markdown\",\"params\":{\"fontSize\":12,\"openLinksInNewTab\":false,\"markdown\":\"---\\n# GenAI Observability\\n\\nThis dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [Shield360](https://shield360.ai).\\n\\n---\"},\"uiState\":{},\"data\":{\"aggs\":[],\"searchSource\":{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}}},\"hidePanelTitles\":true,\"description\":\"\",\"enhancements\":{}},\"title\":\"\"},{\"type\":\"lens\",\"gridData\":{\"x\":0,\"y\":9,\"w\":10,\"h\":7,\"i\":\"fc9f8451-fa92-46b2-a47a-94c45a7338aa\"},\"panelIndex\":\"fc9f8451-fa92-46b2-a47a-94c45a7338aa\",\"embeddableConfig\":{\"attributes\":{\"title\":\"\",\"visualizationType\":\"lnsLegacyMetric\",\"type\":\"lens\",\"references\":[{\"type\":\"index-pattern\",\"id\":\"apm_static_data_view_id_default\",\"name\":\"indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f\"}],\"state\":{\"visualization\":{\"layerId\":\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\",\"accessor\":\"d3a9a186-1a69-4130-a4e5-d863954e517c\",\"layerType\":\"data\",\"colorMode\":\"Background\",\"palette\":{\"type\":\"palette\",\"name\":\"status\",\"params\":{\"name\":\"status\",\"reverse\":false,\"rangeType\":\"number\",\"rangeMin\":0,\"rangeMax\":1.5,\"progression\":\"fixed\",\"stops\":[{\"color\":\"#209280\",\"stop\":0},{\"color\":\"#d6bf57\",\"stop\":0.66},{\"color\":\"#cc5642\",\"stop\":1.33}],\"steps\":3,\"continuity\":\"all\",\"maxSteps\":5}}},\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filters\":[],\"datasourceStates\":{\"formBased\":{\"layers\":{\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\":{\"columns\":{\"d3a9a186-1a69-4130-a4e5-d863954e517c\":{\"label\":\"Total Successful GenAI Requests\",\"dataType\":\"number\",\"operationType\":\"count\",\"isBucketed\":false,\"scale\":\"ratio\",\"sourceField\":\"gen_ai.total.requests\",\"params\":{\"format\":{\"id\":\"number\",\"params\":{\"decimals\":0}},\"emptyAsNull\":true},\"customLabel\":true}},\"columnOrder\":[\"d3a9a186-1a69-4130-a4e5-d863954e517c\"],\"incompleteColumns\":{},\"sampling\":1,\"indexPatternId\":\"apm_static_data_view_id_default\"}},\"currentIndexPatternId\":\"apm_static_data_view_id_default\"},\"indexpattern\":{\"layers\":{}},\"textBased\":{\"layers\":{}}},\"internalReferences\":[],\"adHocDataViews\":{}}},\"hidePanelTitles\":true,\"enhancements\":{}},\"title\":\"Total Successful GenAI Requests\"},{\"type\":\"lens\",\"gridData\":{\"x\":10,\"y\":9,\"w\":11,\"h\":7,\"i\":\"4230457e-9ccc-4fcd-8ae6-5942d0e86360\"},\"panelIndex\":\"4230457e-9ccc-4fcd-8ae6-5942d0e86360\",\"embeddableConfig\":{\"attributes\":{\"title\":\"\",\"visualizationType\":\"lnsLegacyMetric\",\"type\":\"lens\",\"references\":[{\"type\":\"index-pattern\",\"id\":\"apm_static_data_view_id_default\",\"name\":\"indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f\"}],\"state\":{\"visualization\":{\"layerId\":\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\",\"accessor\":\"d3a9a186-1a69-4130-a4e5-d863954e517c\",\"layerType\":\"data\",\"colorMode\":\"Background\",\"palette\":{\"type\":\"palette\",\"name\":\"status\",\"params\":{\"name\":\"status\",\"reverse\":false,\"rangeType\":\"number\",\"rangeMin\":0,\"rangeMax\":1.5,\"progression\":\"fixed\",\"stops\":[{\"color\":\"#209280\",\"stop\":0},{\"color\":\"#d6bf57\",\"stop\":0.66},{\"color\":\"#cc5642\",\"stop\":1.33}],\"steps\":3,\"continuity\":\"all\",\"maxSteps\":5}}},\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filters\":[],\"datasourceStates\":{\"formBased\":{\"layers\":{\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\":{\"columns\":{\"d3a9a186-1a69-4130-a4e5-d863954e517c\":{\"label\":\"Total Successful VectorDB Requests\",\"dataType\":\"number\",\"operationType\":\"count\",\"isBucketed\":false,\"scale\":\"ratio\",\"sourceField\":\"db.total.requests\",\"params\":{\"format\":{\"id\":\"number\",\"params\":{\"decimals\":0}},\"emptyAsNull\":true},\"customLabel\":true}},\"columnOrder\":[\"d3a9a186-1a69-4130-a4e5-d863954e517c\"],\"incompleteColumns\":{},\"sampling\":1,\"indexPatternId\":\"apm_static_data_view_id_default\"}},\"currentIndexPatternId\":\"apm_static_data_view_id_default\"},\"indexpattern\":{\"layers\":{}},\"textBased\":{\"layers\":{}}},\"internalReferences\":[],\"adHocDataViews\":{}}},\"hidePanelTitles\":true,\"enhancements\":{}},\"title\":\"Total Successful GenAI Requests\"},{\"type\":\"lens\",\"gridData\":{\"x\":21,\"y\":9,\"w\":11,\"h\":7,\"i\":\"921e9588-c4f9-48f2-8842-fb94fa871789\"},\"panelIndex\":\"921e9588-c4f9-48f2-8842-fb94fa871789\",\"embeddableConfig\":{\"attributes\":{\"title\":\"\",\"visualizationType\":\"lnsLegacyMetric\",\"type\":\"lens\",\"references\":[{\"type\":\"index-pattern\",\"id\":\"apm_static_data_view_id_default\",\"name\":\"indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f\"}],\"state\":{\"visualization\":{\"layerId\":\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\",\"accessor\":\"d3a9a186-1a69-4130-a4e5-d863954e517c\",\"layerType\":\"data\",\"colorMode\":\"Background\",\"palette\":{\"type\":\"palette\",\"name\":\"status\",\"params\":{\"name\":\"status\",\"reverse\":false,\"rangeType\":\"number\",\"rangeMin\":0,\"rangeMax\":1.5,\"progression\":\"fixed\",\"stops\":[{\"color\":\"#209280\",\"stop\":0},{\"color\":\"#d6bf57\",\"stop\":0.66},{\"color\":\"#cc5642\",\"stop\":1.33}],\"steps\":3,\"continuity\":\"all\",\"maxSteps\":5}}},\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filters\":[],\"datasourceStates\":{\"formBased\":{\"layers\":{\"b7f3b353-d163-4371-9c4b-ae3c1a76712f\":{\"columns\":{\"d3a9a186-1a69-4130-a4e5-d863954e517c\":{\"label\":\"Total Usage Tokens\",\"dataType\":\"number\",\"operationType\":\"sum\",\"sourceField\":\"gen_ai.usage.total_tokens\",\"isBucketed\":false,\"scale\":\"ratio\",\"params\":{\"format\":{\"id\":\"number\",\"params\":{\"decimals\":0}},\"emptyAsNull\":true},\"customLabel\":true}},\"columnOrder\":[\"d3a9a186-1a69-4130-a4e5-d863954e517c\"],\"incompleteColumns\":{},\"sampling\":1,\"indexPatternId\":\"apm_static_data_view_id_default\"}},\"currentIndexPatternId\":\"apm_static_data_view_id_default\"},\"indexpattern\":{\"layers\":{}},\"textBased\":{\"layers\":{}}},\"internalReferences\":[],\"adHocDataViews\":{}}},\"hidePanelTitles\":true,\"enhancements\":{}},\"title\":\"Total Successful GenAI Requests\"},{\"type\":\"lens\",\"gridData\":{\"x\":32,\"y\":9,\"w\":16,\"h\":7,\"i\":\"4fc53e67-90b4-45a2-9eb2-7c7daadcff6e\"},\"panelIndex\":\"4fc53e67-90b4-45a2-9eb2-7c7daadcff6e\",\"embeddableConfig\":{\"attributes\":{\"title\":\"\",\"visualizationType\":\"lnsGauge\",\"type\":\"lens\",\"references\":[{\"type\":\"index-pattern\",\"id\":\"apm_static_data_view_id_default\",\"name\":\"indexpattern-datasource-layer-cedecfe6-b308-4a3b-8d5d-b3856f68bc5c\"}],\"state\":{\"visualization\":{\"shape\":\"horizontalBullet\",\"layerId\":\"cedecfe6-b308-4a3b-8d5d-b3856f68bc5c\",\"layerType\":\"data\",\"ticksPosition\":\"auto\",\"labelMajorMode\":\"auto\",\"metricAccessor\":\"13325455-54c7-4511-97bc-b40379cde51c\",\"palette\":{\"name\":\"custom\",\"type\":\"palette\",\"params\":{\"steps\":3,\"name\":\"custom\",\"reverse\":false,\"rangeType\":\"percent\",\"rangeMin\":null,\"rangeMax\":null,\"progression\":\"fixed\",\"stops\":[{\"color\":\"#00ff0080\",\"stop\":33.33},{\"color\":\"#eaff0080\",\"stop\":66.66},{\"color\":\"#E7664C\",\"stop\":100}],\"colorStops\":[{\"color\":\"#00ff0080\",\"stop\":null},{\"color\":\"#eaff0080\",\"stop\":33.33},{\"color\":\"#E7664C\",\"stop\":66.66}],\"continuity\":\"all\",\"maxSteps\":5}},\"colorMode\":\"palette\"},\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filters\":[],\"datasourceStates\":{\"formBased\":{\"layers\":{\"cedecfe6-b308-4a3b-8d5d-b3856f68bc5c\":{\"columns\":{\"13325455-54c7-4511-97bc-b40379cde51c\":{\"label\":\"Avg Usage Cost\",\"dataType\":\"number\",\"operationType\":\"average\",\"sourceField\":\"gen_ai.usage.cost\",\"isBucketed\":false,\"scale\":\"ratio\",\"params\":{\"emptyAsNull\":true,\"format\":{\"id\":\"number\",\"params\":{\"decimals\":1,\"suffix\":\"$\",\"compact\":false}}},\"customLabel\":true}},\"columnOrder\":[\"13325455-54c7-4511-97bc-b40379cde51c\"],\"incompleteColumns\":{},\"sampling\":1,\"indexPatternId\":\"apm_static_data_view_id_default\"}},\"currentIndexPatternId\":\"apm_static_data_view_id_default\"},\"indexpattern\":{\"layers\":{}},\"textBased\":{\"layers\":{}}},\"internalReferences\":[],\"adHocDataViews\":{}}},\"hidePanelTitles\":true,\"enhancements\":{}},\"title\":\"\"}]","timeRestore":false,"title":"GenAI Observability","version":1},"coreMigrationVersion":"8.8.0","created_at":"2024-06-03T04:45:39.460Z","id":"e6384f67-5e58-43f2-b2a2-44c16e207dbd","managed":false,"references":[{"id":"apm_static_data_view_id_default","name":"fc9f8451-fa92-46b2-a47a-94c45a7338aa:indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f","type":"index-pattern"},{"id":"apm_static_data_view_id_default","name":"4230457e-9ccc-4fcd-8ae6-5942d0e86360:indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f","type":"index-pattern"},{"id":"apm_static_data_view_id_default","name":"921e9588-c4f9-48f2-8842-fb94fa871789:indexpattern-datasource-layer-b7f3b353-d163-4371-9c4b-ae3c1a76712f","type":"index-pattern"},{"id":"apm_static_data_view_id_default","name":"4fc53e67-90b4-45a2-9eb2-7c7daadcff6e:indexpattern-datasource-layer-cedecfe6-b308-4a3b-8d5d-b3856f68bc5c","type":"index-pattern"},{"id":"dd5a07f8-eb90-41fa-92e1-7e0535e92881","name":"tag-ref-dd5a07f8-eb90-41fa-92e1-7e0535e92881","type":"tag"}],"type":"dashboard","typeMigrationVersion":"8.9.0","updated_at":"2024-06-03T04:45:39.460Z","version":"WzE3MSwxXQ=="}
{"excludedObjects":[],"excludedObjectsCount":0,"exportedCount":3,"missingRefCount":0,"missingReferences":[]}