Monitor OpenAI using OpenTelemetry

Shield360 uses OpenTelemetry instrumentation to help you monitor LLM applications built using OpenAI models. This includes tracking performance, token usage, costs, and how users interact with the application.

The integration is compatible with:

• OpenAI Python SDK client >=1.92.0
• OpenAI Typescript SDK client >=1.13.0
• OpenAI Go SDK, via Shield360’s InstrumentedClient wrapper

Get started

Python / TypeScript

Auto-instrumentation means you don’t have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding Shield360 in your application, all the necessary monitoring configurations are automatically set up.

Install Shield360

Open your command line or terminal and run:

Python

pip install shield360

Typescript

npm install shield360

Initialize Shield360 in your Application

Python

Zero Code Instrumentation

Perfect for existing applications - no code modifications needed:

Via CLI Arguments

# Configure via CLI arguments
shield360-instrument \
  --service-name my-ai-app \
  --environment production \
  --otlp-endpoint YOUR_OTEL_ENDPOINT \
  python your_app.py

Via Environment Variables

# Configure via environment variables
export OTEL_SERVICE_NAME=my-ai-app
export OTEL_DEPLOYMENT_ENVIRONMENT=production
export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

# Run with zero code changes
shield360-instrument python your_app.py

Note

Perfect for: Legacy applications, production systems where code changes need approval, quick testing, or when you want to add observability without touching existing code.

One-Line Instrumentation

Via Function Parameters

import shield360

shield360.init(otlp_endpoint="YOUR_OTEL_ENDPOINT")

Via Environment Variables

Add the following two lines to your application code:

import shield360

shield360.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

Typescript

One-Line Instrumentation (SDK)

Via Function Parameters

import shield360 from "shield360"

shield360.init({ otlpEndpoint: "YOUR_OTEL_ENDPOINT" })

Via Environment Variables

Add the following two lines to your application code:

import shield360 from "shield360"

shield360.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

Replace: YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using Shield360 and a local OTel Collector.

To send metrics and traces to other Observability tools, refer to the supported destinations.

For more advanced configurations and application use cases, visit the SDK configuration reference or TypeScript SDK reference.

Go

The Go SDK wraps your OpenAI client with an InstrumentedClient that automatically emits traces and metrics for every API call - with zero changes to your application logic.

The integration supports:

• Chat completions (standard and streaming)
• Embeddings
• Image generation (dall-e-2, dall-e-3)

Install the Go SDK

Open your terminal and run:

Latest Version

go get github.com/ThinkfleetAI/shield360-go

Specific Version

go get github.com/ThinkfleetAI/shield360-go@v1.2.3

Replace v1.2.3 with the version you want to install.

Initialize Shield360

Add this once at the start of your application (e.g. in main()):

Via Config Struct

import (
    "context"
    shield360 "github.com/ThinkfleetAI/shield360-go"
)

if err := shield360.Init(shield360.Config{
    OtlpEndpoint:    "YOUR_OTEL_ENDPOINT",
    ApplicationName: "my-ai-app",
    Environment:     "production",
}); err != nil {
    log.Fatal(err)
}
defer shield360.Shutdown(context.Background())

Via Environment Variable

import (
    "context"
    shield360 "github.com/ThinkfleetAI/shield360-go"
)

if err := shield360.Init(shield360.Config{
    ApplicationName: "my-ai-app",
}); err != nil {
    log.Fatal(err)
}
defer shield360.Shutdown(context.Background())

Then set your OTLP endpoint via environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

Replace YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 for a local Shield360 deployment.

Create an instrumented client

Replace your existing OpenAI client creation with the Shield360 instrumented client:

import "github.com/ThinkfleetAI/shield360-go/instrumentation/openai"

client := openai.NewClient("your-openai-api-key")

Optional configuration:

// Custom base URL (e.g. for Azure OpenAI or local models)
client := openai.NewClient("your-api-key",
    openai.WithBaseURL("https://your-custom-endpoint/v1"),
)

Use the client

Use the instrumented client exactly as you would a normal OpenAI client:

Chat completion:

resp, err := client.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
    Model: "gpt-4o",
    Messages: []openai.Message{
        {Role: "system", Content: "You are a helpful assistant."},
        {Role: "user", Content: "What is OpenTelemetry?"},
    },
    MaxTokens:   256,
    Temperature: 0.7,
})
if err != nil {
    return err
}
fmt.Println(resp.Choices[0].Message.Content)

Streaming:

stream, err := client.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
    Model: "gpt-4o",
    Messages: []openai.Message{
        {Role: "user", Content: "Tell me a story."},
    },
})
if err != nil {
    return err
}
defer stream.Close()

for {
    chunk, err := stream.Recv()
    if err == io.EOF {
        break
    }
    if err != nil {
        return err
    }
    if len(chunk.Choices) > 0 {
        fmt.Print(chunk.Choices[0].Delta.Content)
    }
}

Embeddings:

resp, err := client.CreateEmbedding(ctx, openai.EmbeddingRequest{
    Model: "text-embedding-3-small",
    Input: "The quick brown fox",
})

Image generation:

resp, err := client.CreateImage(ctx, openai.ImageRequest{
    Model:  "dall-e-3",
    Prompt: "A futuristic city skyline at sunset",
    N:      1,
    Size:   "1024x1024",
})

What gets collected

Every call to the instrumented client automatically records:

Data	Attribute
Operation name	gen_ai.operation.name
Model requested	gen_ai.request.model
Model used	gen_ai.response.model
Input tokens	gen_ai.usage.input_tokens
Output tokens	gen_ai.usage.output_tokens
Estimated cost	gen_ai.usage.cost
Finish reason	gen_ai.response.finish_reasons
Tool calls	gen_ai.tool.name, gen_ai.tool.call.id
Time to first token	gen_ai.server.time_to_first_token (streaming)
Time per output token	gen_ai.server.time_per_output_token (streaming)

Metrics emitted:

• gen_ai.client.token.usage - token usage histogram (input/output)
• gen_ai.client.operation.duration - total operation duration
• gen_ai.server.time_to_first_token - TTFT for streaming
• gen_ai.client.operation.time_to_first_chunk - client-side TTFT
• gen_ai.client.operation.time_per_output_chunk - per-chunk latency
• gen_ai.server.request.duration - estimated server processing time

---

Quickstart: LLM Observability

Production-ready AI monitoring setup in 2 simple steps with zero code changes

Configuration

Configure the Shield360 SDK according to you requirements.

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks