Send OpenAI SDK Traces to KloudMate

The OpenTelemetry project maintains an instrumentation for the OpenAI SDK that creates a span for each chat completion and embedding request, with the model, token counts, and, if you turn it on, the messages. Send those spans to KloudMate to see each call in AI Observability with its estimated cost.

Use this guide when your app calls the OpenAI SDK directly. If you build on an agent framework, follow the guide for that framework instead, so each model call is recorded once.

Prerequisites

A KloudMate API key, from Settings → Ingest Keys.
Python 3.10 or later, or Node.js 22.15 or later.

Install the instrumentation and the OpenTelemetry launcher:

pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-openai-v2

Set the exporter and content variables, then start your app with opentelemetry-instrument:

export OTEL_SERVICE_NAME="support-agent"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.kloudmate.com:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=YOUR_KLOUDMATE_API_KEY"
export OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental"
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="span_only"

opentelemetry-instrument python app.py

Set OTEL_EXPORTER_OTLP_PROTOCOL explicitly. The launcher defaults to gRPC, which doesn’t work with the HTTP endpoint on port 4318.

span_only puts prompts and responses on the spans, where AI Observability reads them. Without OTEL_SEMCONV_STABILITY_OPT_IN, the instrumentation follows an older version of the conventions and writes content to log events instead. To leave content out, omit OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT.

Install the OpenTelemetry SDK and the OpenAI instrumentation:

npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto @opentelemetry/instrumentation-openai

Create telemetry.ts, which starts the SDK before your app imports openai:

import { OpenAIInstrumentation } from '@opentelemetry/instrumentation-openai';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { NodeSDK } from '@opentelemetry/sdk-node';

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [new OpenAIInstrumentation()],
});
sdk.start();

Load it ahead of your app, for example node --import ./telemetry.js app.js, and set the same OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_* variables shown in the Python tab. The Node.js instrumentation also covers the Responses API. It writes message content to log events rather than spans, so the conversation view stays empty for these calls.

KloudMate expects the raw API key in the Authorization header, with no Bearer prefix. If the KloudMate agent runs on the same machine, set the endpoint to http://localhost:4318 and leave out the header.

Group a conversation

The instrumentation doesn’t know which conversation a call belongs to, so each call shows as its own run. To group the calls of one conversation, wrap them in a span that sets gen_ai.conversation.id:

from opentelemetry import trace

tracer = trace.get_tracer("support-agent")

with tracer.start_as_current_span("handle_message") as span:
    span.set_attribute("gen_ai.conversation.id", chat_id)
    reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages)

KloudMate copies the ID only to spans that arrive in the same export as the wrapping span. If a request runs longer than the exporter’s batch delay, 5 seconds by default, set OTEL_BSP_SCHEDULE_DELAY (in milliseconds) above the request’s duration.

Known limitations

The Python instrumentation covers Chat Completions and Embeddings. Calls to the Responses API aren’t traced.
The Python instrumentation doesn’t report cached input tokens, so the cache hit rate reads zero and the estimated cost doesn’t include the cache discount.

Check the data

Make one request, then open AI Observability → Traces. The call appears as Chat · gpt-4o-mini, or under your wrapping span’s name, with its tokens and estimated cost.