Send GenAI Traces

AI Observability works with any app or agent that exports OpenTelemetry traces with GenAI spans. You don’t install a KloudMate SDK. Turn on the OpenTelemetry support your framework already has, or add an instrumentation library, and point the exporter at KloudMate.

Pick your framework

Framework	Instrumentation	Guide
Claude Code and Cowork	Built in	Claude Code and Cowork
Claude API and SDK	Built in	Claude API and SDK
OpenAI SDK (Python, Node.js)	OpenTelemetry instrumentation	OpenAI SDK
OpenAI Agents SDK	OpenInference	OpenAI Agents SDK
OpenAI Codex	Built in	OpenAI Codex
CrewAI	OpenLLMetry	CrewAI
Google ADK	Built in	Google ADK
LangChain and LangGraph	OpenInference	LangChain and LangGraph
LlamaIndex	OpenInference	LlamaIndex
Mastra	Built in	Mastra
Pydantic AI	Built in	Pydantic AI
Vercel AI SDK	Built in	Vercel AI SDK

Other libraries work too, as long as they emit spans that follow the OpenTelemetry GenAI semantic conventions. That includes the OpenTelemetry instrumentations for other model providers and the other frameworks that OpenLLMetry and OpenInference instrument.

Point the exporter at KloudMate

Most OpenTelemetry SDKs read the standard exporter variables:

export OTEL_SERVICE_NAME="support-agent"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.kloudmate.com:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=YOUR_KLOUDMATE_API_KEY"

Create the API key in Settings → Ingest Keys. KloudMate expects the raw key in the Authorization header, with no Bearer prefix.
Don’t add /v1/traces to OTEL_EXPORTER_OTLP_ENDPOINT, because the exporter appends it. When a framework asks for a traces URL instead, use https://otel.kloudmate.com:4318/v1/traces.
If the KloudMate agent runs on the same machine, set the endpoint to http://localhost:4318 and leave out the header.

AI Observability reads traces only. Some instrumentations write prompts and responses to log events by default. Each framework guide shows how to put them on spans instead.

What KloudMate reads from a span

If you write spans by hand, set these attributes. Older names, such as gen_ai.system, are renamed to the current ones at ingest.

Attribute	Used for
`gen_ai.operation.name`	Required on every GenAI span. `chat`, `generate_content`, `text_completion`, and `embeddings` are LLM calls. `invoke_agent` starts a turn. `execute_tool` is a tool call.
`gen_ai.request.model`, `gen_ai.provider.name`	The Models and Providers tables, and the price lookup.
`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`	Token volume and cost. Cached input and reasoning tokens are counted as part of these totals.
`gen_ai.usage.cache_read.input_tokens`	Cache Hit Rate, and the cache price in the cost estimate.
`gen_ai.response.finish_reasons`	Cut off on runs and the cut-off count on Overview.
`server.address`	The Host column, and the price lookup for calls through a gateway such as OpenRouter.
`gen_ai.agent.name`	The run name and the Agents table. Set it on the model calls too, or the agent’s cost can’t be added up.
`gen_ai.conversation.id`	Groups the traces of one conversation.
`gen_ai.tool.name`, `gen_ai.tool.type`	The Tools tab. For MCP tools, set `mcp.method.name` to `tools/call` and `mcp.server.name` to the server name.
`gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.tool.call.arguments`, `gen_ai.tool.call.result`	The conversation view and tool arguments and results.
`error.type` and the span status	Error rate and the Error column.

Group a conversation

When a framework doesn’t send a conversation ID, each trace shows as its own run. Set gen_ai.conversation.id on a span that wraps the AI work, and KloudMate copies it to the GenAI spans underneath:

from opentelemetry import trace

tracer = trace.get_tracer("support-agent")

with tracer.start_as_current_span("handle_message") as span:
    span.set_attribute("gen_ai.conversation.id", chat_id)
    run_agent(message)

Use an ID that stays the same for the whole conversation, such as your chat or thread ID. Don’t use a user ID, because one user can have many conversations.

KloudMate copies the ID only to spans that arrive in the same export as the wrapping span. If a request runs longer than the exporter’s batch delay, 5 seconds by default, set OTEL_BSP_SCHEDULE_DELAY (in milliseconds) above the request’s duration.

Check that data arrives

Run your app once, then open AI Observability → Traces. If the run doesn’t appear:

Check that the exporter sends traces over HTTP to port 4318, and that OTEL_EXPORTER_OTLP_PROTOCOL is http/protobuf.
Check that the Authorization header holds a valid key with no Bearer prefix.
Search for your service in Traces in the main sidebar. If the spans are there but AI Observability is empty, they don’t carry gen_ai.operation.name, so the instrumentation isn’t emitting GenAI spans.

Known limitations

Token counts are used as the framework reports them. Some instrumentations leave out cached or cache-write tokens. Missing cache reads make the cache hit rate read low and the cost read high. Missing cache writes make the cost read low.
Two instrumentations on the same call count it twice. For example, LangChain instrumentation plus OpenAI SDK instrumentation records each model call twice. Instrument one layer only.
Batch API calls are priced at the standard rate. No framework marks a call as a batch call.