Send LlamaIndex Traces to KloudMate

The OpenInference instrumentation for LlamaIndex creates a span for each agent run, model call, embedding, and tool call. Send those spans to KloudMate to see each run in AI Observability.

Use OpenInference rather than LlamaIndex’s own llama-index-observability-otel package. Its spans don’t use the GenAI attribute names, so model calls and tokens don’t appear in AI Observability.

Prerequisites

A KloudMate API key, from Settings → Ingest Keys.
Python 3.10 or later.

Install the instrumentation and the OpenTelemetry launcher:

pip install openinference-instrumentation-llama-index opentelemetry-distro opentelemetry-exporter-otlp-proto-http

Point the exporter at KloudMate and start your app with opentelemetry-instrument, which loads the LlamaIndex instrumentation automatically:

export OTEL_SERVICE_NAME="rag-service"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.kloudmate.com:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=YOUR_KLOUDMATE_API_KEY"

opentelemetry-instrument python app.py

Install the instrumentation and the OpenTelemetry SDK:

pip install openinference-instrumentation-llama-index opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

Set up a tracer provider and instrument LlamaIndex at startup:

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

LlamaIndexInstrumentor().instrument(tracer_provider=provider)

OTLPSpanExporter() reads the endpoint and header from the same OTEL_EXPORTER_OTLP_* variables shown in the other tab. Call provider.shutdown() before a script exits, so the last spans are sent.

KloudMate expects the raw API key in the Authorization header, with no Bearer prefix. If the KloudMate agent runs on the same machine, set the endpoint to http://localhost:4318 and leave out the header.

Group a conversation

Wrap each call in using_session() with an ID that stays the same for the whole conversation:

from openinference.instrumentation import using_session

with using_session("chat-42"):
    response = await agent.run(question)

Control what’s captured

OpenInference records prompts, responses, and retrieved documents by default. To leave out inputs and outputs, set these variables before your app starts:

export OPENINFERENCE_HIDE_INPUTS="true"
export OPENINFERENCE_HIDE_OUTPUTS="true"

Token counts, models, and latency are still recorded. The OpenInference configuration lists the finer-grained settings, such as OPENINFERENCE_HIDE_RETRIEVAL_DOCUMENTS.

Check the data

Run a query once, then open AI Observability → Traces. The run appears with its model calls, tokens, and estimated cost.