Instrument a Python LLM App with OpenLLMetry

This guide walks you through instrumenting a real LLM application (a small LangChain agent that calls tools) and viewing its traces in KloudMate. By the end you’ll see each agent run as a single trace: the model’s reasoning, every tool call, token usage per call, and where the time went.

You’ll build a customer-support agent for an online store. The agent answers a question by deciding which tools to call (look up an order, check a return policy), then writing a reply.

OpenLLMetry does the instrumentation. It’s an OpenTelemetry-native SDK from Traceloop that auto-instruments LangChain and the underlying model calls, so you add a few lines of setup and change nothing in the agent itself.

Prerequisites

Python 3.10 or later.
An OpenAI API key (where to find it).
A KloudMate workspace API key, from Settings → Ingest Keys.

Step 1: Set up the project

Create a directory, activate a virtual environment, and install the packages:

mkdir support-agent && cd support-agent
python3 -m venv venv
source ./venv/bin/activate
pip install traceloop-sdk langchain langchain-openai

traceloop-sdk brings in OpenLLMetry and the OpenTelemetry exporter. The LangChain packages are the app itself.

Step 2: Point OpenLLMetry at KloudMate

OpenLLMetry exports over OTLP/HTTP, so it needs your KloudMate endpoint and API key. Set them as environment variables so you don’t hard-code secrets:

export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
export KM_API_KEY="YOUR_KLOUDMATE_API_KEY"

You’ll initialize OpenLLMetry once, before the agent runs. A single Traceloop.init() call wires up the exporter and instruments LangChain and OpenAI automatically:

import os
from traceloop.sdk import Traceloop

Traceloop.init(
    app_name="support-agent",
    api_endpoint="https://otel.kloudmate.com:4318",  # the SDK appends /v1/traces
    headers={"Authorization": os.environ["KM_API_KEY"]},
    disable_batch=True,  # export each span as it finishes while you're testing
)

Step 3: Build the agent

Create app.py. The agent has two tools backed by in-memory data so the example runs without a database. In a real app these would be API or database calls.

import os
from traceloop.sdk import Traceloop

# Initialize OpenLLMetry BEFORE importing/using LangChain so the calls are traced.
Traceloop.init(
    app_name="support-agent",
    api_endpoint="https://otel.kloudmate.com:4318",
    headers={"Authorization": os.environ["KM_API_KEY"]},
    disable_batch=True,
)

from langchain.agents import create_agent
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

# --- Mock data (stand-ins for real APIs) ---
ORDERS = {
    "4412": {"status": "Delivered", "category": "audio", "total": 99.95},
}
RETURN_POLICIES = {
    "audio": {"window_days": 30, "restocking_fee_pct": 10},
    "default": {"window_days": 30, "restocking_fee_pct": 0},
}

# --- Tools ---
@tool
def get_order_status(order_id: str) -> dict:
    """Look up an order's status, category, and total by its order ID."""
    order = ORDERS.get(order_id.strip().lstrip("#"))
    return order or {"error": f"No order found with id {order_id}."}

@tool
def get_return_policy(category: str) -> dict:
    """Get the return window and restocking fee for a product category."""
    return RETURN_POLICIES.get(category.lower(), RETURN_POLICIES["default"])

TOOLS = [get_order_status, get_return_policy]

# --- Agent ---
SYSTEM_PROMPT = (
    "You are ShopMate, a concise customer-support agent. "
    "Use the tools to look up real order and policy data. Never guess order "
    "details or refund amounts. Give the customer a short, helpful answer."
)

def build_agent():
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    return create_agent(llm, tools=TOOLS, system_prompt=SYSTEM_PROMPT, name="support_agent")

if __name__ == "__main__":
    agent = build_agent()
    question = "I'd like to return order #4412. What's the policy and how much would I get back?"
    result = agent.invoke({"messages": [{"role": "user", "content": question}]})
    print(result["messages"][-1].content)

Step 4: Run it

python3 app.py

The agent answers the question, and OpenLLMetry exports the trace to KloudMate. Behind that one answer, the agent made several model calls and two tool calls, all captured as a single trace.

Step 5: View the trace in KloudMate

Open KloudMate, go to Traces, and filter by the service support-agent. Open the most recent trace.

You’ll see the full agent run as a waterfall: the agent run at the top, with the model calls and the get_order_status and get_return_policy tool calls nested underneath. Select any model span to read the exact prompt and response, the token usage, and the finish reason.

For a tour of everything KloudMate surfaces on an agent trace (the conversation transcript, token and cache breakdown, tool calls, and the AI Flow graph), see Agent Traces. The run also appears in AI Observability → Traces as support_agent Turn, with its token usage and estimated cost.

Step 6: Tag traces with a user and session

Production agents serve many customers across many conversations. Attach a user and session to every span so you can find one customer’s traces later, or follow a single conversation end to end. Set these association properties before each run:

Traceloop.set_association_properties(
    {"user_id": "cust_1042", "session_id": "sess_8f21"}
)
result = agent.invoke({"messages": [{"role": "user", "content": question}]})

KloudMate stores these on every span in the run, so you can filter and group traces by user_id or session_id when you investigate an issue. The session_id also becomes the conversation ID (gen_ai.conversation.id) in AI Observability.

Step 7: Prepare for production

Batch exports. Drop disable_batch=True so spans export in the background instead of one HTTP request per span.
Decide what to capture. OpenLLMetry records prompt and completion content by default. If your prompts carry sensitive data, set TRACELOOP_TRACE_CONTENT=false to record metadata (tokens, model, latency) without the message bodies.
Keep init first. Call Traceloop.init() before the rest of your app imports run, so every model and tool call is instrumented.