Instrument Python Apps with OpenLLMetry

In this guide, we will walk you through the process of setting up and using OpenLLMetry in Python. You can use the following steps to instrument a Python program with OpenLLMetry and then emit and visualize its traces within KloudMate.

The instrumentation demonstrated in this guide lets OpenLLMetry capture and send OpenAI model KPIs to KloudMate, where there is a dedicated ‘Traces ’ section to visualize the captured data. In addition to this OpenLLMetry also has instrumentations for vector databases such as Pinecone , and LLM frameworks such as LangChain. Feel free to adapt the instructions to your preferred framework.

1. Prerequisites

Ensure that you have Python 3 installed on your local machine
Create an OpenAI account and get your OpenAI API key (here’s how)
Create a KloudMate account and generate your KloudMate API Key

The KloudMate API Key can be obtained by logging into your KloudMate account and navigating to Settings >> API Keys.

2. Example Application

For this setup, we will be using a basic Python program that uses OpenAI API. Adhere to the steps outlined below to set up the environment for the program.

3. Installation

Create a new directory and activate a virtual environment:

mkdir openLLMetry-getting-started
cd openLLMetry-getting-started
python3 -m venv venv
source ./venv/bin/activate

Next, install the traceloop-sdk and openai.

pip install traceloop-sdk openai

4. Instrumentation

Create a file named app.py and add the following code to it:

from traceloop.sdk import Traceloop
from openai import OpenAI
from traceloop.sdk.decorators import workflow, task
import time

client = OpenAI(api_key="<your-openai-api-key>")

Traceloop.init(
    app_name="your app name", 
    api_endpoint="https://otel.kloudmate.com:4318", 
    headers={"Authorization": "<your-kloudmate-api-key>"}
)

@task(name="joke_creation")
def create_joke():
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Tell me a joke about OpenTelemetry"}],
    )

    return completion.choices[0].message.content

@workflow(name="pirate_joke_generator")
def joke_workflow():
    eng_joke = create_joke()
    print(eng_joke + "\n\n")

joke_workflow()

time.sleep(10)

Ensure you replace <your-openai-api-key> and <your-kloudmate-api-key> with your actual API keys.

5. Run the Instrumented Program

Use the following command to run the program

python3 app.py

6. Visualize Traces

You will be able to view the LLM attributes as shown below:

You will also be able to visualize the traces and spans: