LLM Observability with KloudMate and OpenLLMetry

In this guide, we will walk you through the process of setting up and using OpenLLMetry in Python. You can use the following steps to instrument a Python program with OpenLLMetry and then emit and visualize its traces within KloudMate.

The instrumentation demonstrated in this guide lets OpenLLMetry capture and send OpenAI model KPIs to KloudMate, where there is a dedicated 'Traces' section to visualize the captured data. In addition to this OpenLLMetry also has instrumentations for vector databases such as Pinecone, and LLM frameworks such as LangChain. Feel free to adapt the instructions to your preferred framework.

Step 1: Prerequisites

Step 2: Example Application

For this setup, we will be using a basic Python program that uses OpenAI API. Adhere to the steps outlined below to set up the environment for the program.

Step 3: Installation

1. Create a new directory and activate a virtual environment:

2. Next, install the traceloop-sdk and openai.

Step 4: Instrumentation

1. Create a file named app.py and add the following code to it:

Step 5: Run the Instrumented Program

1. Use the following command to run the program

Step 6: Visualize the traces emitted to KloudMate

1. Login to KloudMate and navigate to the 'Traces' section.