LLM Observability with KloudMate and OpenLLMetry

in this guide, we will walk you through the process of setting up and using openllmetry in python you can use the following steps to instrument a python program with openllmetry and then emit and visualize its traces within https //app kloudmate com the instrumentation demonstrated in this guide lets openllmetry capture and send openai model kpis to kloudmate, where there is a dedicated ' traces ' section to visualize the captured data in addition to this openllmetry also has instrumentations for vector databases such as pinecone , and llm frameworks such as langchain feel free to adapt the instructions to your preferred framework llm observability with kloudmate and traceloop step 1 prerequisites ensure that you have python 3 installed on your local machine create an openai account and get your openai api key ( https //help openai com/en/articles/4936850 where do i find my openai api key ) create a https //app kloudmate com/signup and generate your kloudmate api key the kloudmate api key can be obtained by logging into your kloudmate account and navigating to settings >> https //app kloudmate com/settings >> edit workspace step 2 example application for this setup, we will be using a basic python program that uses openai api adhere to the steps outlined below to set up the environment for the program step 3 installation 1\ create a new directory and activate a virtual environment mkdir openllmetry getting started cd openllmetry getting started python3 m venv venv source /venv/bin/activate 2\ next, install the traceloop sdk and openai pip install traceloop sdk openai step 4 instrumentation 1\ create a file named app py and add the following code to it from traceloop sdk import traceloop from openai import openai from traceloop sdk decorators import workflow, task import time client = openai(api key=\<api key>) traceloop init(app name="your app name", api endpoint=https //otel kloudmate com 4318, headers={"authorization" \<kloudmate api key>}) @task(name="joke creation") def create joke() completion = client chat completions create( model="gpt 3 5 turbo", messages=\[{"role" "user", "content" "tell me a joke about opentelemetry"}], ) return completion choices\[0] message content @workflow(name="pirate joke generator") def joke workflow() eng joke = create joke() print(eng joke + "\n\n") joke workflow() time sleep(10) the kloudmate api key can be obtained by logging into your kloudmate account and navigating to settings >> https //app kloudmate com/settings >> edit workspace step 5 run the instrumented program 1\ use the following command to run the program python3 app py step 6 visualize the traces emitted to kloudmate 1\ login to https //app kloudmate com and navigate to the ' traces ' section you will be able to view the llm attributes as shown below you will also be able to visualize the traces and spans related resources https //docs kloudmate com/what is opentelemetry https //docs kloudmate com/openllmetry opentelemetry based observability for llms