Prometheus Monitoring

this document is primarily designed for users who have previously integrated prometheus with their services and are using it for metrics monitoring prometheus is an open source tool for monitoring metrics it collects and stores data as a time series, capturing metrics that evolve over time prerequisite the opentelemetry collector must be installed and running it is recommended that the collector be run on the same server as prometheus the user must sign up for kloudmate and obtain an api authentication key the user must have access to the prometheus yaml configuration file step 1 access the server and open the prometheus configuration file located at /etc/prometheus/prometheus yaml /etc/prometheus/prometheus yaml sudo nano /etc/prometheus/prometheus yml copy the contents of this file and paste them into a text editor, such as notepad step 2 configure the prometheus receiver in the opentelemetry configuration file to send metrics navigate to the /etc/otel /etc/otel directory and open config yaml config yaml paste the copied prometheus configuration into the config yaml config yaml file receivers prometheus config for example, receivers prometheus config scrape configs \ job name 'otel collector' scrape interval 5s static configs \ targets \['0 0 0 0 8888'] \ job name k8s kubernetes sd configs \ role pod relabel configs \ source labels \[ meta kubernetes pod annotation prometheus io scrape] regex "true" action keep metric relabel configs \ source labels \[ name ] regex "(request duration seconds |response duration seconds )" action keep please verify the indentation after adding prometheus configuration step 3 add the required processor and exporter on the edited open telemetry configuration file processors batch send batch size 5000 timeout 10s exporters otlphttp endpoint 'https //otel kloudmate com 4318' headers authorization xxxxxxxx # use the auth key service pipelines metrics receivers \[prometheus] processors \[batch] exporters \[otlphttp] step 4 execute the provided command to restart and verify the status of the opentelemetry (otel) collector subsequently, monitor metrics on the kloudmate dashboard additionally, configure an alarm in kloudmate to receive notifications when cpu utilization for a specific application rises sudo systemctl restart otelcol contrib sudo systemctl status otelcol contrib