MongoDB integration with KloudMate using Opentelemetry

mongodb is a widely used nosql database designed for scalability and flexibility it stores data in a json like format (bson), which allows for dynamic schema design and handling of unstructured data known for its high performance in read and write heavy environments, mongodb is ideal for applications like real time analytics, content management, and iot systems available in both open source and enterprise editions, it offers features like horizontal scaling through sharding and built in replication for high availability this document will guide you on collecting mongodb server metrics using the opentelemetry (otel) mongodb metrics receiver this receiver can also help capture metrics on a per database or per collection basis from mongodb instances running on aws ec2, azure vms, or on premise servers pre requisites the mongodb server must be running this receiver supports mongodb versions 4 0+, 5 0, 6 0, 7 0 install the opentelemetry collector on the specific server that requires metric monitoring see installing and configuring opentelemetry collector mongodb recommends setting up a least privilege user (lpu) with a clustermonitor role in order to collect metrics please refer to lpu sh for an example of how to configure these permissions to create this please follow the below steps $mongosh >use admin >db createuser({ user "user", pwd "password", roles \[{ role "clustermonitor", db "admin" }] }) >exit step 1 configure the receivers to scrape metrics as well as logs to start monitoring mongodb with otel collector, you need to configure the mongodb receiver linux users open the file located at /etc/otelcol contrib/config yaml using your preferred text editor windows users create a new file called config yaml in the c \program files\opentelemetry collector folder you can use notepad or any text editor to do this add the suitable extensions extensions health check pprof endpoint 0 0 0 0 1777 zpages endpoint 0 0 0 0 55679 file storage create directory true in this step, two receivers are integrated to retrieve telemetry data from mongodb mongodb receiver to retrieve metrics file log receiver to retrieve logs to get the metrics use only the mongodb receiver receivers mongodb hosts \ endpoint localhost 27017 username otel #change the correct username password passwd #change the correct password collection interval 60s initial delay 1s tls insecure true insecure skip verify true filelog include \[/var/log/mongodb/ log] storage file storage retry on failure enabled true step 2 set up the processor component to identify resource information from the host and either append or replace the resource values in the telemetry data with this information please choose one of the following configuration options based on your provider (aws ec2, azure vm, or on premises server) server(can be on premise, non cloud, or cloud) processors batch send batch size 5000 timeout 10s resourcedetection detectors \[system] system resource attributes host name enabled true host id enabled true os type enabled false resource attributes \ key service name action insert from attribute host name aws ec2 (optional) to retrieve aws ec2 instance tags along with logs and metrics, users need to associate an iam role with the ec2 instance that includes the ec2\ describetags policy processors batch send batch size 5000 timeout 10s resourcedetection/ec2 detectors \["ec2"] ec2 tags \ ^tag1$ \ ^tag2$ \ ^label $ \ ^name$ resource attributes \ key service name action insert from attribute ec2 tag name \ key service instance id action insert from attribute host id azure virtual machines processors batch send batch size 5000 timeout 10s resourcedetection/azure detectors \["azure"] azure tags \ ^tag1$ \ ^tag2$ \ ^label $ \ ^name$ resource attributes \ key service name action insert from attribute azure vm name \ key service instance id action insert from attribute host id step 3 configure the exporter, and extension and save the configuration set up the kloudmate backend on the exporter part of the open telemetry configuration file and configure the pipeline exporters debug verbosity detailed otlphttp endpoint 'https //otel kloudmate com 4318' headers authorization xxxxxxxx # use the auth key service pipelines metrics receivers \[mongodb] processors \[batch, resourcedetection, resource] # apply the correct resourcedetection exporters \[otlphttp] logs receivers \[filelog] processors \[batch, resourcedetection, resource] # apply the correct resourcedetection exporters \[otlphttp] extensions \[health check, pprof, zpages, file storage] step 4 to restart and verify the status of the opentelemetry (otel) collector, follow these steps for linux execute the following commands sudo systemctl restart otelcol contrib sudo systemctl status otelcol contrib these commands will restart the otel collector and display its current status for windows open the services window press win + r , type services msc , and press ok alternatively, search for " services " in the windows start menu in the services window, locate the " opentelemetry collector " service right click the service and select " restart " subsequently, monitor the metrics on the kloudmate dashboard and set up an alarm to receive notifications if the potential metrics for a specific application rise default metrics for mongodb true false 280,0,70false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type false unhandled content type for full details on metrics, click here