MongoDB Telemetry Data to KloudMate Using Fluentbit

7min
This document provides a guide on how to send MongoDB telemetry data to KloudMate using Fluentbit.
In the described setup, the latest MongoDB instance is running on an EC2 server. An exporter is required to export metrics to a monitoring backend. In this process, the metrics will be exported by the MongoDB exporter and Fluentbit will be responsible for sending the metrics data to KloudMate backend. 
Step 1: Prerequisites:
﻿Fluentbit binaries and packages as per your system requirement.
MongoDb installed and running on http://127.0.0.1:27017﻿
Step 2: Install and configure MongoDB exporter:
Add user and group for exporter
Text
$sudo groupadd --system <mongodbuser>
$sudo useradd -s /sbin/nologin --system -g <mongodbuser> <mongodbuser>
$sudo groupadd --system <mongodbuser>
$sudo useradd -s /sbin/nologin --system -g <mongodbuser> <mongodbuser>
﻿
Download and extract MongoDB Exporter
Text
$wget https://github.com/percona/mongodb_exporter/releases/download/v0.39.0/mongodb_exporter-0.39.0.linux-amd64.tar.gz   
 /$tar xvzf mongodb_exporter-<latest>
 $sudo mv mongodb_exporter /usr/local/bin/

$wget https://github.com/percona/mongodb_exporter/releases/download/v0.39.0/mongodb_exporter-0.39.0.linux-amd64.tar.gz   
 /$tar xvzf mongodb_exporter-<latest>
 $sudo mv mongodb_exporter /usr/local/bin/
﻿
Enable MongoDB authentication
Text
$mongosh
>use admin
>db.createUser({ user: "USER", pwd: "PASSWORD", roles: [{ role: "clusterMonitor", db: "admin" }] })
>exit
$mongosh
>use admin
>db.createUser({ user: "USER", pwd: "PASSWORD", roles: [{ role: "clusterMonitor", db: "admin" }] })
>exit
﻿
Create a system file for the MongoDB exporter and run it as a service.
Text
$sudo nano /etc/systemd/system/mongodb_exporter.service
$sudo nano /etc/systemd/system/mongodb_exporter.service
﻿
[Unit]
Description=MongoDB Exporter

[Service]
User=<mongodbuser>
Type=simple
Restart=always
ExecStart=/usr/local/bin/mongodb_exporter --mongodb.uri=mongodb://USER:[email protected]:27017 --collect-all --compatible-mode

[Install]
WantedBy=multi-user.target
Text
$sudo systemctl daemon-reload
$sudo systemctl start mongodb_exporter.service
$sudo systemctl status mongodb_exporter.service 

$sudo systemctl daemon-reload
$sudo systemctl start mongodb_exporter.service
$sudo systemctl status mongodb_exporter.service 
﻿
Step 3: Test the available metrics.
Text
$curl http://localhost:9216/metrics
$curl http://localhost:9216/metrics
﻿
Step 4: Configure fluentbit to send telemetry data to KloudMate.
Text
[SERVICE]
    flush        1
    daemon       Off
    log_level    debug
    parsers_file parsers.conf
    plugins_file plugins.conf
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020
    storage.metrics on


[INPUT]
    Name                tail
    Path                /var/log/mongodb/*.log

[INPUT]
    name prometheus_scrape
    host 0.0.0.0
    port 9216

[OUTPUT]
    name  stdout
    match *

[OUTPUT]
    Name                 opentelemetry
    Match                *
    Host                 otel.kloudmate.com
    Port                 4318
    Metrics_uri          /v1/metrics
    Logs_uri             /v1/logs
    Log_response_payload True
    Header               Authorization <Auth token>
    Tls                  On
    Tls.verify           Off
    # add user-defined labels
    add_label            db logs
    add_label            service.name <service name>
[SERVICE]
    flush        1
    daemon       Off
    log_level    debug
    parsers_file parsers.conf
    plugins_file plugins.conf
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020
    storage.metrics on


[INPUT]
    Name                tail
    Path                /var/log/mongodb/*.log

[INPUT]
    name prometheus_scrape
    host 0.0.0.0
    port 9216

[OUTPUT]
    name  stdout
    match *

[OUTPUT]
    Name                 opentelemetry
    Match                *
    Host                 otel.kloudmate.com
    Port                 4318
    Metrics_uri          /v1/metrics
    Logs_uri             /v1/logs
    Log_response_payload True
    Header               Authorization <Auth token>
    Tls                  On
    Tls.verify           Off
    # add user-defined labels
    add_label            db logs
    add_label            service.name <service name>
﻿
After restarting the fluentbit service, the logs and metrics of MongoDB will be successfully delivered to KloudMate.
MongoDB integration with KloudMate using Opentelemetry
Docker-compose for MongoDB and KloudMate Backend Integration