Amazon ECS Container Logs to KloudMate
This document provides a guide on how to send Amazon ECS container (Fargate launch type) logs to KloudMate using OpenTelemetry.
In the described setup, Fluentbit is used as the log exporting agent to send logs from the application container. FireLens is the log driver used for exporting logs. The OpenTelemetry plugin allows you to take logs from Fluentbit and submit them to a KloudMate HTTP endpoint.
Step 1: Prerequisites:
- An ECS cluster running containers with the desired logs to be sent.
Recommendation: Use the latest fluentbit image to send container logs. Create a custom AWS fluentbit image with a proper configuration as well as required parsers.
Step 2: Create a Dockerfile for fluentbit.
Recommendation: Instead of using AWS firelens image use the official fluentbit image.
Step 3: Structure the configuration file for successful log delivery:
Step 4: Add parsers file as per use case.
Here is an example of nginx parsers:
Step 5: Push this image to the Amazon Elastic Container Registry (ECR).
Step 6: Task Definition to update the service:
This task definition contains two containers, one is nginx and the other is a fluentbit. Make sure to configure the config file of fluentbit before proceeding with ECS tasks.