Mask and Drop Sensitive Data

Preventing sensitive data—such as Personally Identifiable Information (PII), passwords, or API keys—from leaving your infrastructure is a critical security practice. Additionally, dropping noisy, low-value telemetry (like repeated health checks) helps keep your dashboards clean and reduces data transfer costs.

The OpenTelemetry Collector provides several processors, including the attributes, filter, and transform processors, to redact, mask, or drop telemetry before it is exported to KloudMate.

Deleting Sensitive Attributes

If specific attributes consistently contain sensitive data you do not need, remove the attribute entirely with the attributes processor.

Example: Drop PII Attributes

This configuration removes the user.email and credit_card_number attributes from any incoming spans or logs.

processors:
  attributes/drop_pii:
    actions:
      - key: user.email
        action: delete
      - key: credit_card_number
        action: delete

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [attributes/drop_pii, batch]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [attributes/drop_pii, batch]
      exporters: [otlphttp]

Masking Values with the Transform Processor

If you need to keep an attribute but redact portions of its value (e.g., masking a password within a URL or a token within a log body), use the transform processor. It uses the OpenTelemetry Transformation Language (OTTL).

Example: Mask Passwords in HTTP URLs

This configuration uses a regex pattern to find password=... in the http.url attribute and replaces the value with password=***.

processors:
  transform/mask_passwords:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          - replace_pattern(attributes["http.url"], "password=[^&]*", "password=***")

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [transform/mask_passwords, batch]
      exporters: [otlphttp]

Dropping Unnecessary Logs or Spans

To discard entire spans, logs, or metrics based on their attributes, use the filter processor. It is useful for removing repetitive noise, such as load balancer health checks.

Example: Drop Health Check Telemetry

This configuration evaluates incoming spans and logs. If a span targets the /health endpoint, or a log body contains the exact text "Healthcheck passed", the entire span or log record is dropped and not sent to KloudMate.

processors:
  filter/drop_healthchecks:
    error_mode: ignore
    traces:
      span:
        - 'attributes["http.target"] == "/health"'
    logs:
      log_record:
        - 'body == "Healthcheck passed"'

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [filter/drop_healthchecks, batch]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [filter/drop_healthchecks, batch]
      exporters: [otlphttp]

Pipeline Execution Order

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: 
        - filter/drop_healthchecks # 1. Drop noisy spans completely
        - attributes/drop_pii      # 2. Delete sensitive attributes
        - transform/mask_passwords # 3. Regex replacement on remaining spans
        - batch                    # 4. Batch for export
      exporters: [otlphttp]