Promtail

Promtail is a log collection agent built for Loki-style pipelines. It discovers log files, attaches labels, applies pipeline stages, and pushes entries to a Loki-compatible endpoint.

To use Promtail with Parseable, route Promtail into an OpenTelemetry Collector that can receive Loki logs and export them to Parseable as OTLP logs. This keeps Promtail in the part of the pipeline it was built for, while the Collector handles the conversion and delivery to Parseable.

Prerequisites

You need Promtail, the OpenTelemetry Collector Contrib distribution with the Loki receiver, a Parseable ingest endpoint, an API key, and a dataset name. The examples below use promtail-logs.

Configure Promtail

Point Promtail at the Collector's Loki receiver instead of sending directly to Parseable.

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /run/promtail/positions.yaml

clients:
  - url: http://otel-collector:3500/loki/api/v1/push
    batchwait: 1s
    batchsize: 1048576

scrape_configs:
  - job_name: kubernetes-pods
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_namespace]
        target_label: namespace
      - source_labels: [__meta_kubernetes_pod_name]
        target_label: pod
      - source_labels: [__meta_kubernetes_pod_container_name]
        target_label: container
    pipeline_stages:
      - cri: {}

For host logs, replace the Kubernetes discovery block with a static file target:

scrape_configs:
  - job_name: varlogs
    static_configs:
      - targets:
          - localhost
        labels:
          job: varlogs
          __path__: /var/log/*.log

Keep your existing pipeline_stages if you already parse JSON, extract labels, handle multiline logs, or rewrite the output message.

Configure the OpenTelemetry Collector

Configure the Collector to receive Loki logs from Promtail and export them to Parseable.

receivers:
  loki:
    protocols:
      http:
        endpoint: 0.0.0.0:3500

processors:
  batch:

exporters:
  otlphttp/parseable_logs:
    endpoint: "http://<parseable-ingestor-endpoint>:8000"
    encoding: proto
    headers:
      Authorization: "Bearer <parseable-api-key>"
      X-P-Stream: promtail-logs

service:
  pipelines:
    logs:
      receivers: [loki]
      processors: [batch]
      exporters: [otlphttp/parseable_logs]

The OTLP HTTP exporter appends /v1/logs for the logs pipeline. Change X-P-Stream if you want the logs to land in a different Parseable dataset.

Verify Logs

Start the Collector first, then start Promtail. Check Promtail logs for successful pushes to the Collector and check Collector logs for successful exports to Parseable.

In Parseable, open the promtail-logs dataset and look for fields from the original Promtail labels, such as namespace, pod, container, and job.

If logs do not appear, confirm that Promtail can reach the Collector on port 3500, the Collector is using the Contrib build with the Loki receiver, and the API key can write to the dataset named by X-P-Stream.