Fluentd

Fluentd is a good fit when you already have a log pipeline with parsers, filters, labels, and buffered delivery. Parseable can be added as an HTTP destination, so Fluentd can keep collecting and enriching events while Parseable stores them for search and analysis.

Prerequisites

Install Fluentd with the HTTP output plugin available. You also need a Parseable ingest endpoint, an API key with write access, and a dataset name. The examples below use fluentd-logs.

Configure Fluentd

This example tails application logs, parses each line as JSON, and forwards the resulting records to Parseable.

<source>
  @type tail
  path /var/log/myapp/*.log
  pos_file /var/log/fluentd/myapp.pos
  tag myapp.logs
  <parse>
    @type json
  </parse>
</source>

<match myapp.logs>
  @type http
  endpoint http://<parseable-ingestor-endpoint>:8000/api/v1/ingest
  http_method post
  content_type application/json
  json_array true
  headers {"X-API-Key":"<parseable-api-key>","X-P-Stream":"fluentd-logs"}

  <format>
    @type json
  </format>

  <buffer>
    @type file
    path /var/log/fluentd/buffer/parseable
    flush_interval 5s
  </buffer>
</match>

Keep your existing <filter> blocks if you already use Fluentd to add Kubernetes labels, service names, or environment metadata. Those fields will arrive in Parseable with the event.

Verify Logs

Restart Fluentd and check its own logs for output errors. Then open the fluentd-logs dataset in Parseable and confirm that new events appear after Fluentd flushes its buffer.

If the dataset is empty, verify that the tail path matches real files, the parser matches the log format, and the output block includes both X-API-Key and X-P-Stream.