rsyslog/rsyslog-etl

The rsyslog/rsyslog-etl image is a specialized ingestion role that accepts syslog over UDP, TCP, and RELP, then forwards events to a Vespa HTTP endpoint using omhttp.

Overview

The rsyslog-etl image extends the standard rsyslog/rsyslog base with a small ETL-oriented configuration. It receives network syslog and sends each message to a Vespa document API endpoint. The packaged rules use a simple JSON document template containing the original message text and generate the Vespa document path dynamically from environment variables.

This image is a concrete example of rsyslog being used as an ETL transport and delivery component. It is optimized for Vespa-oriented pipelines, not presented here as a generic all-destinations ETL appliance.

Warning

rsyslog/rsyslog-etl is currently experimental. Treat it as a specialized example image for Vespa-oriented ETL pipelines rather than a broadly supported general-purpose container role.

Note

  • UDP (514/udp) and TCP (514/tcp) are enabled by default.

  • RELP (2514/tcp) is enabled by default.

  • The packaged ETL output is HTTP to Vespa, not a local file sink.

  • The packaged configuration is explicitly unencrypted on the input side.

Environment Variables

ENABLE_UDP

Enable UDP syslog reception. Default on.

ENABLE_TCP

Enable TCP syslog reception. Default on.

ENABLE_RELP

Enable RELP syslog reception on 2514/tcp. Default on.

ENABLE_VESPA

Enable the packaged Vespa HTTP action. Default on.

VESPA_NAMESPACE

Vespa document namespace used in the generated REST path.

VESPA_DOCTYPE

Vespa document type used in the generated REST path.

VESPA_SERVER

Hostname or IP address of the Vespa HTTP endpoint.

VESPA_PORT

TCP port of the Vespa HTTP endpoint.

RSYSLOG_HOSTNAME

Hostname used inside rsyslog. Defaults to the value of /etc/hostname when unset.

PERMIT_UNCLEAN_START

Skip configuration validation when set. By default rsyslogd -N1 validates the configuration.

RSYSLOG_ROLE

Role name consumed by the entrypoint. The current image sets this to collector.

Port Mapping Reference

Protocol

Container Port

Example External

Controlled by

UDP Syslog | 514/udp

514/udp

ENABLE_UDP

TCP Syslog | 514/tcp

514/tcp

ENABLE_TCP

RELP | 2514/tcp

2514/tcp

ENABLE_RELP

Consumption Model

The repository includes a sample Docker Compose deployment under packaging/docker/rsyslog/compose-etl/compose.yml. That example persists the rsyslog spool directory, exposes UDP/TCP/RELP listener ports, and passes the Vespa connection variables as environment settings.