imbeats: Beats v2 input module¶

imbeats receives Elastic Beats and Elastic Agent output.logstash events via Lumberjack protocol v2 over TCP or TLS, keeps the original JSON payload in msg, maps decoded event fields into the top-level structured tree $!, and stores transport/protocol metadata under $!metadata!imbeats.

Module Name:	imbeats
Author:	Adiscon and contributors
Available since:	8.2604.0

Purpose¶

imbeats accepts Elastic Beats and Elastic Agent traffic that uses the Logstash-style Lumberjack v2 protocol. Configure Beats or Elastic Agent with output.logstash and point it at the rsyslog listener. The module reuses rsyslog’s netstrm transport subsystem, so it can listen via plain TCP or the configured TLS stream driver.

The first implementation supports:

Lumberjack v2 only
W window frames
J JSON event frames
C compressed frames
cumulative A acknowledgements

The first implementation intentionally optimizes the internal event shape for common Elasticsearch-oriented pipelines:

msg keeps the original JSON payload
decoded Beat event fields are added under top-level $!
transport and protocol metadata is stored under $!metadata!imbeats
listener-side size limits reject oversized windows, frames, and compressed payload expansion before unbounded allocation

This default may be revisited later. A user-selectable representation mode is not part of the initial release.

End-to-end Elastic Agent setup¶

Install rsyslog, imbeats, and a TLS stream-driver package through your operating system packages. On Debian or Ubuntu systems using packages that ship the GnuTLS stream driver separately, a typical TLS prerequisite is:

sudo apt install rsyslog rsyslog-gnutls

If your distribution packages imbeats.so separately, install that package as well. The exact package name depends on the distribution.

The example below listens on the common Beats/Logstash port 5044 and uses GnuTLS. Replace the certificate paths with files issued for your rsyslog receiver host.

module(load="imbeats")

input(type="imbeats"
      port="5044"
      ruleset="beats_to_file"
      streamdriver.name="gtls"
      streamdriver.mode="1"
      streamdriver.authmode="anon"
      streamdriver.cafile="/etc/rsyslog.d/tls/ca.pem"
      streamdriver.certfile="/etc/rsyslog.d/tls/server-cert.pem"
      streamdriver.keyfile="/etc/rsyslog.d/tls/server-key.pem")

ruleset(name="beats_to_file") {
  action(type="omfile" file="/var/log/imbeats.log")
}

Configure Elastic Agent or Filebeat to use the Logstash output and point it at the rsyslog host:

outputs:
  default:
    type: logstash
    hosts: ["rsyslog.example.net:5044"]
    compression_level: 9
    ssl.enabled: true
    ssl.certificate_authorities:
      - /etc/elastic-agent/certs/ca.pem

For Filebeat standalone configuration, the same output settings are placed under output.logstash:

output.logstash:
  hosts: ["rsyslog.example.net:5044"]
  compression_level: 9
  ssl.enabled: true
  ssl.certificate_authorities:
    - /etc/filebeat/certs/ca.pem

Use certificate verification in production. Test-only settings such as ssl.verification_mode: none are useful for isolated lab checks but should not be used for production ingestion.

The example above lets Elastic Agent or Filebeat validate the rsyslog server certificate without requiring the sender to present a client certificate. For mutual TLS, also configure the sender with its own client certificate and key, then switch the rsyslog listener to the appropriate certificate-validation auth mode and permitted peer settings.

Troubleshooting Elastic Agent delivery¶

If rsyslog reports that gtls or ossl cannot be loaded, install the matching TLS stream-driver package, such as rsyslog-gnutls or rsyslog-openssl.
If the Beat or Agent logs certificate validation errors, confirm that the sender trusts the CA that issued the rsyslog listener certificate and that the configured host name matches the certificate.
If the sender connects but does not deliver events, verify that it is using output.logstash and not an Elasticsearch or HTTP output.
imbeats accepts compressed Lumberjack v2 frames. Keep compression enabled on the sender unless you are isolating a transport problem.
Use the events.received, events.submitted, compressed_frames, and protocol_errors counters to distinguish traffic, parsing, and protocol failures.

Configuration Parameters¶

Module Parameters¶

Currently none.

Input Parameters¶

Parameter	Summary
address	Bind the imbeats listener to a specific local address instead of all interfaces.
gnutlsPriorityString	Override the GnuTLS priority string used by the selected TLS stream driver.
KeepAlive	Enable socket-level TCP keepalive on accepted imbeats connections.
KeepAlive.Interval	Set the interval between TCP keepalive probes for imbeats sessions.
KeepAlive.Probes	Set how many TCP keepalive probes are sent before the peer is considered dead.
KeepAlive.Time	Set the idle time before TCP keepalive probing begins on imbeats sessions.
listenPortFileName	Write the actual bound port to a file after an imbeats listener starts.
maxDecompressedSize	Limit how large a compressed Lumberjack frame may become after decompression.
maxFrameSize	Limit the JSON or compressed frame payload size imbeats accepts from one Lumberjack frame.
maxWindowSize	Limit how many events imbeats accepts in one Lumberjack batch window.
name	Set the `inputname` property used for messages received by this imbeats input.
NetworkNamespace	Open imbeats listener sockets inside the specified Linux network namespace.
PermittedPeer	Restrict accepted TLS peers to the configured certificate names.
port	Set the TCP port on which the imbeats listener accepts Lumberjack v2 clients.
ruleset	Bind the imbeats input to a specific ruleset instead of the default ruleset.
StreamDriver.AuthMode	Set the TLS authentication mode used by the configured imbeats stream driver.
StreamDriver.CAFile	Specify the CA bundle used to validate TLS peers for imbeats.
StreamDriver.CertFile	Specify the local certificate presented by TLS-enabled imbeats listeners.
StreamDriver.CheckExtendedKeyPurpose	Enable extended key usage checks when validating imbeats TLS certificates.
StreamDriver.CRLFile	Specify the certificate revocation list file used by TLS-enabled imbeats listeners.
StreamDriver.KeyFile	Specify the private key file used by TLS-enabled imbeats listeners.
StreamDriver.Mode	Select whether imbeats uses plain TCP or a TLS-enabled stream driver mode.
StreamDriver.Name	Select the netstrm backend used by imbeats, for example `ptcp`, `gtls`, or `ossl`.
StreamDriver.PermitExpiredCerts	Control how the imbeats TLS stream driver handles expired peer certificates.
StreamDriver.PrioritizeSAN	Prefer subject alternative names over common names when validating imbeats TLS peer names.
StreamDriver.TlsRevocationCheck	Enable TLS revocation checking for certificates presented to the imbeats listener.
StreamDriver.TlsVerifyDepth	Set the maximum certificate chain depth accepted during imbeats TLS validation.

Examples¶

RainerScript¶

module(load="imbeats")

input(type="imbeats"
      port="5044"
      ruleset="beats_to_es"
      streamdriver.name="gtls"
      streamdriver.mode="1"
      streamdriver.authmode="anon"
      streamdriver.cafile="/etc/rsyslog.d/ca.pem"
      streamdriver.certfile="/etc/rsyslog.d/server-cert.pem"
      streamdriver.keyfile="/etc/rsyslog.d/server-key.pem")

ruleset(name="beats_to_es") {
  action(type="omfile" file="/var/log/imbeats-debug.log")
}

YAML¶

version: 2
modules:
  - load: imbeats

inputs:
  - type: imbeats
    port: "5044"
    ruleset: beats_to_es
    streamdriver.name: gtls
    streamdriver.mode: 1
    streamdriver.authmode: anon
    streamdriver.cafile: /etc/rsyslog.d/ca.pem
    streamdriver.certfile: /etc/rsyslog.d/server-cert.pem
    streamdriver.keyfile: /etc/rsyslog.d/server-key.pem

rulesets:
  - name: beats_to_es
    script: |
      action(type="omfile" file="/var/log/imbeats-debug.log")

Statistic Counters¶

The module exposes these impstats counters:

connections.accepted
connections.closed
protocol_errors
batches.received
batches.acked
events.received
events.submitted
events.failed
compressed_frames
json_decode_failures
ack_failures

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project