imkubernetes: Kubernetes input module¶

Module Name:	imkubernetes
Author:	rsyslog project
Available since:	8.2604.0

imkubernetes is an experimental Kubernetes-first input module that tails node-local pod log files, parses CRI or Docker json-file records, preserves stream and timestamp metadata, and can enrich each record with pod metadata from the Kubernetes API.

Purpose¶

Warning

imkubernetes is experimental. Its configuration interface, metadata shape, and operational behavior may change based on real-world Kubernetes deployments. Please test it carefully before production use and share feedback, bug reports, and missing use cases through the rsyslog GitHub issue tracker or discussions.

imkubernetes is designed for node-agent deployments, typically as part of a DaemonSet. It reads the host’s Kubernetes container log files directly instead of talking to a Docker daemon, which makes it fit current Kubernetes runtimes better than imdocker.

The module supports two common file layouts:

pod log files such as /var/log/pods/<namespace>_<pod>_<uid>/<container>/<restart>.log
container symlink paths such as /var/log/containers/<pod>_<namespace>_<container>-<containerid>.log

For CRI logs, imkubernetes merges partial records before submission. For Docker json-file logs, it extracts the embedded timestamp, message payload, and stream.

Message metadata¶

imkubernetes stores parsed fields under $!kubernetes and, when present, container IDs under $!docker.

Common $!kubernetes properties include:

namespace_name
pod_name
pod_uid
container_name
restart_count for pod log paths
stream (stdout or stderr)
log_format (cri or docker_json)
log_file

When EnrichKubernetes is enabled, the module also adds selected pod metadata from the Kubernetes API, including pod UID, labels, annotations, owner references, pod IP, and host IP.

Configuration Parameters¶

Note

Parameter names are case-insensitive. CamelCase is recommended for readability.

Module Parameters¶

Parameter	Summary
LogFileGlob	Glob that selects Kubernetes log files to tail; default `/var/log/pods///*.log`.
PollingInterval	Seconds between filesystem scans for matching log files; default `1`.
Ruleset	Ruleset that receives records submitted by `imkubernetes`; default is the default ruleset.
CacheEntryTtl	Seconds to keep pod metadata in the in-memory cache; default `300`.
FreshStartTail	Start newly discovered files at end-of-file instead of replaying existing content; default `off`.
DefaultSeverity	Default severity assigned to submitted records that are not mapped to `stderr`; default `info`.
DefaultFacility	Default syslog facility assigned to submitted records; default `user`.
EscapeLF	Escapes embedded line feeds in the submitted message payload; default `on`.
EnrichKubernetes	Enable Kubernetes API lookups for pod metadata enrichment; default `on`.
KubernetesUrl	Base URL of the Kubernetes API server; default `https://kubernetes.default.svc.cluster.local:443`.
Token	Literal bearer token used for Kubernetes API authentication.
TokenFile	Path to a file containing the Kubernetes API bearer token; default `/var/run/secrets/kubernetes.io/serviceaccount/token`.
tls.caCert	CA certificate bundle used to verify the Kubernetes API TLS certificate; default `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`.
AllowUnsignedCerts	Disable TLS peer verification for Kubernetes API requests; default `off`.
SkipVerifyHost	Disable TLS hostname verification for Kubernetes API requests; default `off`.

Deployment notes¶

imkubernetes is intended to run close to the node logs. Typical deployments:

mount /var/log/pods read-only and keep the default LogFileGlob
mount /var/log/containers read-only and override LogFileGlob if you prefer the symlink view
mount the service-account token and CA certificate when enrichment is enabled

If you only need file-derived metadata, set EnrichKubernetes to off and the module will avoid Kubernetes API calls entirely.

Statistic Counter¶

This plugin maintains per-module statistics. The statistic name is imkubernetes.

The following counters are maintained:

submitted - records submitted to the main queue
parse.errors - lines that could not be parsed as CRI or Docker JSON and were submitted as raw payloads
files.discovered - log files discovered by glob scans
records.cri - CRI records parsed successfully
records.dockerjson - Docker json-file records parsed successfully
kube.cache_hits - pod metadata cache hits
kube.cache_misses - pod metadata cache misses
kube.api_errors - Kubernetes API request failures
ratelimit.discarded - records dropped by rate limiting

Examples¶

Collect pod logs with in-cluster enrichment¶

module(
  load="imkubernetes"
  LogFileGlob="/var/log/pods/*/*/*.log"
  PollingInterval="1"
  KubernetesUrl="https://kubernetes.default.svc.cluster.local:443"
  TokenFile="/var/run/secrets/kubernetes.io/serviceaccount/token"
  tls.caCert="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
)

template(name="k8s-json" type="list" option.jsonf="on") {
  property(outname="msg" name="msg" format="jsonf")
  property(outname="namespace" name="$!kubernetes!namespace_name" format="jsonf")
  property(outname="pod" name="$!kubernetes!pod_name" format="jsonf")
  property(outname="container" name="$!kubernetes!container_name" format="jsonf")
  property(outname="stream" name="$!kubernetes!stream" format="jsonf")
  property(outname="pod_ip" name="$!kubernetes!pod_ip" format="jsonf")
}

action(type="omfile" file="/var/log/rsyslog-k8s.json" template="k8s-json")

The same module-level settings can be provided through YAML configuration:

modules:
  - load: imkubernetes

inputs:
  - type: imkubernetes
    logfileglob: "/var/log/pods/*/*/*.log"
    pollinginterval: 1
    kubernetesurl: "https://kubernetes.default.svc.cluster.local:443"
    tokenfile: "/var/run/secrets/kubernetes.io/serviceaccount/token"
    tls.cacert: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
    ruleset: main

Collect container symlink logs without API enrichment¶

module(
  load="imkubernetes"
  LogFileGlob="/var/log/containers/*.log"
  EnrichKubernetes="off"
)

modules:
  - load: imkubernetes

inputs:
  - type: imkubernetes
    logfileglob: "/var/log/containers/*.log"
    enrichkubernetes: off
    ruleset: main