From Splunk Wiki

Jump to: navigation, search

UDP inputs with Splunk

By design, UDP is a lossy protocol. In cases where all data MUST be logged in complete form, UDP should not be used. Instead, a tcp or file/dir monitor should be implemented as the data input.
UDP data can be lost at any point in the communication process. This includes the originating server, network switch, network router, receiving server, the OS, and splunk. There is the possibility for the OS to drop UDP data at it's discretion. There is also the possibility for Splunk to lose UDP data due to the buffer being filled.

Tuning Tips

  • Set the OS UDP buffer size to a larger amount (default *nix values are in the 64k+ range)
    • Each OS has a specific kernel parameter. See table BELOW
  • Set the UDP buffer size in Splunk (inputs.conf)
    • set the following under your "[udp://]" stanza.
  • _rcvbuf = 5000000
    • This will set it to 5 megabytes.
    • The default setting of 1 megabyte, can typically handle ~2000 UDP messages per second.
    • we have seen this value set to 10 MB


   * Run "netstat -s" before and after your test, or over a short period of time. Examine the UDP section for packet loss.


Settings to look for at the OS level:

Operating System command to change the value
Linux sysctl -w net.core.rmem_max=8388608
Solaris ndd -set /dev/udp udp_max_buf 8388608
FreeBSD, Darwin sysctl -w kern.ipc.maxsockbuf=8388608
AIX no -o sb_max=8388608 (note: AIX only permits sizes of 1048576, 4194304 or 8388608)

Working example from Linux:

inputs.conf - setting the value of _rcvbuf to 20000000

kernel tuning (via sysctl) -

net.core.rmem_default = 8388608

net.core.rmem_max = 16777216

Personal tools
Hot Wiki Topics

About Splunk >
  • Search and navigate IT data from applications, servers and network devices in real-time.
  • Download Splunk