From Splunk Wiki
Some notes on changes that have occurred on the light forwarder across versions. This is meant as info for troubleshooting, investigation, rather than a nice friendly document that helps to accomplish anything in particular.
The SplunkLightForwarder app now no longer takes special actions when it is enabled/disabled. This means you should be able to enable/disable it by editing
MANIFEST, removing the directory, etc without trouble.
There is no more
setup.conf file (the delivered file is empty), there are no .xml file twiddles, and all of the necessary tweaks are declared in
When splunkd is reading through the
composite-splunkd.xml, instantiating pipelines and processors, it makes use of the values in
default-mode.conf to influence whether it should or should not actually create them. Investigating the behavior will likely require use of
splunk start --debug.
The intended run-time behavior is unchanged from 3.4.9.
3.4.8 - 3.4.9
The "SplunkLightForwarder" app is changed from 3.4.6 to use the parsing pipeline. All inputs pass their data to the parsing pipeline as in traditional Splunk, but the parsing pipeline is truncated, forwarding the data with minimal processing (only utf8 processor).
The configuration is essentially the same method as the 3.4 - 3.4.6 configuration, using setup.conf, default-mode.conf, and the '
splunk enable app SplunkLightForwarder' command to trigger xml file twiddling.
The number of files twiddled is reduced to one, in setup.conf:
[files] parsing = config/parsing/config.xml
The number of processors disabled is increased, in default-mode.conf:
#Turn off a processor [pipeline:indexerPipe] disabled_processors=diskusage, signing, tcp-output-generic-processor
3.4 - 3.4.6
The "SplunkLightForwarder" application moved the policy of what should be enabled/disabled out of the python scripts (3.3.x) and into conf files in an application directory.
In 3.4 -> 3.4.6, there were no processors loaded for the parsing or indexing queue, and these queues were not intended to be used. The input-method specific queues simply contained a sendout processor. (This seemed to cause the lightforwarder to instantiate the sendout processor multiple times, and make multiple connections to the receiver, at least in some cases.)
The configuration of this is two part:
- The app specifies in
setup.confthat it wants certain modules disabled in
[modules], and also that it wants certain
config.xmlfiles replaced in
[files], and disables certain processors using
- A command is run to apply these settings , eg
splunk enable app SplunkLightForwarder
The modules are not enabled/disabled unless the command is explicitly run. The config.xml files for modules are renamed to config.xml.disabled, and the files in the 'files' section are copied from etc/apps/SplunkLightForwarder/config.... to etc/modules/... replacing the default files.
setup.conf from 3.4.6 prior to 3.4.8:
[apps] samples = disabled [modules] distributedDeployment = disabled distributedSearch = disabled input/FIFO = disabled input/UDP = disabled input/tcp = disabled input/syslogFIFO = disabled input/syslogUDP = disabled [files] input/exec = config/input/exec/config.xml input/tailfile = config/input/tailfile/config.xml input/wineventlog = config/input/wineventlog/config.xml input/fschangemonitor = config/input/fschangemonitor/config.xml
default-mode.conf from 3.4.6:
#Turn off a processor [pipeline:indexerPipe] disabled_processors=diskusage, signing
The "Lightweight Forwarder" simply disabled a variety of unnecessary input modules so they were not loaded.
The means to set this up was "
splunk set server-type forwarder".
The implementation was to rename various config.xml files in etc/modules/input/<type>/ to config.xml.disabled, this was what the set server-type command did.
There was an additional step that was documented to avoid parsing by routing inputs directly to the index queue, which we called Transplant Parsing.
intputs.conf: [inputstanza] queue=indexQueue