Community:Comma-Separated Multi-Value Field Extraction In Single-line Event

From Splunk Wiki

Jump to: navigation, search

Field Extraction: Comma-separated multi-value

This is an example of how to use fields.conf TOKENIZER, and makemv.

The following log contains one to three host names in each events;e.g. esx01(14). We would like to have a field called ChangeMv which contains all the host names as multi-value, instead of a single value. You can use "makemv" in the search string, or make use of props.conf/transforms.conf/fields.conf as auto field extraction.

Here is the sample log.

# Sample Log
- test_multi.log
# cat test_multi.log 
2011-09-06 10:22:07 VMhosts[0000]: Changed: esxi01(14), esxi07(4), esxi08(1);
2011-09-06 11:22:07 VMhosts[0000]: Changed: esxi01(15), esxi07(4), esxi08(5);
2011-09-06 12:32:17 VMhosts[0000]: Changed: esxi01(15);
2011-09-06 13:32:17 VMhosts[0000]: Changed: esxi07(8), esxi08(20);
2011-09-06 14:42:17 VMhosts[0000]: Changed: esxi08(20);
2011-09-06 15:42:22 VMhosts[0000]: Changed: esxi01(10), esxi08(20);
2011-09-06 17:52:47 VMhosts[0000]: Changed: esxi01(11), esxi08(19);
2011-09-06 18:52:57 VMhosts[0000]: Changed: esxi01(11), esxi07(8), esxi08(4);
2011-09-06 19:02:37 VMhosts[0000]: Changed: esxi01(11), esxi07(8);
2011-09-06 20:02:37 VMhosts[0000]: Changed: esxi01(11);

# Solution 1: By search

source="*test_multi.log" earliest=-1d@d 
   |  rex "Changed: (?<ChangedMv>[^\;]+)" 
   | makemv delim="," ChangedMv 
   |  eval ChangedMv=trim(ChangedMv, " ") 
   |  table ChangedMv

# Solution 2:  By configuration files; auto field extraction

TOKENIZER = (\w+\(\d+\))

REPORT-mvalue = multivalue00

REGEX = Changed:\s(.*?);
FORMAT = ChangedMv::$1

Do you want to test it?

1. Edit the configuration files

2. Create the log file and index it by oneshot command;
$SPLUNK_HOME/bin/splunk add oneshot <file path> -host multivalue -auth admin:changeme

3. Login to Splunkweb and search "host=multivalue"
=> Check the extracted field value for ChangedMv field

Personal tools
Hot Wiki Topics

About Splunk >
  • Search and navigate IT data from applications, servers and network devices in real-time.
  • Download Splunk