Community:Comma-Separated Multi-Value Field Extraction In Single-line Event

Field Extraction: Comma-separated multi-value

This is an example of how to use fields.conf TOKENIZER, and makemv.

The following log contains one to three host names in each events;e.g. esx01(14). We would like to have a field called ChangeMv which contains all the host names as multi-value, instead of a single value. You can use "makemv" in the search string, or make use of props.conf/transforms.conf/fields.conf as auto field extraction.

Here is the sample log.

#
# Sample Log
#
- test_multi.log
# cat test_multi.log 
2011-09-06 10:22:07 172.16.10.1 VMhosts[0000]: Changed: esxi01(14), esxi07(4), esxi08(1);
2011-09-06 11:22:07 172.16.10.1 VMhosts[0000]: Changed: esxi01(15), esxi07(4), esxi08(5);
2011-09-06 12:32:17 172.16.10.1 VMhosts[0000]: Changed: esxi01(15);
2011-09-06 13:32:17 172.16.10.1 VMhosts[0000]: Changed: esxi07(8), esxi08(20);
2011-09-06 14:42:17 172.16.10.1 VMhosts[0000]: Changed: esxi08(20);
2011-09-06 15:42:22 172.16.10.1 VMhosts[0000]: Changed: esxi01(10), esxi08(20);
2011-09-06 17:52:47 172.16.10.1 VMhosts[0000]: Changed: esxi01(11), esxi08(19);
2011-09-06 18:52:57 172.16.10.1 VMhosts[0000]: Changed: esxi01(11), esxi07(8), esxi08(4);
2011-09-06 19:02:37 172.16.10.1 VMhosts[0000]: Changed: esxi01(11), esxi07(8);
2011-09-06 20:02:37 172.16.10.1 VMhosts[0000]: Changed: esxi01(11);

#
# Solution 1: By search
#

source="*test_multi.log" earliest=-1d@d 
   |  rex "Changed: (?<ChangedMv>[^\;]+)" 
   | makemv delim="," ChangedMv 
   |  eval ChangedMv=trim(ChangedMv, " ") 
   |  table ChangedMv

#
# Solution 2:  By configuration files; auto field extraction
#

-fields.conf
[ChangedMv]
TOKENIZER = (\w+\(\d+\))

-props.conf
[host::mutlivalue]
REPORT-mvalue = multivalue00

-transforms.conf
[multivalue00]
REGEX = Changed:\s(.*?);
FORMAT = ChangedMv::$1

Do you want to test it?

1. Edit the configuration files

2. Create the log file and index it by oneshot command;
$SPLUNK_HOME/bin/splunk add oneshot <file path> -host multivalue -auth admin:changeme

3. Login to Splunkweb and search "host=multivalue"
=> Check the extracted field value for ChangedMv field