Community:WorkingWithSplunkApplication

From Splunk Wiki

Jump to: navigation, search

Working with Splunk applications

If you look through the list of applications available on SplunkBase, you will find a number of them listed as using field names for particular programs and devices in compliance with the Splunk common information model. These applications take the IT data generated by particular software and devices and convert their output to match Splunk's standards. This document covers the technical details of installing such applications. If you desire a deeper look at how to approach these issues, see this how-to.

The first step in installing this application is to unpack it into your $SPLUNK_HOME/etc/apps/ directory. Then, there are three issues you need to address make sure it's properly configured:

  • Make sure that your Splunk server is loading the appropriate data.
  • Make sure that the source type listed in the application, and the source type your Splunk server is using for the related data, are identical.
  • Make sure that the included event types will properly match your data.

Have you already indexed this data?

To determine if your Splunk server has already indexed the data in question, you'll need to search for the data. One method is to do a source search.

If the data has already been indexed, see the section Applying this application to already indexed data. If it hasn't been, see the section Applying this application to data you haven't indexed yet.

Note: When you set up an app, you may change how data is being indexed. For example, enabling the Unix app adds the input of /var/log. If you are already indexing /var/log, any new data will now go into the OS index.

Applying this application to already indexed data

If the data is already indexed, you need to check the assigned source type in your Splunk server against the sourcetype statement in the application's inputs.conf file.

When the source types match

If these two source types already match, you actually don't need the application's inputs.conf file at all, as the data is already indexed and properly source typed. There are two options for disabling the included inputs.conf:

  • Delete the file (not recommended)
  • Add a # at the beginning of each line in the included inputs.conf. This action will comment these lines out so that Splunk will ignore them (recommended). Here's an example of an inputs.conf stanza that has been commented out:
# [tail://\exchangepath\server_name.log]
# disabled = false
# sourcetype = exchange2003-eventtracker

Proceed to the section "Checking event types."

When the source types don't match

If the source types don't match, however, you need to make note of the source type assigned to the data in question in your Splunk server. Once you've done so, change:

  • The sourcetype statement in the inputs.conf included in this application
  • The corresponding clause in the application's props.conf file

For example, given an inputs.conf file of:

[tail://\exchangepath\server_name.log]
disabled = false
sourcetype = exchange2003-eventtracker

and a props.conf file of:

[exchange2003-eventtracker]
TRANSFORMS-exchange = exchange2003-eventtracker-transform
KV_MODE	=	none

where your Splunk server sees the file being tailed as the sourcetype exchangehost5-eventtrack, you would change the sourcetype line in inputs.conf to:

sourcetype = exchangehost5-eventtrack

and the stanza name in props.conf to:

[exchangehost5-eventtrack]

You may also choose to change the name of the transform (shown on the 2nd props.conf line) as well. If you do so, also change the first line of the related stanza in this application's transforms.conf so it matches what's in props.conf.

Now proceed to the section "Checking event types."

Applying this application to data you haven't indexed yet

If your data hasn't been indexed yet, you need to tell Splunk to index it. You can do this in two ways:

Indexing data with the Splunk UI

If you decide to use the Splunk UI, look in this application's inputs.conf file and make sure to manually set the source type that Splunk assigns to this data so that it exactly matches the source type used in the application. Then, look earlier in this document for how to disable this application's inputs.conf file.

Indexing data using the application's inputs.conf file

If you choose to use this application's inputs.conf file instead of setting up indexing through the Splunk UI, open the inputs.conf file for editing. You'll need to change the first line of the stanza that applies to this data. For example, you might change the line:

[tail://\exchange_path\server_name.log]

to match the location of the data you're interested in. Leave the tail:// portion alone, just change the file path or or other Splunk method as you need to.

Once these changes are made, proceed to the section "Checking event types."

Checking event types

Most data source applications contain event types as well as input definitions. It is imperative that you examine the pre-packaged event types and make sure that they will properly match your data, as it is nearly impossible to predict how every environment is set up.

There are some issues to keep in mind when making any changes to this application's event types. For one, the tags assigned to these event types are part of the Splunk common information model. Other applications will expect to see these tags to generate reports, alerts, and more. So if you add, remove, or edit tags, keep this issue in mind. Of course you may have tags of your own that you want to use as well, and this is fine. Just stick to the standard event type tags as much as possible.

To help you do this, it's a good idea to look at the event type that isn't matching and just edit it so that it does match. That way you don't have to mess with the tags. For example, an application containing an event type for when a firewall logs a packet might contain the following two stanzas in its eventtypes.conf file:

[firewall-accept]
disabled = false
query = signature=firewall action="accepted established" OR action="accepted new" OR action=permit
tags = pci host communicate communicate/pass success

[firewall-deny]
disabled = false
query = signature=firewall action="deny" OR action=dropped
tags = pci host communicate communicate/deny success

But what if the query statement (the actual search being run for this event type) doesn't actually match the proper events from your firewall logs? Check your application's eventtypes.conf file to see if there are any comments (lines beginning with #) that suggest that an event type needs to be customized. In other cases, you may simply spot (or test) event types and find that some need to be tweaked.

To make your adjustments, start by running searches until you are able to properly match the events you're looking for. Then, to add the new event type to your system, either:

  • Save the event types through Splunk, making sure to use the same tags as were in the original event type statement (or other appropriate event type tags from the Splunk interface standard). If you do this, be sure to comment out or remove the original event type statement from the application's eventtypes.conf file. The instructions for this are nearly identical to the earlier instructions on how to do the same to a stanza in inputs.conf.
  • In the eventtypes.conf file that came with this application, replace the query for the event type in question with your new search. For example, say that you ran this search:

signature=firewall action="accepted" OR action="connection established"

To change the event type in this application's eventtypes.conf, you would then take the original stanza:

[firewall-accept]
disabled = false
query = signature=firewall action="accepted established" OR action="accepted new" OR action=permit
tags = pci host communicate communicate/pass success

and change it to:

[firewall-accept]
disabled = false
query = signature=firewall action="accepted" OR action="connection established"
tags = pci host communicate communicate/pass success

It's not a bad idea to change the name of the event type as well, such as changing firewall-accept to custom-firewall-accept. If you choose a standard term to add to the new event type names, you can tell immediately which ones you customized or created from scratch.

Again, making sure that you stick with Splunk interface standard event type (and other) tags will ensure that your application continues to operate well in conjunction with other Splunk interface standard-compliant applications.

Loading your application

Once you save all of your changes to the .conf files, restart your Splunk server to load your new application.

Personal tools
Hot Wiki Topics


About Splunk >
  • Search and navigate IT data from applications, servers and network devices in real-time.
  • Download Splunk