Set up Splunk for Cisco IronPort Web Security Appliance

From Splunk Wiki

Jump to: navigation, search

Use the Splunk for Cisco IronPort WSA add-on to consume, analyze, and report on data from Cisco IronPort WSA devices.

Splunk for Cisco IronPort WSA is designed to work in conjunction with the Splunk Cisco Security Suite app. Install these products together to access reports and dashboards that give you visual insight into the performance and effectiveness of your Cisco IronPort WSA implementations.

When you first download the Splunk for Cisco IronPort WSA add-on, you'll decompress the file and place the resulting folder into either %PROGRAMFILES%\Splunk\etc\apps (if you use Windows) or $SPLUNK_HOME/etc/apps (if you use Unix/Linux). Then restart Splunk via the CLI (using the splunk restart command) or the GUI.

After you restart your Splunk instance you can select the Splunk for Cisco IronPort WSA add-on from the Home page. This will bring you to the Setup page for the add-on. See "Getting data into the add-on," below for more information about this page.

Note: When you first enter the Cisco Security Suite, it will give you an opportunity to enable or disable the Splunk for Cisco IronPort WSA add-on. Keep in mind that if you disable Splunk for Cisco IronPort WSA here, you will not receive data from it when you use the Cisco Security Suite.

Get data into the add-on

To use the Splunk for Cisco IronPort WSA, you need to get data from its access logs into Splunk. One way to do this is to configure your Cisco IronPort WSA appliance to export its access logs to a directory that is accessible by your Splunk implementation.

Follow these steps to set up a log subscription in the Cisco IronPort WSA appliance, have it push the log to a place where Splunk can get it, and then configure Splunk to get the log data and process it so that it is usable by the Splunk for Cisco IronPort WSA add-on.

1. In your Cisco IronPort WSA appliance, set up a log subscription for the access logs and/or W3C access logs. You can configure the appliance to format these logs in either Squid or W3C format.

    • In general it is best to use the Squid format if possible because it reduces the number of steps required to configure the inputs for the Splunk for Cisco IronPort WSA add-on. Note that the Squid logging option provides a fixed format. The Splunk for Cisco Ironport WSA add-on is designed to extract fields using this format, so don't alter it.
    • If you decide to go with the W3C format, note that in order for the W3C format to work with Splunk for Cisco Ironport WSA, you need to supply the field header to Splunk in order to properly extract fields. See the subtopic "Extract relevant Cisco IronPort WSA fields from W3C-formatted access logs," below, for more information.

2. Arrange to have the logs exported or transmitted from Cisco IronPort WSA.

    • Logs can be exported on a regular interval to a remote server via FTP or SCP. If your logs are strictly text-based, they can also be transmitted via UDP/TCP using network port 514.
    • Ensure that the logs are being sent to a directory on a machine that is accessible by your Splunk implementation.

3. Configure Splunk to monitor a directory or folder for the incoming logs (if they are being exported there via FTP or SCP) or listen to UDP/TCP port 514 (if that is how you are receiving them).

    • For more information about configuring a monitor input for a file or directory data source, see "Monitor files and directories" in the Getting Data In Manual. Alternatively you can take a look at the recipe in the same Manual.
    • For more information about configuring a TCP or UDP network port input, see "Get data from TCP and UDP ports" in the Getting Data In Manual. Alternatively you can take a look at the "Syslog - TCP" and "Syslog - UDP" recipes in the same Manual.
    • When you configure the inputs for the Cisco for IronPort WSA add-on in Manager, you should override the source types that would ordinarily automatically be assigned to them. Be sure to give an input for Squid format logs a source type of cisco_wsa_squid. An input for W3C format logs, on the other hand, should have a source type of cisco_wsa_w3c. For more information, see "Override automatic source type assignment" in the Getting Data In Manual.

4. Set up additional configurations, as required, and as described in the following subsections.

    • If you export your Cisco IronPort logs in the Squid format and set their input source type to cisco_wsa_squid, there's nothing more to configure at this point.
    • If you export your Cisco IronPort logs in the Squid format but require an alternative name for your source type due to naming conventions within your organization, or if you have already indexed your Cisco IronPort WSA access logs with different source types and cannot reindex them, you will need to manually configure search-time field extractions and event types for your IronPort data. For more information, see the subtopic "Extract relevant Cisco IronPort WSA fields from Squid-formatted access logs," below.
    • If you export your Cisco IronPort access logs in W3C format, you need to create a special search-time field extraction in order for Splunk to process it properly. For more information see the subtopic "Extracting relevant Cisco IronPort WSA fields from W3C-formatted access logs," below.

Extract relevant Cisco IronPort WSA fields from Squid-formatted access logs

As mentioned above, all of your inputs for Squid-formatted Cisco IronPort WSA access logs should be configured so they give this incoming data a source type of cisco_wsa_squid. But if for some reason this isn't possible (you have already indexed a significant amount of Squid-formatted access log data without that source type, or your need to give it a source type that fits a specific naming convention), then you need to set up the following configurations to ensure that the Splunk for Cisco IronPort WSA can extract the fields it requires for its reports and dashboards.

Depending on your situation, you must either rename the existing source type OR map the required search-time field extractions and event type to your source type. You do not need to perform both sets of actions.

Rename your existing source type

To rename the existing source type, simply add the following stanza to props.conf in $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/local:

[put_your_ironport_wsa_squid_log_sourcetype_here]
rename = cisco_wsa_squid

For more information about source type renaming, see "Rename source types" in the Getting Data In Manual.

Map your existing source type to the required field extractions and event type

To map your existing source type to the lookup-based field extractions and event type, add the following stanza to props.conf in $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/local:

[put_your_ironport_wsa_squid_log_sourcetype_here]
KV_MODE = none
MAX_TIMESTAMP_LOOKAHEAD = 19
REPORT-extract = squid
lookup_table = cat_lookup x_webcat_code_abbr

And then add the following stanza to eventtypes.conf in $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/local to create an ironport_proxy event type:

[ironport_proxy]
search = sourcetype=<put_your_ironport_web_squid_sourcetype_here>

For more information about lookups, see "Look up fields from external data sources" in the Knowledge Manager Manual.

For more information about event types, see "About event types" in the Knowledge Manager Manual.

Extracting relevant Cisco IronPort WSA fields from W3C-formatted access logs

If you are indexing Cisco IronPort WSA access logs that are in the W3C format, you need to create a DELIMS-based extraction for this log format so the add-on knows which fields the values map to. The FIELDS value for this extraction will be set to the fields listed in the header of your W3C-formatted access logs. The field names must match up with the order in which the fields were selected in the management interface of the Cisco IronPort WSA appliance. Alternatively, you can determine the field values by viewing the the top of the W3C-formatted access log file.

To create this field extraction, add the following entry to props.conf in $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/local.

[ironport-w3c]
DELIMS = " "
FIELDS = "time","c_ip","field3",....,"field30" 

Note: In FIELDS sure to list all of the fields included in the W3C-formatted log header. To function properly, the reports and dashboards powered by the Splunk for Cisco IronPort WSA add-on require the following fields:

  • cs_username
  • c_ip
  • x_webcat_code_abbr
  • x_webroot_threat_name
  • x_wbrs_score
  • sc_bytes
  • cs_url
  • s_hostname
  • x_acltag

For more information about DELIM-based search-time field extractions, see "Create and maintain search-time field extractions through configuration files" in the Knowledge Manager Manual.

Rename your existing source type

If you already have Cisco IronPort WSA data in Splunk that is in W3C format, you'll need to rename the existing sourcetype. To do this, simply add the following stanza to props.conf in $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/local:

[put_your_ironport_wsa_W3C_log_sourcetype_here]
rename = cisco_wsa_w3c

For more information about source type renaming, see "Rename source types" in the Getting Data In Manual.

Find your Cisco IronPort WSA data in Splunk

If you have set up the access log inputs correctly, you should be able to search on your IronPort WSA events by their source type values: cisco_wsa_squid and cisco_wsa_w3c. You can also locate events by searching on the extracted fields and the ironport_proxy event type.

Configure the add-on

Here's some information about configuring the Splunk for Cisco IronPort WSA add-on.

Change the default schedule for scheduled searches

Splunk for Cisco IronPort WSA ships with three searches that run every 3 hours (if you have a Splunk Enterprise license).

  • Cisco WSA - Acceptable Use - DataCube
  • Cisco WSA - Security - Datacube
  • Cisco WSA - Network Resources - Datacube

To change the schedule for any of these searches, navigate to Manager > Searches and reports and click the name of the search you want to update to open its detail page. On the update page, you can change the interval schedule for the search.

For more information about defining search schedule intervals, see the subsection on search scheduling in "Create an alert" in the User Manual.

Configure and modify lookup values

You can modify the usage and severity value for a particular category by editing the following file in the lookups directory of this add-on: $SPLUNK_HOME/etc/apps/Splunk_CiscoIronportWebSecurity/lookups/category_map.csv

Troubleshoot your Splunk for Cisco Ironport WSA install

  • No Cisco IronPort WSA data is present in your Cisco Security Suite reports and dashboards. First, make sure that you've downloaded and installed Splunk for Cisco IronPort WSA correctly. Keep in mind that you have to have the add-on enabled on the Cisco Security Suite app setup page (navigate to Manager > Apps and click Set up for the Cisco Security Suite app).
If that doesn't seem to be the problem, verify that you've set up your input configuration correctly. Are the logs being created by the IronPort WSA appliance? Are they being pushed out via FTP/SCP to the correct folder (or transmitted to the expected port via UDP/TCP)? Is Splunk monitoring the correct folder or listening to the proper UDP/TCP port? Have you set up any necessary configurations so that the correct source types and event types are assigned and fields are extracted? (Incorrect source typing is often the culprit in these situations.)
  • We have Cisco IronPort WSA data, but our reports and dashboards aren't functioning properly. The reports in the Splunk for Cisco IronPort WSA add-on rely on the search eventtype=ironport_proxy and the following usage field values:
  • Business Usage (usage="Business")
  • Productivity Loss (usage="Personal")
  • Legal Liability (usage="Violation")
  • Internet Tools (usage="Borderline")
For instructions on modifying these values, see "Configure and modify lookup values," above.
Personal tools
Hot Wiki Topics


About Splunk >
  • Search and navigate IT data from applications, servers and network devices in real-time.
  • Download Splunk