From Splunk Wiki
Are you using NFS as your index storage?
Splunk writes/reads happen frequently and can involve large chunks of data at once (*.data files). If you're using NFS, you might be experiencing I/O latency. The queues will get blocked if the incoming rate is larger than what the FileSystem can handle. More at: http://answers.splunk.com/questions/250/can-splunk-read-data-to-index-off-an-nfs-mount-can-splunk-store-indexed-data-to