Features
Wolfgang Selzer, Exponant
A search tool that indexes and correlates common IT data is making waves locally.
A skany system administrator what’s high on his list of pain points and chances are you’ll hear something about extracting meaningful information from log files quickly.
Most IT systems, be they small business applications, enterprise middleware stacks or mail servers, produce text files that log what happened and when it happened. But when things go wrong, it often needs considerable skill and experience to make sense of the data, more so when that data is scattered across multiple log files. Compounding the problem is that most log data is unstructured and no two companies are exactly alike.
Enter Splunk, a deep diving data tool named after the old nickname for caving, spelunking. Splunk indexes almost any kind of data and provides a platform for users on which to build their own queries. So the systems administrator at a large telco, for example, can drill into log data in minutes to discover that a database problem is being caused by a failed link on the other side of the country.
Wolfgang Selzer, MD of Centurion-based Exponant, says that that kind of operational efficiency is attractive.
“The intelligence that can be gathered from all these logs in the Splunk way is the real value. At the moment we believe it’s doing so well because it has a recession-proof tool – it’s all about operational efficiency. Being able to diagnose and resolve problems faster and cheaper goes straight to the bottom line.”
And Splunk is doing well. Selzer says that as the official partner for the product in South Africa, he has access to a number of downloads by local companies. Although it’s free to use Splunk to crunch data up to 500Mb per day, some companies have already purchased an enterprise licence, including Vodacom.
“We were approached by Vodacom to become a conduit and supply professional services on the Splunk side. In Exponant we had an information security and open source team and Splunk in the US made us its sole partner in Southern Africa.
With that we started going to market using a Web 2.0 type business model – we don’t have to go and find customers, they come and find us since people are already aware of Splunk. So we follow up the downloads by local companies and offer ourselves as a professional services team to back it up with local support. But we’re also actively targeting customers.”
Splunk’s free-to-use model is subversive, says Selzer.
“The Splunk market is one where a lot of users have downloaded it and seen it work but it’s not mainstream in the company yet. Our approach is not to force them to upgrade but to be there until they can no longer do without it. And then it gets on management’s radar and they want SLAs and an agreed level of service. There are a few new customers who have broken through the free limit and are now using the paid-for version.
After a security incident in a government department, we were able to demonstrate the value of getting an answer within 15 minutes as opposed to three days. There’s also a customer who hasn’t reached the data limit but wants the full version for the mission-critical stuff and the support.”
Getting creative
“If you are an ISP looking at mail logs, for example, it’s very easy to look at those events and see whether your SLA is up to scratch,” he says.
“That sort of thing isn’t trivial to extract from a text log file using standard commandline search tools. It makes consuming log data and timestamp information very easy: you can quickly dig in, find anomalies, exclude certain things and correlate other logs as well. The real value is when you start time correlation:
you can see that at this point, something broke, but five seconds before that there was a database error. There’s one single view of everything. It’s one thing having logs enabled but it’s really hard to make sense of them. It doesn’t have to be security-related – it could be any application. You can analyse a Java application and see what component is giving me the most trouble, what class is giving me the most exceptions.
You do need domain knowledge to get the most out of it: a financial trader sitting in front of a set of his logs will know what’s important and what isn’t.”
Because constructing Splunk plugins requires domain-specific knowledge, an industry of partners has sprung up to address company requirements. Selzer says one particular type of partner provides a value-added service on top of Splunk with reporting and focusing on applying Splunk to a particular technology infrastructure, such as SAP.
“We see our role as building an ecosystem. Although we’re aware of the typical vendordistributor- reseller model, we want to apply more new-age thinking because there are so many different types of partners: technology, services, and reseller."
