A while back I believe I mentioned how, back in 1999, I used to joke that the Bad Guys really needed a database to keep track of vulnerabilities and hosts which they had already scanned. I don't think they heeded my call at the time, nor do I suspect that they are particularly prone to doing so now but an interesting pattern is emerging which I feel worthy of an honourable mention.
For quite some time (since June 2001), I've been placing my IDS logs on the web for IDS researchers and SANS students to use. In particular as an extra bonus I process my daily logs with a small Perl script which produces HTML for me to post. The data which is produced by the script is a good representation, relatively easy to browse and search. I am personally fond of the section with the distribution of attack methods which I always read first as it gives me a good handle on the overall picture.
One of the ideas which has always been drilled into me, in particular by Stephen Northcutt, has been that of "log fusion". There is a lot of value in looking at logs in context and attempting to combine information from, say, Apache, Squid, Sendmail and your IDS (not to mention syslog, of course). In particular this technique is perfect for rooting out false positives concerning web traffic, the best example being the good ol' 3DNS "attacks" which compared against Squid logs become load-balancing attempts after a web request.
Now, who says that you cannot apply "log fusion" in the opposite direction? I've recently been comparing the web accesses to the daily Snort data with attacks and suddently my heart jumped: in the list of attacks I had sites with source addresses which matched exactly the source address of hosts checking the daily log the following day!
What happened? Well, very simple really: if you had developed a new attack and wanted to test it against a set of Snort rules to see if it was detected you would probably run it on a small private segment. But what about modified Snort rulesets? So you search the web with Google and you fall upon my site which not only publishes daily Snort logs but also the rules being used. What follows is that you try your attack against my web site on the assumption that the Snort sensor is there. Then you check the following day to see how much of it was detected. Not bad at all: free remote verification!
There is only one small catch in the whole argument: who says that the producer of the public data is the only IDS running on the network?
Posted by arrigo at January 20, 2003 06:11 PM