At times I wonder what is wrong with Host-based Intrusion Detection Systems: they seem to be ignored, maligned and generally considered "useless".
It should be noted that by HIDS I don't mean your ever so pointless anti-virus software (as previously discussed in these pages) or your fashionable "personal firewall" but software designed to detect intrusions at the host-level. Anti-virus software and "personal firewalls" fit in a very very loose definition of HIDS: they both protect you from external attack, not attacks perpetrated on your own system.
What I call a HIDS is a piece of software which detects violations of system-level policies. One of the best and oldest examples is Tripwire (or its GNU-licensed replacement AIDE) which is designed to monitor changes made to a filesystem.
Modern company security practices seem to overlook internal security (see also a separate rant) because it involves lots of unfashionable effort. Amongst the victims is HIDS often maligned as being impractical, "trigger happy" and heavy on the configuration side. The usual excuse surfaces: "I have to put work into it so it is not worth my time". The actual truth is that it isn't exciting to check for modification of key system files except as a forensic exercise later. The official motto is: "Prevention? No thank you".
Let us consider Tripwire: like all software it can be installed and configured in many pointless ways. For example you could monitor your whole hard-disk for file changes, guaranteed to fill your logs with false alarms, or you could have the file checksum database on the same disk as the one being monitored ready to be modified by the wily hacker. A more sensible configuration such as limited monitoring of key system files (how about /etc, /boot, /vmunix, /lib and /sbin to start with?) with the database on read-only media (a floppy disk with the write-protect tab set is an ideal choice - the database isn't very big) can already help quite a bit. SImply run tripwire via cron at suitable intervals, say hourly, and have the results mailed to you.
The side-effect of an "instrumented" server is that not only can you detect malicious changes to your filesystem but also those clumsy-handed sysadmin days when "I thought I was editing on machine x but actually it was machine y" takes place.
Clearly Tripwire alone isn't enough, you need to instrument your systems a bit better. Once upon a time full system accounting was relatively common for reasons which had little to do with security, i.e. charging on a per-cpu cycle basis, although it did give rise to a famous "catch the hacker" story when an accounting discrepacy was found. It is difficult to advocate full accounting in 2004 as the volume of data has become prohibitive (it is worthwhile remembering that full accounting means that every single command typed is recorded for posterity) but there is value in limited accounting. This already takes place in the various log files which litter a modern system which contain a wealth of very useful information.
How do we detect an "insider job" on a modern Unix system? Well, let us assume that we do have Tripwire installed but little more. Let us further assume that no files have been modified but our log watcher (there are ample choices here) has detected that the volume of logs is increasing above the usual rate. An example? The mail log is twice the size of a usual day or perhaps the database access log has grown excessively. What happens next? Instead of crying wolf the first requirement is an in-depth analysis of the logs in question. Perhaps it is end-of-year and the database is burning away for the accountants or there is a new virus on the loose which is killing sendmail.
This is just the beginning but how many locations actually realise what wealth of information about system misuse lies in their logs? Very few sadly. A remarkable amount of analysis can be done with a "poor man's HIDS", also known as syslog. It might not be hacker-resistant (it doesn't take much to wipe logs) but it can definitely be used as an early warning mechanism.
And once we've re-learned how to use logs how about giving HIDS another whirl?
Not a day passes without the IT world being reminded about the terrible threat of virii, trojans and those pesky "Internet hackers". As a matter of fact there is so much available on the topic that you might as well switch off your preferred virus newsfeed and assume that about 80% of all e-mail you receive is either viral or spam.
Of course the beauty of all this is that companies are looking at their perimeter as if it was the only chance of survival they had in the dark woods of the Internet. They believe anything their firewall or IDS vendor tells them, stare in awe at highly secure Windows-based firewall solutions (or indeed fancy webserver-based firewall products) and bask in the sunshine knowing that their enterprise is safe.
Once upon a time pretty much everyone quoted that 80% of fraud was internal, that is to say that it was perpetrated by someone with intimate knowledge of the enterprise he wanted to defraud. Take banks which go an extra length to make sure that cashiers can't walk out with cash or indeed many other "physical" examples. It would be pretty logical to assume that in the electronic world of Word documents, e-mails and assorted other digital documents this concern about internal fraud would remain.
Apparently not.
A few basic concepts are a complete mystery to many companies: separation of duties (or: "why should the engineering department see the finance servers?"), document tracing (or: "how the heck did this document get out?") not to mention basic IT security (or: "how did they break into this totally unpatched, wide-open, system?").
The reason for this internal debacle is simple: a combination of laziness and fashion. What makes you look like a digital hero? Fighting the wily hacker or securing the internal servers? The fact that the ever so dangerous wily hacker is more often than not a script kiddie barely out of elementary school is irrelevant, it is still more interesting than actually thinking about the real dangers for a company.
So what happens is that people comfortably log into key servers, prowl databases of confidential information and then walk off with the data. Managers often deny this takes place as their workforce is always invariably honest, happy and well-managed until they discover the inevitable "companyzsucks.com" website run by disgruntled employees... This is then followed by ample management chest thumping, sacking of the disgruntled employees and total disregard for the security of the data they might have had access to.
Some people argue that pentesting is the answer to evaluating internal security but think about it for a second: how could it possibly be the answer? What you have is an external team trying to get to valuable data.
Why don't you try asking your top systems manager to walk out with as much valuable data as possible and see what he does? Then try doing the same with some closet geek in accounts. If you follow this up with some sums about the cost of the wily hacker versus this exercise you might discover that installing a HIDS and proper inter-departmental firewalling is more cost-effective than spending millions on the latest fad in perimeter protection.