March 14, 2003

IDS mantra, part 1: "no, an IDS isn't an anti-virus"

The issue of educating users about security remains one of the great challenges which nobody wishes to tackle. In the meantime amazing thoughts permeate the user community leading to rather pointless uses of technology which then evolve into serious practices.

Something which I've never quite managed to fathom is why one precious lesson from real life security simply will not be digested and applied in IT security. This eventually leads to the question: "why do we insist in consdering IDS just like anti-virus software?" (and indeed: "why do we blindly trust anti-virus software?").

In real life security you might have noticed that guards, sentries, bouncers and porters perform their job by a rather straighforward system: they have a list of conditions which allow entry to the premises. At times this list can be simply described as "the following people are allowed in", sometimes it can be a list of more complicated conditions.

There is one fundamental unifying principle: you define what is allowed.

Why? Well, let us imagine that instead of listing the people allowed into a research lab you listed every person not allowed in. There are about 8 billion people on Earth with more being added every day. This means a rather long list and, furthermore, one which will never be complete. My humble suggestion is that this simple change would not enhance the work of the poor security guard.

Now let us transpose this to the electronic world: what does an anti-virus do? Very simple: it has a long list of patterns of known virii to be matched against the data at hand. If there is a match then "Houston, we have a problem", otherwise all is pretty. Of course there are heuristics on top which attempt to match patterns of "viral behaviour" to catch as-yet undefined virii. The bottom line is that:

1) you need to keep the patterns updated or the whole program is worthless,
2) if something new is written which is not in the pattern and doesn't exhibit known viral behaviour then it gets through.

To be pedantic one should say that you should constantly update your patterns, not just once a week or once a day, because you don't know when the virii are being released. It might come as a surprise but virus writers don't tend to wait for 6pm on Friday to relase their latest creations allowing virus analysts to work through the weekend and have rules ready for the 8am Monday updates.

Now, how does an anti-virus software help you to catch the virus released precisely ten minutes after your last update which uses a new polymorphic technique? Simple: it doesn't.

Bearing the above in mind let me introduce the concept of IDS. An IDS is meant to be an "Intrusion Detection System" which has an amazing resemblance to what a sentry posted outside a door does in real life.

So why exactly are we feeding IDS a set of rules designed to detect malicious traffic? Well, one of the reasons is human curiosity. Consider the following: you are given the choice to work in a lab examining all sorts of malicious code to develop signatures or you can spend a month analysing traffic to define the expected behaviour for your network. Not surprisingly people choose the first option.

Choosing the first option means that you will develop nothing more than a network anti-virus lookalike.

When the latest (at the time of writing) MS/SQL worm hit the network there was a rush to write signatures for it. Why did nobody stop and ask the right question: "why exactly are we allowing port 1433/udp incoming into our network?"

In the same way an IDS should be setup to alert on non-legitimate traffic and then have a look at signatures on the data which is being let through. This process is called "white listing" and is exactly what a security guard does (or should do) when it vets people before allowing them access.

The usual objection is: "I can't define my legitimate traffic!". The reply to this depends on the network we are looking at: if you are an ISP then I am prepared to believe you but if you are a corporate site then your answer means "I have no clue of what travels on my corporate network or indeed of what travels through my firewalls". This in not the ideal situation to be in.

So, an IDS is an anti-virus if you set it up to act like one and, like anti-virus software, will fail you because the chap writing the latest virus toolkit is always one-up on you. Why? Simple, he buys the latest anti-virus software and tests his code until it is no longer detected.

The real work is in training people to learn about how their network is meant to behave and then define rules appropriately to catch unexpected behaviour. Unfortunately it is rather less boring and ego-boosting than a post on BugTraq but would improve security dramatically.

Posted by arrigo at March 14, 2003 01:20 AM