Your data is important to you, but do you know if others are trying to get at it?

An Intrusion Detection System (IDS) is a system that is able to detect those that are not behaving as they should. In the “real world”, your average home or office alarm system is an IDS, it detects intruders and then does something about it by flashing lights, screeching sirens, and ringing the security company. In the IT world, things are more complex, because, unlike your house, your IT system is rarely locked and unused when you are away. The IDS has to discriminate between all the traffic on your systems that is supposed to be there and weed out that which shouldn’t be there.

How does an IDS work?
An IDS can be either a software or a hardware solution that is designed to detect unauthorised use of, or attack on, a computer system or network. The IDS looks for unauthorised attempts to gain access to a system, escalate privileges on an authorised system, or decrease the availability of a system, either from inside the organisation or from the Internet. An IDS is just one part of an interlocked and overlapping security policy.

IDSes come in many forms, with different ways of monitoring and analysing the available data. IDSes monitor events at three different levels: network, host, and application. They can analyse these events using two techniques: signature detection and anomaly detection. Some IDSes have the ability to take action when an attack is detected, but this is something we believe you should think very carefully about and obtain legal advice before attempting.

Of the two detection methods, signature detection is most commonly used in commercial IDS products, but anomaly detection is newer and growing.

Signature-based detection
Signature-based detection looks for activity that matches a predefined string that uniquely describes a known attack. Signature-based IDSes must be specifically programmed to detect each known attack. This technique is extremely effective against known attacks but it must be updated constantly to keep abreast of new attacks.

Anomaly-based detection
Anomaly-based IDSes define intrusions by identifying unusual behaviour (anomalies) that occur on the system or network being protected. The reason they work is based on the fact that the behaviour of normal workers and attackers is different and therefore the two can be individually identified to a degree. The original standard must first be measured by looking at the work patterns and bandwidth of normal use, monitoring is the done by continually comparing that against current use. There is an inherent risk in anomaly based IDSes in that “average” workload is almost impossible to determine. This is countered by the fact that with an anomaly-based IDS it is possible to detect never-before-seen attacks. Some signature-based IDSes include limited instances of anomaly detection, but few rely solely on this technology.

What types of IDS are available?
IDSes are generally broken down by what they monitor: the whole network, a specific host, or even a single application. A truly effective IDS will use a combination of network- and host-based intrusion detection. Figuring out where to use each type and how to integrate the data is a real and growing concern.

Network-based IDSes. Most of the IDSes on the market are based around Network IDSes (NIDS). NIDS work by capturing data from one or more points central to the network and reporting back to a management console. The capture systems must be placed in the network such that they can see all passing traffic. In a fully switched network, there may be difficulties in capturing data unless you can configure your switches to pass a copy of all the traffic to a specific port for the IDS.

Pros:

• You can listen to a fairly large network with just a few machines.
• The system is transparent as the unit collects traffic information.
• All traffic between the console and the NIDS collector can be encrypted or on a separate network for complete security.

Cons:

• There may be a lot of traffic passing the system, possibly more than the system can process. This will cause difficulties in detecting intruders when loads are high.
• The need to process packets quickly may mean that you have to turn off some of the features to keep up with traffic volumes.
• Fully switched networks can be difficult to capture as traffic is not replicated across all ports like it is in a non-switched network.
• Unable to analyse encrypted traffic.

Host-based IDSes.
The Host based IDS (HIDS) look at what is happening on the computer it is installed on. This allows the IDS to look very specifically at what is happening on that machine via the log files and/or the internal auditing systems. There are two main types of HIDS: host wrappers/personal firewalls and agent-based software.

Host wrappers or personal firewalls are configured to look at all network packets, attempted connections, or attempted logins to the monitored machine. Host-based agents are designed to monitor accesses and changes to critical system files and changes in user privilege.

Ideally your HIDS will simplify the administration of a set of hosts by having the administration functions and attack logs all report to a central IT security console.

Pros:

• Able to detect a large range of local attacks.
• Encryption is generally not in the way if the data is decrypted at the server.
• No problem with switched networks.

Cons:

• Each host often must be installed and maintained separately.
• Because the IDS is on the host, the IDS may be attacked and disabled first.
• May not see a widely dispersed network scan.
• May get swamped in a Denial of Service Attack (DoS).
• Consumes processing power and network resources of the server it’s protecting.

What sort of intruders are we looking for?

Theoretically, if a vulnerability or attack is known, all systems should be patched, or workarounds applied and thus the need for a signature based IDS would be nil. Unfortunately the reality is that many systems are not patched or upgraded as vulnerabilities are discovered. This is clearly indicated by the number of system compromises that occur everyday, and the fact that most of the problems are predominantly old, well-known problems, with fixes available.

Problem response
Most IDS system consoles are configured to log everything to a database and e-mail the Security Officer (SO) should a problem occur. The problem with this is that an attack like Nimda or Code Red could easily see an administrator flooded with e-mail, and you end up with a system that provides too much information to process. One system we have seen in use is to set the threshold for e-mailing or paging fairly high, but to also have the console beep an alert when it detects a problem. This allows the SO to keep an eye on the odd problem as it crops up, but if the system is sounding increasing alerts, they will know there is a real problem.

If the SO is not always around, or there is a reason for heightened security, some IDSes can be configured to automatically respond to attacks. This may take the form of a simple e-mail or page as above, or could include a more active response to stop the attack in progress and then block that entry point.

Direct intervention to disrupt communications between an attacker and victim is often called session sniping or knockdown, which is performed by injecting packets to break down the connection that triggered the response. The most effective way to knockdown a TCP connection is to forge packets to reset the connection. To do this, the IDS must forge packets to send to one or both systems with the TCP Reset bit set.

Other intervention methods include reconfiguring the perimeter routers and firewalls to block the IP address of the attacker, or block the protocols that are being used. In severe cases, it may be better to break all communications to the targeted system than have it compromised. Further responses may include attempts to actively gain information about the attacker's host or site, or even attack it in return. Again, we stress that you should seek legal advice before turning these functions on.

How well do they work?
IDSes are used by many organisations, large and small, to protect their networks and systems. Any business that takes its security seriously should have an intrusion detection system as part of its security suite.

On their own, IDSes work fairly well, but they are often too late detecting the problem and shut the gate just as the horse slips out. Implementing IDSes as one layer in a multi-layer overall security architecture (such as firewalls, access control and authentication mechanisms, monitoring tools, vulnerability scanning tools, ID systems, and security training) makes penetration by external intruders more difficult while making intrusion prevention and detection somewhat easier.

Intrusion detection is needed because in practice, firewalls cannot provide complete protection against intrusion. Experience teaches us that we should never rely on a single defensive line or technique. A firewall generally serves as an effective filter, stopping many attacks before they can enter an organisation's networks. However, firewalls are vulnerable to errors in configuration and ambiguous or undefined security policies. They are generally unable to protect against malicious mobile code, insider attacks, and unsecured internal networks and interfaces. Firewalls rely on the existence of a central point through which traffic flows when the growing trend is towards geographically distributed networks with inside and outside users traversing the same subnets and, therefore, the absence of central points for firewall monitoring purposes.

The principle of Defence in Depth is common in physical security, and so it should be the same in IT Security.

Subscribe now to Australian Technology & Business magazine.

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.