Your data is important to you, but do you know if others are trying to get at it?

An Intrusion Detection System (IDS) is a system that is able to detect those that are not behaving as they should. In the “real world”, your average home or office alarm system is an IDS, it detects intruders and then does something about it by flashing lights, screeching sirens, and ringing the security company. In the IT world, things are more complex, because, unlike your house, your IT system is rarely locked and unused when you are away. The IDS has to discriminate between all the traffic on your systems that is supposed to be there and weed out that which shouldn’t be there.

Intruder alert: Intrusion Detection Systems 1. IDS types 2. Intruders, responses and how well do they work? 3. IDS vs IPS 4. Where to deploy? About RMIT Test Labs

How does an ids work?
An IDS can be either a software or a hardware solution that is designed to detect unauthorised use of, or attack on, a computer system or network. The IDS looks for unauthorised attempts to gain access to a system, escalate privileges on an authorised system, or decrease the availability of a system, either from inside the organisation or from the Internet. An IDS is just one part of an interlocked and overlapping security policy.

IDSes come in many forms, with different ways of monitoring and analysing the available data. IDSes monitor events at three different levels: network, host, and application. They can analyse these events using two techniques: signature detection and anomaly detection. Some IDSes have the ability to take action when an attack is detected, but this is something we believe you should think very carefully about and obtain legal advice before attempting.

Of the two detection methods, signature detection is most commonly used in commercial IDS products, but anomaly detection is newer and growing.

Signature-based detection
Signature-based detection looks for activity that matches a predefined string that uniquely describes a known attack. Signature-based IDSes must be specifically programmed to detect each known attack. This technique is extremely effective against known attacks but it must be updated constantly to keep abreast of new attacks.

Anomaly-based detection
Anomaly-based IDSes define intrusions by identifying unusual behaviour (anomalies) that occur on the system or network being protected. The reason they work is based on the fact that the behaviour of normal workers and attackers is different and therefore the two can be individually identified to a degree. The original standard must first be measured by looking at the work patterns and bandwidth of normal use, monitoring is the done by continually comparing that against current use. There is an inherent risk in anomaly based IDSes in that “average” workload is almost impossible to determine. This is countered by the fact that with an anomaly-based IDS it is possible to detect never-before-seen attacks. Some signature-based IDSes include limited instances of anomaly detection, but few rely solely on this technology.

What types of IDS are available?

Network-based IDSes. Most of the IDSes on the market are based around Network IDSes (NIDS). NIDS work by capturing data from one or more points central to the network and reporting back to a management console. The capture systems must be placed in the network such that they can see all passing traffic. In a fully switched network, there may be difficulties in capturing data unless you can configure your switches to pass a copy of all the traffic to a specific port for the IDS.

Pros:

• You can listen to a fairly large network with just a few machines.
• The system is transparent as the unit collects traffic information.
• All traffic between the console and the NIDS collector can be encrypted or on a separate network for complete security.

Cons:

• There may be a lot of traffic passing the system, possibly more than the system can process. This will cause difficulties in detecting intruders when loads are high.
• The need to process packets quickly may mean that you have to turn off some of the features to keep up with traffic volumes.
• Fully switched networks can be difficult to capture as traffic is not replicated across all ports like it is in a non-switched network.
• Unable to analyse encrypted traffic.

Host-based IDSes.
The Host based IDS (HIDS) look at what is happening on the computer it is installed on. This allows the IDS to look very specifically at what is happening on that machine via the log files and/or the internal auditing systems. There are two main types of HIDS: host wrappers/personal firewalls and agent-based software.

Host wrappers or personal firewalls are configured to look at all network packets, attempted connections, or attempted logins to the monitored machine. Host-based agents are designed to monitor accesses and changes to critical system files and changes in user privilege.

Ideally your HIDS will simplify the administration of a set of hosts by having the administration functions and attack logs all report to a central IT security console.

Pros:

• Able to detect a large range of local attacks.
• Encryption is generally not in the way if the data is decrypted at the server.
• No problem with switched networks.

Cons:

• Each host often must be installed and maintained separately.
• Because the IDS is on the host, the IDS may be attacked and disabled first.
• May not see a widely dispersed network scan.
• May get swamped in a Denial of Service Attack (DoS).
• Consumes processing power and network resources of the server it’s protecting.

What sort of intruders are we looking for?

Theoretically, if a vulnerability or attack is known, all systems should be patched, or workarounds applied and thus the need for a signature based IDS would be nil. Unfortunately the reality is that many systems are not patched or upgraded as vulnerabilities are discovered. This is clearly indicated by the number of system compromises that occur everyday, and the fact that most of the problems are predominantly old, well-known problems, with fixes available.

Problem response
Most IDS system consoles are configured to log everything to a database and e-mail the Security Officer (SO) should a problem occur. The problem with this is that an attack like Nimda or Code Red could easily see an administrator flooded with e-mail, and you end up with a system that provides too much information to process. One system we have seen in use is to set the threshold for e-mailing or paging fairly high, but to also have the console beep an alert when it detects a problem. This allows the SO to keep an eye on the odd problem as it crops up, but if the system is sounding increasing alerts, they will know there is a real problem.

If the SO is not always around, or there is a reason for heightened security, some IDSes can be configured to automatically respond to attacks. This may take the form of a simple e-mail or page as above, or could include a more active response to stop the attack in progress and then block that entry point.

Direct intervention to disrupt communications between an attacker and victim is often called session sniping or knockdown, which is performed by injecting packets to break down the connection that triggered the response. The most effective way to knockdown a TCP connection is to forge packets to reset the connection. To do this, the IDS must forge packets to send to one or both systems with the TCP Reset bit set.

Other intervention methods include reconfiguring the perimeter routers and firewalls to block the IP address of the attacker, or block the protocols that are being used. In severe cases, it may be better to break all communications to the targeted system than have it compromised. Further responses may include attempts to actively gain information about the attacker’s host or site, or even attack it in return. Again, we stress that you should seek legal advice before turning these functions on.

How well do they work?
IDSes are used by many organisations, large and small, to protect their networks and systems. Any business that takes its security seriously should have an intrusion detection system as part of its security suite.

On their own, IDSes work fairly well, but they are often too late detecting the problem and shut the gate just as the horse slips out. Implementing IDSes as one layer in a multi-layer overall security architecture (such as firewalls, access control and authentication mechanisms, monitoring tools, vulnerability scanning tools, ID systems, and security training) makes penetration by external intruders more difficult while making intrusion prevention and detection somewhat easier.

Intrusion detection is needed because in practice, firewalls cannot provide complete protection against intrusion. Experience teaches us that we should never rely on a single defensive line or technique. A firewall generally serves as an effective filter, stopping many attacks before they can enter an organisation’s networks. However, firewalls are vulnerable to errors in configuration and ambiguous or undefined security policies. They are generally unable to protect against malicious mobile code, insider attacks, and unsecured internal networks and interfaces. Firewalls rely on the existence of a central point through which traffic flows when the growing trend is towards geographically distributed networks with inside and outside users traversing the same subnets and, therefore, the absence of central points for firewall monitoring purposes.

The principle of Defence in Depth is common in physical security, and so it should be the same in IT Security.

IDS vs IPS

Let’s clear up a common misconception. Intrusion detection and intrusion prevention aren’t different names for the same market segment—they’re different names for two distinct categories of security products.

• IDSes are installed on network segments.
• IPSes are installed on servers and desktops.
• IDSes require expert tuning.
• IDSes require more administrative overhead.
• IDSes can’t parse encrypted traffic.
• IDSes and IPSes should both have a central management console.
• IDSes have more potential for identifying hackers.
• IPSes can better protect applications.
• Intrusion prevention products are ideal for blocking Web defacement.
• Neither an IDS nor an IPS is a replacement for firewalls.

Understand the IDS challenge
Intrusion detection systems (IDSes) are surveillance products. Using an IDS is somewhat like putting an x-ray machine on your network so that you can examine your packets to see what’s inside. IDSes are really more similar to protocol analysers or smart sniffers than they are to intrusion prevention systems (IPSes).

There are a few fundamental problems with how some IDSes work today. First, as more and more network traffic becomes encrypted, IDSes become useless because they can’t parse encrypted traffic. Second, as networks become more heavily switched, they typically see only a small amount of the traffic on your network. On a switched network, you need to greatly increase the number of intrusion detection sensors to monitor traffic on all the network segments. On large networks, this means that the total cost of ownership of IDSes can be very high. Third, IDSes generate a huge number of false positives, telling you that your network is being attacked when it’s not. These three problems are leading many companies to switch to IPSes.

Leading vendors in the intrusion detection market include Cisco, ISS, and NFR. Some IDSes are sold as software packages you install on top of a leading operating system. Others are sold as turnkey appliances, commonly called “sensors” by the companies that make them. Typically, these devices work by monitoring the traffic on the network, noting which devices they are communicating with and categorising the types of traffic interacting with the devices. Traffic patterns are compared against known attack signatures, and alarms are typically set to go off according to certain thresholds and severity levels. For example, a syn-flood attack might be set to a severity level of high, and an ICMP flood might be set to medium.

It is truly important to tune an IDS to report only the minimum data needed to detect an attack. Storing information on every packet header and payload is not useful, and in the long run, it will just create more work and overhead by taking up valuable disk space, requiring additional backups, and increasing storage requirements.

Preventing intrusions
IPSes are like deadbolts. They simply stop the attack. They do not analyse it and then effect a response. Where IDSes generally monitor network segments, IPSes are typically host-based products that get installed on the actual servers and desktops they are slated to protect.

Leading vendors in the intrusion prevention market include SecureWave and Entercept. These products typically work at the application level by analysing a proposed user action before it accesses and/or modifies any mission-critical files. The requested behaviour of the application must match the desired behaviour that has been previously defined by a standard set of rules. If the proposed action is unusual, the rules that govern the application’s behaviour will prevent the action from executing.

Some IPSs compare a checksum of the executable with a known good checksum list. If the proposed execution is legitimate, the application is allowed to execute. If there is a mismatch in the checksum hash, the application is not allowed to execute. Unlike IDSes, with IPSes, the logic is applied before the application is executed in memory. Other IPSes work by intercepting systems calls.

An IDS still has its place
Although IDSes have their problems, they can still offer value to an organisation or law enforcement agency under the right circumstances. For example, if your network is under attack and there has been a large loss of valuable assets such as credit card numbers or if money has illegally been transferred to the wrong accounts, using an IDS is a smart way to try to catch the perpetrator.

Because IDSes need to collect a large array of traffic to understand anomalous patterns, they typically require a lot of massaging to tune them, interpret the information, and identify false positives. In fact, monitoring IDSes can be a full-time job. We have seen instances where a hacker has actually exploited an IDS, causing it to create a denial of service attack against the organisation it’s in place to protect.

Bottom line
If you work for a financial institution, you should probably deploy both an IDS and an IPS. If your systems contain medical records that include detailed patient information that doctors use to make treatment decisions, you should probably deploy an IDS and an IPS. However, if losing your data would, at worst, create a big inconvenience while your operations team secured the perimeter and the hosts and restored the data, it might be more worthwhile for your organisation to install only an IPS. Certainly, if there are no staff resources dedicated to tuning an IDS or providing the ongoing expert analysis required, there is no point in installing one.

Where to deploy?

Depending on your security practices and topology, you'll typically consider four areas for monitoring. These are as follows:

• Network perimeterâ€"This includes any entry/exit point, such as on both sides of the firewall, dial-up servers, and on links to any collaborative networks. These links tend to be low-bandwidth and are usually the entry point of an external attack.
• WAN backboneâ€"This is a frequent area of unauthorised activity.
• Server farmsâ€"Servers are generally placed on their own network segments and connected to switches. The problem with placing a sensor in this location is that IDS systems cannot keep up with high-volume traffic. If traffic is too high to monitor all of your servers, choose the targets of highest value and install sensors to monitor those specific targets.
• LAN backbonesâ€"IDSes are usually impractical for LAN backbones because of their high amount of traffic.

When deciding where to deploy your sensor(s), consider what is most valuable and the attacker's most logical avenue of approach. You also need to make sure that your IDS doesn't degrade the performance of the network segment that you're monitoring.

Software-based IDS
A software-based IDS is a solution that you load on a compatible operating system to monitor and respond to network activity. An example of a software IDS is Internet Security Systems' RealSecure. Its system consists of two major elements:

• The RealSecure Sensor is software that you load and configure on a platform to provide broad-based detection, prevention, and response for attacks and misuse that originate from across a network. It sends automatic responses to improper activity and logs events to a database, and it can block/terminate a connection, send an e-mail, suspend or disable an account, and create a user-defined alert.
• TheRealSecure SiteProtector is the software-based management platform. SiteProtector unifies the management of RealSecure IDS sensors and allows grouping of these sensors to provide real-time internal and external correlation of threats. The RealSecure SiteProtector also enables you to operate and monitor remote sensors and respond to identified intrusions.

IDS appliances
IDS appliances are complete and fully loaded systems that require no additional hardware or software to monitor the network segments. An example of an IDS appliance is Cisco IDS (formerly known as NetRanger). This system consists of two major elements:

• The Secure IDS Sensor is the appliance that you place at a specific connection to be monitored on your network, or you can install several appliances to monitor multiple locations. It detects unauthorised activity navigating the network by analysing traffic against rules-based signature files. When unauthorised activity is detected, the sensor can send alarms to a management console with details of the activity and can control other systems, such as routers, to terminate the unauthorised session(s).
• The Secure IDS Director is a software-based management system that centrally monitors the activity of single or multiple Cisco Secure IDS sensors located on local or remote network segments. The Cisco Secure IDS Director allows network and security technicians to quickly pinpoint the location and type of attack, qualify its severity, and instantly respond.

• Cisco
  IDS V4.0
  www.cisco.com
• Computer Associates eTrust Intrusion Detection
  www.ca.com
• Enterasys
  Dragon 6
  www.enterasys.com
• ISS
  RealSecure Network Sensor 7.0
  www.iss.net
• Snort (Free) Open Source Network Intrusion Detec. www.snort.org
  
• Symantec
  Host Intrusion Detection System
  www.symantec.com
• Tripwire
  Host-based IDS
  www.tripwiresecurity.com

Subscribe now to Australian Technology & Business magazine.

About RMIT Test Labs

RMIT IT Test Labs is an independent testing institution based in Melbourne, Victoria, performing IT product testing for clients such as IBM, Coles-Myer, and a wide variety of government bodies. In the Labs’ testing for Technology & Business, they are in direct contact with the clients supplying products. Their findings are their own—only the specifications of the products to be tested are provided by the magazine. For more information on RMIT, please contact the Lab Manager, Steven Turvey, at stevet@rmit.edu.au.

Intruder alert: Intrusion Detection Systems 1. IDS types 2. Intruders, responses and how well do they work? 3. IDS vs IPS 4. Where to deploy? About RMIT Test Labs

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.