How Does an IDS/IDP System Detect Attacks?

Intentionally open-ended, this question is a probe to see whether you know the basic detection methods employed in IDP systems. Be prepared to go into depth on signature matching. There are two basic methods that IDS/IPS systems employ to detect and act on traffic. Rules-based or signature matching uses “signatures” or content patterns that match a given definition. For instance, the signature for the IIS command execution vulnerability is “cmd.exe”. This is the same model used by virus scanners. As long as signatures are updated, new attacks are detected. More sophisticated
IDP systems may employ stateful rules in addition to signature matching. With these rules, an IDP is capable of making sense out of two seemingly benign events — for instance, a TCP host scan followed by unsuccessful SSH login attempts. This is interesting if the source has an external IP address.

The second method uses “normal” traffic patterns as a baseline and then runs comparisons against this baseline to detect any deviations or anomalies. This is anomaly detection. Normal traffic is usually defined in terms of specific ports/protocols/services and source-destination pair traffic patterns. These patterns
are usually deterministic and populated through a “learning” period.

For more information on signature matching and anomaly detection, Google “signature matching site:sans.org.”