Event Deduplication / Aggregation

Event Deduplication (also known as Event Aggregation) is a technique for detecting that a new incoming alert (event):

1. is similar to an existing event that has already been reported
2. suppressing the duplicate alert

This is somewhat related to Alert Suppression and Event Escalation, but allows finer and configurable control over when to suppress an alert.

Event Deduplication can be configured in the Advanced Services part of the Console.

Here you can select between the default Simple event deduplication (which doesn't do very much deduplication), or advanced event deduplication. Advanced Event Deduplication is what this page will describe further.

Event Reset

The first thing to decide is whether or not to alert when a duplicate event arrives. Usually the new event would be shown if the previous event situation has been 'reset', meaning the system will no longer consider new events duplicates of the previous event because something has changed. Usually this means the previous event was acknowledged, or the underlying error was fixed. If this is the case, and a new event arrives, it should not be considered a duplication but rather a new situation that should be alerted on again.

State vs Event-type events

Some events are 'state' events, meaning they are in a good or bad state (responding to ping or not responding to ping, low disk space or OK disk space, etc). Those are easy to define as 'Fixed' or not.

Other event types are stateless, or Event-type events, meaning they happened, but don't represent a good or bad state. An error listed in the event log, an error received via SNMP Trap or syslog, or a change detected in a file are such situations. These are not good or bad states, they simply occurred.

For Event-type events, you decide how to consider them 'fixed' for use in Deduplication resetting. You can consider them immediately fixed, or fixed after a certain amount of time.

Deduplication ID

The key principle to understand is the Deduplication ID. The Deduplication ID is a text string that represents the essence of the event. If two events have the same Deduplication ID, they are considered the same event for Event Deduplication purposes. You will therefore want to define the Deduplication ID so it combines events that you want considered the same.

For example, the default fields used are:

• Computer ID - an internal unique ID assigned to each monitored computer
• Monitor ID - an internal unique ID assigned to each monitor
• Cleaned Description - the event description text with user names, paths, dates, amounts, etc removed

With these default settings, the two events below would be considered identical, and thus the second event would not fire alerts when using Advanced Event Deduplication:

These are identical because the events are for the same computer and from the same monitor, and the Cleaned Description field (after removing dates, times amounts, etc) is the same too -- they are about low disk space on the same drive on the same computer.

Peeking at the database...

The image above shows some of the fields in the Error History table. Deduplication IDs are shown at the right side. Notice the ErrCount column -- that shows how many incoming alerts were considered duplicates of the shown alert. Instead of alerting on that incoming event, the ErrCount column was incremented and no alerts were fired. If you think you might want a reminder about events that have not reset and thus are suppressing new incoming alerts, look at the Alert Reminders feature.

Global with Override

This setting is a global setting that applies to all monitors. Individual monitors can define their Deduplication ID in a unique way to give added flexibility. This is done in Advanced Monitor Options (bottom of the page).