Pandora: Documentation en: Events

From Pandora FMS Wiki
Jump to: navigation, search

Go back to Pandora FMS documentation index

1 Events

1.1 Introduction

Pandora FMS events system allows to see a real time record of all the events that occur in our monitored systems. The information displayed ranges from any module status change, alerts triggered or retrieved, to system restarts or custom events. By default, in the event view you will see a photo of what is happening of relevance at that time, it is one of the most used views by the operation teams in any type of professional monitoring software.

Events are managed in Events > View Events, where you see the following menu.

Menu eventos.png

This is an example of the default event viewer. The fields displayed in this view can be customized (see Customize Event View section):

Event list.png

In the 726 version of Pandora FMS it was introduced the possibility of sort the events by ID, status, name ...

Event orden.png

The Events viewer shows the event which is a descriptive text of the problem, the origin (agent) which generated it and the event's date. Sometimes there is some other data associated (e.g. the agent's module which generated the event, the group, the tags associated to the module, etc.). If we e.g. click on the magnifying glass, we'll be able to view all the event's details:

Detalle evento 1.jpg
Detalle evento 2.jpg

By default, the events are shown by a specific search, and this can be modified, showing the information in the way that interests us through its different filtering options:

Filtro evento.png

As we can see here, by default (although it can be modified in the setup options), Pandora FMS shows events that are up to eight hours old or less, and shows only those that are not validated. A user who only has access to one group will only see events in that group. By default, groups the events, that is, if we have several events of the same origin and of the same type, it will only show one, and in the detailed view of the event, it will tell us how many events we have equal, grouped in that single item of the list.

There's also the possibility of saving a specific search so that you're able to apply filters you've created before.

Events are the recording and an essential part of a monitoring system.

The operators who see this screen are able to know any information about the monitored system's current state (active events) and all its history (seeing all the validated events), without being forced to look at every single agent. They're also capable of browsing through global figures, data trees, names and visual screens.

The operators should see a "clean" event console which is only going to show the active problems. In this way, you won't even have to create alerts. Just by looking on the screen, you're going to see everything that's going on at all times.

1.2 Operating with events

1.2.1 Validation and status of an event. Auto validation

An event have in three status: new, in process or validated. A default event, as it arrives at the system, is in the New state. When events occur due to module status changes, there will usually be two events: a first event from normal to an incorrect state, and an event back to normal once the problem is solved.

In these cases, the events of transition to an incorrect state (critical or warning) are automatically validated when normalcy is restored. This is what we call event autovalidation and is an essential functionality, as it allows you to hide information that is no longer relevant in the event console. When an event is validated, it disappears from the default initial view of events, since in this view the validated events are not shown by default because they are not considered active problems but past problems.

When we find an event, we can validate it: This will make the system memorize the date and the user who validated the event. It is also possible to leave a comment, e. g."We checked it and emptied some disk on the server":


Event sample4.png

By clicking on the validate button, the screen is refreshed and the validated event "disappears". This is because the default event view only displays non-validated or assigned events, but not validated ones.

Event sample5.png

If we load the event view again, filtering and displaying all events, we will see the event validated (with a green cross on the left) with the information of who validated it, when, and with the text you entered at that time.

On the other hand, instead of validating an event, we can mark it as "in process" in the Responses tab, as you can see below:

Event sample6.png

We can have an event "stopped", or blocked, so that it does not validate itself, and still be seen in the event views, as pending work. It will group the other events that arrive the same (see grouping of events) but will not validate itself. The appearance of the event will be similar to the next one:

Event sample7.png

Also in the Responses tab you can find some other possible actions on the event, such as deleting it or executing custom responses such as the ping on the host.

You can also validate, mark events as "in process" and delete events in an individual way with the features shown below:

Op indi.png

You can also validate, mark events as "in process" and delete events in a massive way as shown below:

Op masiva.png

1.2.2 The Custom Events View

It is possible to customize the fieldas that the Event View shows by default. For that, please click on Events' '-> Custom Events, where you'll be able to choose the fields to be shown.

By default, the fields you're going to see are:

  • Event name
  • Agent name
  • Status
  • Timestamp

Select the fields you want to display from the "Fields available" list and move to "Fields selected" using the arrows. Once selected, press the "Update" button.


Custom events.png

1.2.3 Creating Event Filters

In the section Events> Events Filters. You may create, remove and edit your filters within this window.

Filtros evento.png

If you click on the Create Filter button, you're able to fill out the event fields as shown below.


Crear filtro evento.png


Once the filters have been saved, right from the Event View itself we can load them to display the desired information quickly without having to reconfigure the filter each time:


Event1.JPG


1.2.4 Filtering Events

From the Event View page, you can filter in the event list to search for specific events.

From the event view, you will access the filtering options in Event control filter, and the advanced options with Advanced options:


Event6.JPG


There are many fields, some of which do not require explanation, so we will focus on those that are more relevant or complicated to understand:

  • Event Type: The combo in which you're able to pick the event's type. There are the following types:
    • Agent Creation
    • Alert ceased (oudated)
    • Alert fired
    • Manual alert validation
    • Alert recovered (different to alert ceased)
    • Configuration change (affects to a module from the inventory)
    • Error (generic)
    • Monitor in critical status
    • Monitor in warning status
    • Monitor in normal status
    • Monitor in unknown status
    • Unknown (generic)
    • Systema (generic)
    • New host detected via recon
    • Not normal (generic)
  • Severity:It details the severity of the event, which has nothing to do with the status of the module related to that event. If the event is related to an alert, it will have the same level of severity. These are the five levels of severity that exist:
    • Critical
    • Informational
    • Maintenance
    • Normal
    • Warning
  • Max hour old: The field in which the max. age is determined.
  • Repeated: By default, pandora groups the events, that is, if we have 10 events of the same origin and type, it will only show one, and in the detailed view of the event, it will tell us how many events we have equals, grouped in that single item of the list. We can change this behavior to show us all the individual events.

You may save the filter or load another one by clicking on the 'Load Filter' icon.

1.2.5 Deleting an Event

Another way of managing events is deleting those which aren't interesting any more. Please use the 'deleting events' option to do so. If you click on 'Operation' and 'View Events', there are two ways to delete an event from the event list:

Please click on the gray trash icon within the 'Action' column.

Gest62.png

Automatic event purging

From the configuration it is possible to define the maximum historical events that we want to keep for its elimination. This purging is performed by the automatic maintenance process of the database (Pandora_DB) that should be executed automatically every hour.

Event purge.jpg

Events history

There is also an Enterprise functionality called "event history" that allows you to store in the historical database those events that exceed the deletion date. These events are not accessible through the event view, and are only used for special event history reports.

Event history.jpg

1.2.6 Other Ways of viewing Events

Beside the event's classic view which you may call up by clicking on 'Events' and 'View Events', you're also able to pick public news channels such as 'Sliding Marquee' (a moving list on the top of the browser on a black screen).

1.2.6.1 RSS Events

Pandora FMS also has an RSS Event Provider in order for you to subscribe to it from your favorite news reader. To see the events within a news channel or RSS, please click on 'Events' and 'RSS' and subscribe to it from the news reader.

Gest64.png

Template warning.png

To provide access to event RSS feed, you're required to configure which IPs are allowed to access it. To do so, please click on the field named 'IP list with API access' within 'Setup'.

 


1.2.6.2 Events in the Marquee

It shows the last events in a sliding text-line format. This option is intended to visualize the last events within a monitor like a text screen. You're able to easily customize the number of visualized events or the size, color and filter of the messages by modifying the code within the file named 'operation/events/events_marquee.php'.

Gest65.png

Template warning.png

In order to be able to access the RSS feed of the events, it is necessary to configure the IPs that are allowed access in the IP list with API access field within Setup.

 


1.2.6.3 Sound Alerts

This new way is a lot easier to manage a system without having to always check Pandora's console. You will be able to hear the different tunes if an event occurs even if you are far from the computer (assumed you've attached some powerful loudspeakers). The tune is going to be played until the sound event pauses or if you press the 'OK' button.

The list of sound events which are going to generate the playing of a sound are:

  • An alert firing
  • A module changes to a 'warning' state.
  • A module changes to a 'critical' state.

It's also possible to filter the events by their groups.

1.2.6.3.1 Configuration

There are three types of events the alert sound is going to be attached to. You may configure any appropriate sound from Pandora's Console setup for each type of event.

Event sound.setup.screenshot.png

You're also able to hear the tune even from the setup page. Feel free to test it (if the browser is compatible to multimedia contents) by clicking on the 'Play' button which you're going to find on the right side of any event type.

1.2.6.3.2 Advanced Configuration

It's also possible to extend the list of sounds for all sound events. Please go to the Pandora Console Server and into the directory named '/var/www/pandora_console/'. You may paste your new sounds into the directory named 'include/sounds/' - but if you do, you're also required to consider several things achieving the right performance:

  • The file has to be in a 'WAV' format.
  • It's recommended to take the smallest possible file, because this file must be sent to the browser in order to be played within your browser's window.

There are several possibilities to achieve this:

    • Please select a sound file with only a few second's length for the main alert sound, because it's going to be played ad infinitum.
    • Please convert the sound to 'mono'.
    • Please change the sound's coding to '16bits signed' or less. We're going to lose quality but we're diminishing the file's size by doing this.
  • In order to create or edit sounds, we recommend to use tools as Audacity which is a multi platform open-source tool which is also very easy to use.
1.2.6.3.3 Use

The event sounds are asynchronously 'scanned' every 10 seconds. If an event is received, the preconfigured or default sound for this event is going to be replayed and the window is going to start flickering in red. This window is also going to be placed in foreground of all other opened windows, depending on its browser's and operating system's configuration.

To gain access to the sound events window, you're just required to go to the Pandora Console's left menu and to click on Operation and View Events there. Within the header's event window, please click on Sound Events. It's going to show you a new window, which is a lot smaller than the others.

Event sound.png

This small window is going to be the one which manages all the sound events. It's recommended to leave it open in case any event is received. Inside the window we have several controls that allow us to filter so that the console only jumps according to various filters: group, type of event or specific agent (s). Also, in caseit goes off, a small window will indicate which event is going off.

Press the "Play" button to start the sound console. When an event goes off, press "OK" to restart the console and stop the sound (until another new event rings it again).

Window.event sound.screenshot.png

1.2.7 Exporting Events to a CSV

It's possible to export the event list to a CSV file in order for the events to be processed by or incorporated into other applications.

In order to export the events to a CSV file, please click on 'Operation' -> 'View Events' and 'CSV File'.

1.2.8 Event Statistics

It's possible to gain access to the event's statistics by clicking on 'Operation' -> 'View Events' and 'Statistics'.

Gest66.png

1.3 Event Alerts and Event Correlation

Pandora FMS allows to define alerts on events, which allows working from a much more flexible perspective, since alerts are not generated according to the status of a specific module, but on an event -which may have been generated by several different modules of different agents. This is an Enterprise feature.

There is a corresponding section for creating event alerts in the alert menu.


Menu event alert.jpg

Event alerts are based on filtering rules using logical operators (and, or, xor, nand, nor, nxor), events matching the filtering rules configured will be searched and if matches are found the alert will be triggered.

They also use the templates to define some parameters, such as the days on which the alert will function, however in this case the templates do not determine when the event alert is triggered, but rather it is through the filtering rules that the events that match will be searched and the alert triggered.

Event alerts.png

In order to render the work with them a little easier, the event alert's configuration parameters are identical to the module alerts. A detailed explanation for all of them can be found here. There are only two specific parameters for event alerts:

  • Rule Evaluation Mode: There are two options: 'Pass' and 'Drop'. 'Pass' means that if an event is fulfilled by an alert, the alerts below are going to be evaluated. 'Drop' means that if an event is fulfilled by an alert, the alerts below are going to be stopped from being evaluated.
  • Group by: It allows you to group the rules by agent, module, alert or group. If a rule is e.g. configured for it, it's going to fire if we receive two critical events. If it's grouped by agent, two critical events are required to originate from the same agent. This feature is capable of getting switched off.

Each rule is configured to fire by a specific type of event. The alert will be fired if the condition of the logical equation, which is defined by the rules and its operators, is met.

Event rules.png

The rule's configuration parameters are the following:

Template wip.png

We are working on the translation of the Pandora FMS documentation. Sorry for any inconvenience.

 


  • Name: The name of the rule.
  • User comment: A free-text field intended for a comment.
  • Event: The regular expression that matches the event's text, if it's left blank it is "for any event"
  • Window: The events which have been generated outside the defined time range are going to be rejected.
  • Count: The number of events which have to match the rule to fire the alert.
  • Agent: The regular expression which matches the agent's alias which has generated the event.
  • Module: The regular expression that matches to the module's name which has generated the event.
  • Module Alerts (template): The regular expression that matches the alert's name which has generated the event.
  • Group: The group the agent belongs to. If the recursion box is checked, the rule will also apply to the child groups of the selected group.
  • Severity: The event severity.
  • Tag: The event's associated tags.
  • User: The event's associated user.
  • Event Type .

We could e.g. configure a rule which wears a tag named 'System' and matches to the events generated by any module of any agent of the server group that is named 'cpu_load' in the moment the module moves to a 'critical' state:

Event rule config.png


Info.png

Given the high number of events the Pandora FMS Database is able to store, the server works on an event window which is defined in the 'pandora_server.conf' configuration file by a parameter named 'event_window'. The events which have been generated outside the specified time range are not going to be processed by the server. Within a rule, it doesn't make any sense to specify a time range wider than the one configured within the Server.

 


Template warning.png

In order for the event correlation alerts to work, it is necessary to activate the event correlation server with the parameter eventserver 1 in the Pandora FMS server configuration file.

 


1.4 Events from the Command Line

1.4.1 Generating Events from the Command Line

By using the WEB API, you may interact with Pandora FMS from remote sites, even if you don't have a Database connection, Pandora FMS or an agent installed. You only require a special tool which you can find under:

/usr/share/pandora_server/util/pandora_revent.pl

This tool utilizes a remote HTTP or HTTPS connection to create or validate events under Pandora FMS. Please execute it without parameters to see it's syntax.

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -u <credentials> -create_event <options> 

Where options:

	-u <credentials>			: API credentials separated by comma: <api_pass>,<user>,<pass>
	-name <event_name>			: Free text
	-group <id_group>			: Group ID (use 0 for 'all') 
	-agent					: Agent ID
	
Optional parameters:
	
	[-status <status>]			: 0 New, 1 Validated, 2 In process
	[-user <id_user>]			: User comment (use in combination with -comment option)
	[-type <event_type>]			: unknown, alert_fired, alert_recovered, alert_ceased
							  alert_manual_validation, system, error, new_agent
							  configuration_change, going_unknown, going_down_critical,
							  going_down_warning, going_up_normal
	[-severity <severity>] 		: 0 Maintance,
						  1 Informative,
						  2 Normal,
						  3 Warning,
						  4 Crit,
						  5 Minor,
						  6 Major
	[-am <id_agent_module>]		: ID Agent Module linked to event
	[-alert <id_alert_am>]			: ID Alert Module linked to event
	[-c_instructions <critical_instructions>]
	[-w_instructions <warning_instructions>]
	[-u_instructions <unknown_instructions>]
	[-user_comment <comment>]
	[-owner_user <owner event>]		: Use the login name, not the descriptive
	[-source <source>]			: (By default 'Pandora')
	[-tag <tags>]				: Tag (must exist in the system to be imported)
	[-custom_data <custom_data>]		: Custom data should be a base 64 encoded JSON document (>=6.0)
	[-server_id <server_id>]		: The pandora node server_id (>=6.0)
        [-id_extra <id extra>]      : Id extra
        [-agent_name <Agent name>]  : Agent name, do not confuse with agent alias.
	[-force_create_agent<0 o 1>]: Force the creation of agent through an event this will create when it is 1.
        
Example of event generation:

	./pandora_revent.pl -p http://localhost/pandora_console/include/api.php -u 1234,admin,pandora 
		-create_event -name "SampleEvent" -group 2 -agent 189 -status 0 -user "admin" -type "system" 
		-severity 3 -am 0 -alert 9 -c_instructions "Critical instructions" -w_instructions "Warning instructions" 


Options to validate event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -u <credentials> -validate_event <options> -id <id_event>

Sample of event validation: 

	./pandora_revent.pl -p http://localhost/pandora/include/api.php -u pot12,admin,pandora -validate_event -id 234

You're required to enable the API access and configure it first. To do so, please follow the below mentioned steps:

  • Please enable the API access for the IP (please use '*' for all IPs).
  • Please set an API password
  • Please use a regular user and password or define a specific API user for conducting the operation only.

In order to render the 'unknown', 'critical' or 'warning' instruction fields appear within the event details, the event type is required to consist of the types named 'going_unknown', 'going_down_critical' or 'going_down_warning'.

Examples:

/pandora_revent.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -severity 4
-user "davidv" -owner_user "admin" -source "Commandline" -comment "Prueba de comentario"

1.4.2 Only for generating events from Command Line: 'pandora_revent_create' Command

It comes with the same functionality as the 'pandora_revent' script with the exception of being able to validate events.

/usr/share/pandora_server/util/pandora_revent_create.pl

This tool utilizes a remote HTTP or HTTPS connection to create or validate events under Pandora FMS. Please execute it without parameters to learn it's syntax.

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent_create.pl -p <path_to_consoleAPI> -u <credentials> -create_event <options> 

Where options:

	-u <credentials>			: API credentials separated by comma: <api_pass>,<user>,<pass>
	-name <event_name>			: Free text
	-group <id_group>			: Group ID (use 0 for 'all') 
	-agent					: Agent ID
	
Optional parameters:
	
	[-status <status>]			: 0 New, 1 Validated, 2 In process
	[-user <id_user>]			: User comment (use in combination with -comment option)
	[-type <event_type>]			: unknown, alert_fired, alert_recovered, alert_ceased
							  alert_manual_validation, system, error, new_agent
							  configuration_change, going_unknown, going_down_critical,
							  going_down_warning, going_up_normal
	[-severity <severity>] 		: 0 Maintance,
						  1 Informative,
						  2 Normal,
						  3 Warning,
						  4 Crit,
						  5 Minor,
						  6 Major
	[-am <id_agent_module>]		: ID Agent Module linked to event
	[-alert <id_alert_am>]			: ID Alert Module linked to event
	[-c_instructions <critical_instructions>]
	[-w_instructions <warning_instructions>]
	[-u_instructions <unknown_instructions>]
	[-user_comment <comment>]
	[-owner_user <owner event>]		: Use the login name, not the descriptive
	[-source <source>]			: (By default 'Pandora')
	[-tag <tags>]				: Tag (must exist in the system to be imported)
	[-custom_data <custom_data>]		: Custom data should be a base 64 encoded JSON document (>=6.0)
	[-server_id <server_id>]		: The pandora node server_id (>=6.0)

Example of event generation:

	./pandora_revent_create.pl -p http://localhost/pandora_console/include/api.php -u 1234,admin,pandora 
		-create_event -name "SampleEvent" -group 2 -agent 189 -status 0 -user "admin" -type "system" 
		-severity 3 -am 0 -alert 9 -c_instructions "Critical instructions" -w_instructions "Warning instructions" 

You're required to enable the API access and configure it first. Please follow the below mentioned steps to do so.

  • Please enable the API access for the IP (please use '*' for all IPs)
  • Please set an API password
  • Please use a regular user and password or define a specific API user only for conducting the operations only.

In order to render the 'unknown', 'critical' or 'warning' instruction fields to appear within the event details, the event type is required to be one of the types named 'going_unknown', 'going_down_critical' or 'going_down_warning'.

Examples:

/pandora_revent_create.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -severity 4
-user "davidv" -owner_user "admin" -source "Commandline" -comment "Prueba de comentario"

1.4.3 Custom Fields within Events

Events with custom fields may be generated by the Pandora FMS CLI, e.g. an event generated by the following command:

perl pandora_manage.pl /etc/pandora/pandora_server.conf --create_event 'Custom event' system Firewalls 'localhost' 'module' 0 4  'admin'     '{"Location": "Office", "Priority": 42}'

Would look like the one shown below.

Event custom data.png

Go back to Pandora FMS Documentation Index