Pandora: Documentation en: Transactional Monitoring

From Pandora FMS Wiki
Jump to: navigation, search

1 Transaction monitoring

1.1 Introduction

Pandora FMS v. 7 now incorporates transaction monitoring.

The transactional server component implemented in this version allows you to execute dependent tasks following a user-defined design. This means that it is possible to coordinate different executions to test a target at a given time.

Let's take a case study, for example. It could consist of tracking an order that goes through its different phases, being able to keep track of the times in each of the steps, departments or stages:


Trans sq.png


A four-step graph will be defined which Pandora FMS will use network along with a series of control scripts to monitor the previously indicated process, visually showing the status in which each one of the parts that conform our business process is at all times.

Below we will look at how to fully monitor a transaction process.

1.2 Functioning

We will define a transaction as a set of steps that make up a more complex task. We will call these steps '"phases'".

The set of phases and their workflow (dependency relations) will define a transactional graph.

Pandora FMS, based on the transactional graph, will launch an order to execute control scripts in each of the previously defined phases. These control scripts will perform the monitoring tasks for each phase.

The most common points to check in the control scripts that can give us information on the status of our transaction are:

  • Entries in log files.
  • Presence of temporary files.
  • Direct database queries.
  • Existence of mail in inboxes.

The transactional system is distributed, being able to deploy as many transactional agents in our infrastructure as we need, and it will be the Pandora FMS server that will communicate with these agents, indicating the steps to follow and the moments in which they must run a control script.

1.3 Configuring a transaction

In order to configure and execute transaction monitoring, it will be necessary:

  • Previous analysis of the business process to be monitored:
    • Workflow.
    • Points involved.
    • Control scripts.
  • Deployment of transactional agents in the required equipment to control the information flow.
    • Development and deployment of control scripts in the required transactional agents.
    • Configuration of the transactional agents.
  • From the Pandora FMS console, create the transaction by entering the data in the forms.

1.3.1 Prior analysis

Let's analyze a regular case:

The transaction begins when an order is received on the web portal, where a transactional agent must be deployed, or at least on a machine capable of carrying out remote checks. During this first phase a critical script is executed; the transaction trigger.

In the next phase, a virtualization of the order processing launches in the EMI01 machine, using different ID codes for the order placed in the previous phase. There needs to be another transactional agent installed on this machine, or, if not, on a machine capable of executing remote checks. The control script checks that the process has been correctly completed for this phase by searching for registry file entries, events or temporary files.

The third phase of the transaction is to save the processed order on an Oracle database, on the ORAMON machine. These machines are usually highly protected and it's difficult to install additional software on them, so a transactional agent should be deployed on a remote machine with permission to launch data base queries.

The last step is to confirm whether there is mail in the Logistic department's inbox in the public mail server. Since we can't deploy an agent on this machine, we'll run the queries from another machine that is able to access it. We can use Pandora FMS classic plugins slightly modified to check for incoming emails.

1.3.2 Deployment and development

In the example there are four phases which require four scripts:

  1. Transaction trigger: makes a virtual order to the web portalin order to subsequently trace it.
  2. Control script: looks for a chain in the log file.
  3. Control script: queries the data base.
  4. Control script: confirms whether the corresponding email reached the Logistics inbox.

In the above example the transaction trigger leaves a file copy of an order in a specific point.

Control scripts share a basic structure as in the following example:


Script1.JPG


#!/bin/bash
########################
# Check Control Set 
######################## 
function check_flag() { 
   # Condiciones del script de control 
   if [ "a" == "a" ]; then 
      return 1; 
   fi 
   return 0; 
} 
max_tries=100 
wait=3 
try=0 
while [ $try -le $max_tries ]; do 
   if [ check_flag == 1 ]; then 
      echo 1 
      exit 0 
   fi 
   sleep 
   $wait 
   try=`expr $try + 1` 
done 
echo 0 
exit 1

Once the scripts are working correctly, proceed to deploy transactional agents on the necessary machines and devices.

tar xvzf pandorafms_transactional.tar.gz
cd pandora_transactional 
./pandora_transactional_installer --install

We will only have to modify the agent configuration file (by default /etc/pandora/pandora_transaccional.conf) to point to the Pandora FMS server that will direct the transaction process. And we'll start the agent's service:

/etc/init.d/transactional_daemon start

1.3.3 Creating a transaction

Using the Pandora FMS web console.

Go to Maps -> Transactional map.


Trans1.JPG


Trans2.JPG


Create the new transaction and fill in the required fields.


Trans3.JPG


  • Name.
  • Description.
  • Agent: Pandora FMS agent where the modules generated by the system will be saved (History).
  • Group: to control visibility and access.
  • Loop interval: waiting time before launching the transaction again .

1.3.4 Creating the phase tree

Once the transaction has been created, a tree chart can be shaped by accessing the corresponding form:


Trans4.JPG


We must enter the data for each phase:


Trans5.JPG


  • Index: sole ID for the phase of the current transaction.
  • Name.
  • Agent: where the corresponding control scripts will be executed.
  • Dependencies: prior phases that must be completed before the phase in question initiates; indexes are separated by commas.
  • Enables: phases that are enabled when a phase being edited is completed; the indexes of the phases are separated by commas.
  • Actions: save changes.

START is the default initial phase, used only to define which phases will be the first to be executed.

In the following screenshot, you can see how to insert the index of the first phase to execute (1) Order received.


Trans6.JPG


When saving the change we see that the phase with error status is marked automatically. This is because, in our definition, there is no stage with index 1. Click "Add" to create it:


Trans7.JPG


Once the changes are saved, create the phase by updating the previsualization of the transaction graph and correcting the previous warning:


Trans8.JPG


Note that in the phase 1 dependencies field '0' is shown to indicate that the phase will begin when the START phase has successfully finished.

Create all the phases following the same procedure:


Trans9.JPG


The final phase does not enable anything, it means the transaction is finished.

In the screenshot a yellow warning icon can be seen, indicating that an element is pending configuration (the control scripts).

Once the transaction has been created, a tree chart can be shaped by accessing the corresponding form:


Trans4.JPG


enter the data for each phase:


Trans5.JPG


  • Index: sole ID for the phase of the current transaction.
  • Name.
  • Agent: where the corresponding control scripts will be executed.
  • Dependencies: prior phases that must be completed before the phase in question initiates; indexes are separated by commas.
  • Enables: phases that are enabled when a phase being edited is completed; the indexes of the phases are separated by commas.
  • Actions: save changes.

START is the default initial phase, used only to define which phases will be the first to be executed.

In the following screenshot, you can see how to insert the index of the first phase to execute (1) Order received.


Trans6.JPG


When saving the change we see that the phase with error status is marked automatically. This is because, in our definition, there is no stage with index 1. Click "Add" to create it:


Trans7.JPG


Once the changes are saved, create the phase by updating the previsualization of the transaction graph and correcting the previous warning:


Trans8.JPG


Note that in the phase 1 dependencies field '0' is shown to indicate that the phase will begin when the START phase has successfully finished.

Create all the phases following the same procedure:


Trans9.JPG


The last phase will not enable anything, it will mean that the transaction is complete.

In the screenshot we also observe a yellow warning, indicating that there is something pending to be configured, (the control scripts).

1.3.5 Control scripts configuration

We will have to configure the control scripts associated with each phase, these must have been previously defined to perform the desired checks, and maintain a specific basic structure. The standard output of the script will determine the central value shown in the phase, while the status (correct or incorrect) will be determined by the result of the execution of the script itself, NOT to be confused with the result of the execution (also called errorlevel, or execution code) with the standard output of the script (the value returned by the screen when executing it).

The run result or error level can be checked by running "echo $?" on Linux systems and echo %ERRORLEVEL% on Windows systems.

With this in mind, we access the form using the edit icon:


Trans10.JPG


In the configuration form of the control scripts we can adjust the command to be executed, number of retries and the maximum execution time allowed for the indicated script:


Trans11.JPG


  • Launch script: location or complete path of the control script, call, command, etc. We can use the _name_ macro to indicate the name of the current transaction as an argument to our script.
  • Reattempts: maximum number of execution retries until a positive response is obtained.
  • Maximum waiting time: maximum value, in seconds, that the control script will remain running (feature not available for the Windows transactional agent).

In our example we will use a custom script (echo1.sh) to simulate the actions of the transaction trigger and control scripts for each phase:


Trans12.JPG


Once we have configured our environment correctly, we can initiate the transaction.


Trans13.JPG


Template warning.png

Any changes made to the configuration of a transaction will not be effective until the transaction is restarted.

 


1.3.6 Transaction control

1.3.6.1 Initiate a transaction

Go to the transactions list and click on the "initiate" icon (the black doughnut):


Trans14.JPG


Once the transaction has been initiated the "stop transaction" icon will appear:


Trans15.JPG


The status of the transaction can be seen on the display by clicking the transaction's name:

Trans16.JPG

1.3.6.2 Stopping a transaction

To interrupt a transaction you only have to click the corresponding button, and, after a few seconds, click the "launch" icon to restart it.


Trans17.JPG


1.3.6.3 Viewing a transaction

Go to the view by clicking the name of the transaction on the transaction list.

Viewing an execution in progress (initial phase of transaction):


Trans18.JPG


Viewing an execution in progress (intermediate phase):


Trans19.JPG


Viewing a completed transaction:


Trans20.JPG


1.4 Configuration

1.4.1 Transaction server

The transaction server's configuration parameters are as follows:

transactionalserver 1

transactional_threads 1

transactional_threshold 1

# Work directory
# By default in icomingdir/trans
transactional_pool trans
  • transactionalserver: initiates the transaction server component upon initiating the pandora_server service.
  • transactional_threads: added to the field by agreement, only one internal thread is necessary to manage active transactions from the server. The agent has a subsystem of dynamic threads to manage configured transaction executions.
  • transactional_threshold: wait time between status updates. This field defines the time, in seconds, that the system will wait between changes in status of the elements which make up a transaction (recommended threshold: 4 seconds).
  • transactional_pool: directory where “lock” files are saved. The system uses these files to communicate among the different logical components (default location: /var/spool/pandora/data_in/trans).

For the system to be 100% operational, both the data server (dataserver) and the transaction server (transactionalserver) must be active. Go to tactical server view to check:

Trans21.JPG

1.4.2 transactional agent

The transactional agents have a configuration file in their work directory (by default at: /etc/pandora/pandora_transactional.conf) with the following content:

############################################################
#  Base config file for Pandora FMS transactional agent 
#  Version 2.0                                      
#  Copyright (c) 2003-2016 Artica Soluciones Tecnologicas  
#  http://www.pandorafms.com                          
#############################################################

server_ip=localhost
server_port=41121

# Temporal directory
temporal=/tmp
#temporal=C:\Program Files\pandora_transactional\tmp

# Log directory
log=/var/log/pandora/pandora_transactional.log

# Tentacle binary
tentacle_bin=/usr/bin/tentacle_client

###############################
# Set the name of the agent
###############################
#agent_name=transactional_agent
agent_interval=300

# Performance (in seconds) internal clock
pause=5

# Show all log messages (0 - show none, 1 - show script execution messages, 2 - show all)
verbose=2
  • server_ip: IP address or the name of Pandora's transactional server.
  • server_port: port where the tentacle server listens in.
  • temporal: auxiliary directory for temporary work files.
  • log: log file route.
  • tentacle_bin: tentacle client's binary route.
  • agent_name: optional, use a custom name to ID this agent. The name of the machine is the default ID.
  • pause: default time , in seconds, between phases. Adjusts to the configuration received by the server to maintain synchrony.
  • verbose: controls the quantity of information displayed in the log files. Any output in STDERR will overflow to this file by default:
    • 0: only show critical error messages.
    • 1: show control script executions.
    • 2: show all messages.

All transaction configurations will be published in the Pandora FMS server, and the agent is in charge of scanning these files and initiating the necessary data structures to initiate the transactions solicited.

In the face of any updates during an active transaction the agent will interrupt the transaction, and initiate a new process to instantiate a new one. All previous progress will be lost.

The transaction system executes the complete transaction, which is to say, that if there's an error executing a script the transaction will continue to execute, indicating the phase during which the error occurred. A transaction is considered failed when all phases are indicated as errors.

You can download the transactional agent component from our plugin library [1]