Pandora: Documentation en: WUX Monitoring

From Pandora FMS Wiki
Jump to: navigation, search

1 WEB User experience transactional monitoring (WUX)

1.1 Introduction

Pandora WUX is a Pandora FMS internal component, which allows users to automate web browser interaction.

In order to work, WUXServer must be connected to a Pandora Web Robot Daemon (PWRD) server.

PWRD is a service that executes step by step pre-recorded actions by the user in a web portal.

To achieve this, PWRD uses Selenium web automation technology.

It is therefore imperative that the user shows Pandora WUXServer the actions that must be performed to navigate through the web portal.

With this data, Pandora WUXServer will generate a report with the results of the performances, time spent, and screenshots with the possible errors found.

Pandora WUXServer is able to split the browsing sessions into phases, which simplifies the view and helps us to debug possible bottlenecks.



Wux simple.png


1.2 Configuration

In order to enable WUXServer on our Pandora FMS server,we will need to configure the Pandora Web Robot service (PWRD)

1.2.1 Pandora Web Robot Daemon (PWRD)

Pandora Web Robot Daemon is a service, which provides tools to automate web browser sessions, using a file containing a list of necessary actions in order to navigate the user's web portal.

It comes integrated with the Pandora server and we can find it in the folder /usr/share/pandora_server/util/pwrd when installing the Enterprise server (Linux) or in the module library (Windows)

It contains:

  • Firefox 46 Binary for Windows.
  • Pre-built profile to perform web browser sessions.
  • Server for session automation (Selenium server).


1.2.2 Installation on Linux

The Selenium component of the PWRD daemon requires java in order to work, we will need to install it:

 yum install java 

In order for PWRD to be able to launch Firefox on our Linux server, it will be necessary to install xorg-x11-server-Xvfb and gtk2 to be able to virtualize graphical environments.

yum install xorg-x11-server-Xvfb gtk2

If its not available in your repos, you can find the RPMs files in following links: ftp://rpmfind.net/linux/centos/6.6/os/x86_64/Packages/xorg-x11-server-Xvfb-1.15.0-22.el6.centos.x86_64.rpm

ftp://rpmfind.net/linux/centos/7.4.1708/os/x86_64/Packages/gtk2-2.24.31-1.el7.x86_64.rpm

To do the RPMs installation manually:

yum install xorg-x11-server-Xvfb-1.15.0-22.el6.centos.x86_64.rpm
yum install gtk2-2.24.31-1.el7.x86_64.rpm

Then, proceed to install PWRD daemon, which can be found in the folder /usr/share/pandora_server/util/pwrd/ when installing the Enterprise server.

sudo /bin/bash /usr/share/pandora_server/util/pwrd/install_pwrd.sh --install

Once installed, we must start it.

PWRD provides several working modes:

  • Standalone: This is the standard mode; it will launch a single PWRD instance, each will be assigned to a Pandora FMS server.
  • HUB: Concentrator mode. In this mode, PWRD service will not evaluate browser sessions directly; instead, "nodes" will have to be registered to execute tasks. This is the cluster mode from the PWRD service. Each HUB will be assigned to a Pandora FMS server.


1.2.2.1 PWRD in "Standalone" mode

PWRD in standalone mode, will start the daemon and will prepare it to execute the actions indicated by the user through WUXServer.


Wux pwrd standalone.png


# Start 
/etc/init.d/pwrd start
#Get status
/etc/init.d/pwrd status
# Stop it
/etc/init.d/pwrd stop


1.2.2.2 PWRD in "HUB" mode

Concentrator mode or HUB will start the daemon as a load balancer. In this working mode, the system will balance the load between all nodes registered in the system, assigning the execution of the navigation sessions, depending on their workload.

You can always check the status of the HUB by accessing the HUB administration console:

http://hub:4444/grid/console


Wux pwrd hub.png


# Start
/etc/init.d/pwrd start-hub
# Get status
/etc/init.d/pwrd status-hub
# Stop it
/etc/init.d/pwrd stop-hub


1.2.2.2.1 Add PWRD nodes to our HUB

To add a new PWRD node, you will need:

  • A hub (PWRD in HUB mode)
  • PWRD files, on the same or different machine
  • TCP/4444 connectivity from the computer that hosts the node to the computer that hosts the HUB.

In this work mode, the service will process all queued requests from the hub (HUB), returning the results of the executions to it. It will be the HUB who will speak exclusively with the WUXServer, being transparent to the one who is executing the actions of the user.


 # Start and register in the HUB, replace "hub" with the server IP PWRD HUB
 /etc/init.d/pwrd start-node http://hub:4444/grid/register
Check status
/etc/init.d/pwrd status-node
Stop it
/etc/init.d/pwrd stop-node

1.2.2.3 Pandora Server Configuration

Regardless of the chosen mode, once started, we can start assigning executions of our browsing sessions, adding the configuration parameters of WUXServer to the configuration file of our Pandora server.

Assuming that we have deployed our PWRD server on the same server that runs our Pandora FMS server, we would add the following configuration:

# add to /etc/pandora/pandora_server.conf
wuxserver 1
wux_host 127.0.0.1
wux_port 4444

1.2.3 Installation on Windows

Steps to prepare a working environment for Pandora WUX on Windows.

Nota: We strongly recommend Linux to host the PWRD service.

You can get the files needed for your deployment of the module library in https://library.pandorafms.com/index.php?sec=Library&sec2=repository&lng=en&action=view_PUI&id_PUI=816

1.2.3.1 PWRD deployment on Windows systems

Prerequisites:


We will create the following file distribution:

  • File Decompression PWR_Server.zip a C:\PWR\
  • Export Firefox profile to C:\PWR\profile

Using a profile for the checks is not essential, however it is recommended especially when using proxy or when using autocomplete passwords. To create a profile of Firefox follow the steps in the following screenshots:


Ux1.JPG


Ux2.JPG


Choose the destination directory:


Ux3.JPG


We must start Firefox with the new profile to customize options such as proxy, shown popups, etc:


Ux4.JPG


Then we install the service by executing the BAT file service_installer.bat provided and we will start the service:

net start PWRSRV

From this moment you have the Selenium server running on your machine. However, we can run it manually (by previously stopping the service) if we want to carry out debugging tasks, using the following command:

java -jar selenium-server-standalone-2.53.1.jar -firefoxProfileTemplate C:\selenium\firefox_profile -port 4444 -v

1.3 To record a PWR session

Before monitoring a user experience we must record the actions we want the system to perform, in order to browse through our web portal

To do this we have available the "PWR recorder" utility in the module library.

Run the downloaded file to start the recording environment. Once the environment is started, you can access the recorder by clicking on the following button:


Iniciar grabador.png



You should see a window like the one shown in the image:



Ux10.JPG



From this moment we will be able to browse the website that we want to monitor and the different actions of each step that we advance will appear in the recorder.

To stop recording we will use the following button, located at the top right of the recorder:



Ux11.JPG



After completing the actions, we can perform checks on the page, for example verify the existence of a certain text to ensure that the page loaded is correct. To do this we will right click on a section of text in the browser window while we continue to record, and select the option verifyText:


Ux13.JPG


A new step will appear in the recorder indicating the said text checking action:


Ux14.JPG



We can reproduce the entire sequence using the button Play Entire test suite and check that successfully completes:


Ux15.JPG



After verifying the validity of the sequence of navigation, save it (File -> Save Test Case).This will be the file that we will load when creating a new web analysis module in our Pandora console.


1.3.1 Record a transactional session for Pandora WUX

Pandora WUX allows you to split browsing monitoring of a website into multiple modules, representing each of the steps performed.

To insert a new control point and generate the phase modules (up until that point) right-click on the point where you want to identify the start phase.


Pwrd recorder transaction.PNG


We will write the next text as a comment:

phase_start:name_of_phase

The Phase will encompass the time and result of all commands that exist up until the following comment:

phase_end:name_of_phase

All commands that are run between a phase_start y phase_end label will be covered in this phase.


Pwrd recorder transaction2.PNG


1.4 Creating a Web Analytics module in Pandora FMS Console

Once we have our browsing session recorded, it's time to deploy it as a module in Pandora FMS console.

To do so , we will access to any agent associated with a server with WUX feature enabled and we will create a module "Web analysis":

Wux module.PNG



After selecting "create" we must fill in all fields showed in the form:

Wux new module form.PNG
  • Module Name: Module WUX (Father), all sub-modules with monitoring results will be dependent on this new module.
  • Target Web site: For monitoring performance statistics of website.
  • Run performance test: Indicates that we want to monitor not only the browsing experience, but also the performance statistics of access to the target web site.
  • Historical data: Saving or not saving the historical data modules that monitor that user experience.
  • textarea section: in this text box we must copy (or upload) content files of the browsing session we have previously recorded.


1.4.1 Custom Macros

The operation of custom macros involves the substitution of certain text strings that are present in our file of browsing session with certain customizable values.

In the case of web analytics modules, this feature has been improved, allowing dynamic macros which allow these values to be translated into dates and times variables.

Why this feature then?

Let’s suppose that we need to monitor, through a browsing session, the proper functioning of a web application that focus on booking meeting rooms.

If we establish a date and time fixed, the system would probably cancel the reservation because at some point it might stay in the past , for example could be trying to book a room for last week.

We may also find a maximum of time to carry out that reservation, and that the system requires us to book the room within a certain period of time, for example, not exceeding the current month.

To avoid editing the macro every few days, and worrying about the configuration section, you can use the dynamic macros, telling the system to always book the room for the next day when doing the test.


To use this feature, the values must have a specific format, making possible the following substitutions:

@DATE_FORMAT (current date / time with user-defined format)
@DATE_FORMAT_nh (hours)
@DATE_FORMAT_nm (minutes)
@DATE_FORMAT_nd (days)
@DATE_FORMAT_ns (seconds)
@DATE_FORMAT_nM (month)
@DATE_FORMAT_nY (years)


"n" may be an unsigned number (positive) or negative.

And FORMAT follows the standard strftime de perl


Examples:

@DATE_%Y-%m-%d %H:%M:%S
@DATE_%H:%M:%S_300s
@DATE_%H:%M:%S_-1h



1.5 Data viewing

The information generated by WUX is displayed in modules as follows. Enabling view hierarchically in the list of modules, will help us to display the information much more clearly:


WUX modules.PNG



Within this hierarchy we will find the following modules:

  • module_Global_Status: Indicates the global state of complete browsing.
  • module_Global_Time: Indicates the global time lapsed of complete browsing.
  • module_Global_Screenshot: contains an image with the result of browsing error, it will be generated only in case of error
  • module_Phase X: Name of phase_Status: Indicates the status of the browsing in Phase X
  • module_Phase X: Name of phase_Time: Indicates time lapsed in Phase X

Example: Error capture view.


Wux screenshot.png


Template warning.png

Attention: If you upgraded from versions prior to Pandora FMS 7.0NG 712 you must make a small change.

 


In order to store the screenshots generated by WUXServer, you must execute the following operations on your database schema:

alter table tagente_estado modify column datos mediumtext;
alter table tagente_datos_string modify column datos mediumtext;

If you don't execute this actions, your screenshots may be incomplete or cropped

Accessing the WUX section Agent , we can view additional details of the transaction and its phases:


WUX.PNG



In the screenshot above you can see the difference in time of the two phases that our record is divided into as well as access statistics to the front portal.


Website statistics are summarized in the following concepts:

  • Stats_TT: Total time to obtain the website.
  • Stats_TDNS: Total time to resolve the IP address of the target.
  • Stats_TTCP: Time spent on connecting via TCP..
  • Stats_TSSL: Time spent on establishing SSL communication.
  • Stats_TST : Time lapsed until data transfer started.
  • Stats_TTC : Time spent transferring data, It will gather all the transfer time of the resources.
  • Stats_TTR : Time used to transfer the content of the page.
  • Stats_TTR_main: Time lapsed to transfer the HTML code.
  • Stats_TTR_image: Time used to transfer resources of type of image (png|jpg|jpeg|bmp|tiff|gif|webp|svg).
  • Stats_TTR_css: Time spent on transferring style sheets.
  • Stats_TTR_js: Time spent on transferring the JavaScript files.

1.6 Assign alerts to a web analysis module

Alerts associated with Web analysis modules follow the same dynamics of operating as the entire system of alerts in Pandora FMS.

For compatibility, it is recommended to assign templates of alerts on self-generated sub-elements by the analytics web module, such as:

  • The state of the global browisng
  • Alerts on thresholds times
  • Emails warning template set as "always" for screenshots modules of results


1.7 New widgets for Dashboards

To be able to display user browsing information, two new widget types have been added:

Widget for status of browsing phases

Widget wux fases.PNG



Widget for website statistics

Widget wux stats.PNG