Веб сценарии zabbix

Would you like to learn how to monitor web pages with free and open-source monitoring software called Zabbix? Well then, you are in the right place, because in this tutorial I will show you how to setup web monitoring on Zabbix from scratch using various examples!

Zabbix is not just for website monitoring, it can monitor virtually anything like Linux and Windows servers, VMware platform, Routers, and Switches, Databases, Docker containers, and more. Learn more about Zabbix on the official website.

Web monitoring in Zabbix is a pretty straightforward process. But what if you want to monitor hundreds or more websites in an automated manner? Unfortunately, this cannot be done using the out-of-the-box Zabbix functionality. There is no low-level discovery (LLD) for website monitoring.

However, Zabbix offers tools that can, with a little programming, automatically detect and monitor websites. Don’t worry, you don’t need to write any code, because I will share a template and script that I developed for dynamic website monitoring. With that template, it is enough to give a list of URLs (sitemap link, a path to a text file, macro with URLs separated by a semicolon ), and Zabbix will automatically create web checks for each URL.

But first things first, let me show you how to create a web scenario in Zabbix manually and how to configure alerts that will trigger when your website becomes unavailable.

Step 1: Install Zabbix monitoring system

Don’t have Zabbix installed? Don’t worry, I got you covered. You can install Zabbix in 10 minutes. Depending on your operating system (OS) preferences you can follow CentOS/RHEL, Ubuntu, Debian, Raspberry Pi OS (Rasbian) tutorial.

Zabbix installed and ready? Then, continue with the tutorial.

Step 2: Create a simple Zabbix Web scenario using a template

Web scenario is a key mechanism that enables web monitoring in Zabbix. A web scenario consists of one or more steps (HTTP requests) that are periodically executed in a predefined order.

You can create web scripts directly on the host, but this is not the best practice. It is better to create web scenarios in the template and then link them to the host. Let me show you how it’s done!

a) Create a Zabbix template for website monitoring

Navigate to the “Templates” under the “Configuration” section and click on the “Create template” button. Under the “Template” tab you need to define the “Template name” and the host group – “Groups“.

Next, switch to the “Macros” tab and create the macro {$WEB.URL}, but leave the value field blank for now. You will define a URL in that field later when you link the template with a host.

NOTE: You will later set the URL of the website you want to monitor in the {$WEB.URL} macro. And wherever that macro is used in the template, it will be translated into that URL. That way, you can use one template on multiple hosts and monitor different websites without changing the template.

Nice! Click the “Add” button to create the template and move to the next step.

b) Create a web scenario in the template

Now, go to the newly created template and click the “Create web scenario” button under the”Web scenarios” tab. Set name to “Website Health Check: {$WEB.URL}” and select the browser agent of your choice under the “Agent” dropdown.

After that switch to the “Steps” tab and click on the little “Add” button to create your first step. A new window will appear in which you need to set the “Name” to “Website Availability“, “URL” to “{$WEB.URL}“, enable “Follow redirects“, and “Required status codes” to “200“. Basically, we are telling Zabbix that it must get code 200 if the page is working properly.

Now click that big “Add” button to create a step. And after the window closes, click the second “Add” button to create the web scenario.

And you’re done! You have created your first web scenario! Nice work!

Note that you can enter additional required status codes (comma separated) that will indicate that the page is working properly. In addition to the status code 200, I like to add 401, 403, 410, and 503 codes. Learn more about status codes on the IANA or Wikipedia.

b) Create a trigger (alarm) for the web scenario

If you want to receive notifications when your site is unavailable or slowing down, then you need to create Zabbix triggers. To do this, you need to update our Website Monitoring template by adding a new trigger.

First, we will create a trigger that will check website availability every minute. Navigate to the “Configuration” → “Templates” → search and click the “Website monitoring” template → select the “Triggers” tab → click the “Create trigger” button.

Set trigger name to:

Website not available: "{$WEB.URL}"

Severity can be whatever you prefer. Add trigger expression using that little “Add” or manually as a text using this expression:

New syntax expression (Zabbix 5.4 and later)
last(/Website monitoring/web.test.fail[Website Health Check: {$WEB.URL}])<>0

Old syntax expression (Zabbix 5.2 and older)
{Website monitoring:web.test.fail[Website Health Check: {$WEB.URL}].last()}<>0

Note that you can replace the “last()” function with the “max(5m)” function if you would like to receive a notification only when the website is down for more than 5 minutes.  Alarms can be spammy sometimes when using the “last” function so consider using “avg“, “max” or “min” functions (hysteresis).

Click that big “Add” button when you’re done configuring the trigger and you are done!

The previous trigger will alert you if our website is unavailable, but what if it is available but very very slow? We need another trigger that will alert us to such a problem. Let’s create that trigger! Just repeat the same procedure: navigate to “Configuration”→ “Templates”→ Search and click the “Website monitoring” template→ Select “Triggers” tab→ Click the “Create trigger” button.

Set trigger name to:

Website is SLOW ({ITEM.LASTVALUE}): {$WEB.URL}

Severity can be whatever you prefer. Add trigger expression using that little “Add” or manually as a text using this expression:

New syntax expression (Zabbix 5.4 and later)
avg(/Website monitoring/web.test.time[Website Health Check: {$WEB.URL},Webpage Availability,resp],10m)>8

Old syntax expression (Zabbix 5.2 and older)
{Website monitoring:web.test.time[Website Health Check: {$WEB.URL},Webpage Availability,resp].avg(10m)}>8

Click that big “Add” button when you’re done configuring the trigger and thats it!

NOTE: Consider creating a new macro {$URL.SPEED} on the template and replacing 8 seconds with {$URL.SPEED} in trigger expression. This will be useful if you are planning to change that threshold often or if you have multiple sites with drastically different response times.

WELL DONE!
You created a simple web scenario in Zabbix using a template! Now you need to enable website monitoring by linking that template with the host and defining a URL within the macro!

Step 3: Enable Website monitoring on the host in Zabbix

As you may have noticed we are still not monitoring anything! Because we didn’t link the template to the host! Let’s do that!

Navigate to the “Host” menu under the “Configuration” section and then click the “Create host” option to create a host in Zabbix. Define “Host name”, set something that will describe what you are monitoring. Create your group using the “Groups” field or use an existing one.

Don’t click that “Add” button yet! We need to link a template that we created in the previous step and configure {$WEB.URL} macro. Click the “Templates” tab and link the “Website monitoring” template using the “Link new templates” field.

Switch to the “Macros” tab , click the “Change” for the macro “{$WEB.URL}” and set the URL of the website that you need to monitor in the value field of that macro.

Press that add button and you’re done!

Step 4: Learn how to view web monitoring graphs

There are several ways to visualize data on Zabbix, but to simplify things, let’s see the default visualization for web scenarios. Navigate to “Monitoring”→ “Hosts”→ Search your host→ Click on the “Web” button→ and then click on your web scenario to view graphs.

That’s it! Nice work!

In the previous step, we have configured a simple web scenario that checks the unavailability or slowness of the website. And most of the time that’s all you’ll ever need. However, what if we want to test user authentication on the website? Well, we can configure that on Zabbix, and even more!

Let’s create a multi-step web scenario that will sign in to the website using credentials, click on the profile page, log out, and notify us if any of these steps fail.

We will use this WordPress blog as a example.

a ) Search for post fields in the login form

Before we continue, it is crucial to find out which post fields we need to use to successfully log in to the website using Zabbix. If you are also monitoring a WordPress website then you can skip this step because I have dug up all the post fileds for you.

However, if you want to know to extract post fields from the login form, then check out the instructions in the image below.

Basically, you need to use Chrome or Firefox inspect tool to find out user, password, and submit post fields:

b ) Create a Zabbix template for website monitoring with authorization

Creating web scenarios directly on the host is a bad practice so we will use a template.

Navigate to the “Templates” under the “Configuration” section and click on the “Create template” button. Under the “Template” tab you need to define the “Template name” and the host group under the “Groups” field. I will set the name to “Website monitoring with authorization” and “Groups” to “Devices/Web“.

Next, switch to the “Macros” tab and create macros: {$WEB.URL}, {$WEB.PASW}, and {$WEB.USER}, but leave the value fields blank for now. You will define a URL, username, and password values later when you link this template with a host.

Click the “Create template” button and proceed to the next step.

c ) Create multi-step web scenario on the Zabbix template

Now, go to the newly created template and click the “Create web scenario” button under the”Web scenarios” tab.Set name to “AdvanceWebsite Check (login, click, logout): {$WEB.URL}” and select the browser agent of your choice under the “Agent” dropdown.

Since we are creating multiple web scenarios then it would be wise to set variables for username, password, and submit values. In this way, we can use these variables in as many steps as we want. In our case, we plan to define username and password later using macros so we will set macros instead of real credentials:

Web scenario step 1: Create “Login” web scenario

After that switch to the “Steps” tab and click on the little “Add” button to create your first step. A new window will appear in which you need to set the “Name” to “Login“, “URL” to “{$WEB.URL}/web-login.php” (this URL works only with WordPress).

And now, we can associate the post field names that we got in the step “Search for mail field when signing in” with our newly created variables:

Enable “Follow redirects” and set “Required string” to “>Log Out<” because if the login was successful “Log Out” button will appear on the page. You can find the string “>Log Out<” when viewing a page in HTML format using the Google / Firefox inspect tool.

Click the “Add” button at the bottom of the step window and go to the next step (do not click the “Add” button for the entire web scenario – we are not done yet!).

Web scenario step 2: Create “Click on profile” web scenario

Now we need to create a step that will open another webpage after the login to confirm that the website is alive and responsive. Set the “Name” to “Click on profile“, “URL” to “{$WEB.URL}/wp-admin/profile.php” (this URL will open user profile webpage on the WordPress).

Enable “Follow redirects“, and paste “Update Profile” into the “Required string” because if the webpage is successfully loaded the “Update Profile” button will appear on the page.

Click the “Add” button at the bottom of the step window and go to the next step (do not click the “Add” button for the entire web scenario – we have one more step left!).

Web scenario step 3: Create “Log out” web scenario

This is the last step, let’s create a scenario that will do a logout. Set the “Name” to “Logout“, “URL” to “{$WEB.URL}/wp-login.php?logedout=true” (this URL will logout a user from the WordPress website). Alternatively, you can use “Post fields”, but for simplicity, I will set the post fields directly in the URL by adding “?logedout=true”.

Enable “Follow redirects“, and paste “Username or Email Address” into the “Required string” because if the user is successfully logged out the “Username or Email Address” field will appear on the page.

You can now click the “Add” button for the entire web scenario if all three web scenarios are displayed exactly as in the image below.

d ) Create a trigger for the advance web scenario

If you want to receive notifications when your site is unavailable or if user authorization fails, then you need to create Zabbix triggers. Update the “Website Monitoring with authorization” template by adding a new trigger. Navigate to “Configuration”→ “Templates”→ Search and click the “Website Monitoring with authorization” template→ Select “Triggers” tab→ Click the “Create trigger” button.

Set trigger name to:

Website not available or Login failed: {$WEB.URL}

Severity can be whatever you prefer. Add trigger expression using that little “Add” or manually as a text using this expression:

New syntax expression (Zabbix 5.4 and later)
last(/Website monitoring with authorization/web.test.fail["Advance Website Check (login, click, logout)"])<>0

Old syntax expression (Zabbix 5.2 and older)
{Website monitoring with authorization:web.test.fail["Advance Website Check (login, click, logout): {$WEB.URL}"].last()}<>0

Click that big “Add” button when you’re done configuring the trigger and thats it!

d ) Enable Website monitoring with authorization on the host in Zabbix

Use an existing host or create a new one just for the website monitoring like in the step “Step 3: Enable Website monitoring on the host in Zabbix” but set different template and macros.

Link the template “Website monitoring with authorization”, and under the “Macros” tab select the “Inherited and host macros” option and set values by clicking the “Change” on macros {$WEB.URL}, {$WEB.PASW}, and {$WEB.USER}.

Nice job! You have configured advanced website monitoring with authorization.

Step 6: Automate web scenario and trigger creation

You may have noticed that the process of creating web scenarios on Zabbix is slow and tedious. There is no automation, no low-level discovery (LLD)!

What if you want to monitor a couple of thousand pages in Zabbix? Unfortunately, to do that you would have to create it all manually unless you throw yourself into scripting. Luckily, there is no need for that, because I’ve already created a Zabbix script that automatically discovers and monitors web pages using a list of web page URL’s. The easiest way is to use a sitemap but you can also use a text file or manually enter a list of URLs within the macro.

But first thing first, let’s install some python modules that our script needs and tweak the Zabbix server configuration file.

a) Install py-zabbix python module

Install python 3, python-requests and python-setuptools:

# RHEL / CentOS / Oracle Linux / Rocky Linux 
yum install -y python3 python3-requests python3-setuptools

                                 OR 

# Ubuntu / Debian / Raspberry OS 
sudo apt install python3 python3-requests  python3-setuptools

Install python-zabbix module:

# Install py-zabbix using pip3
sudo pip3 install py-zabbix

                                 OR

# Install py-zabbix from source if you don't want to use pip3
curl -o  py-zabbix-1.1.7.tar.gz https://files.pythonhosted.org/packages/c8/b1/5ecb05b542a78dd6d619c825740fc11b12cb2afd06b9c679129a5c795554/py-zabbix-1.1.7.tar.gz
tar -zxvf py-zabbix-* && cd py-zabbix-*/
python3 setup.py install

b) Download webmonitoring script

Download my script and put it in your Zabbix ExternalScripts directory. Don’t know the path of that directory? Use the Zabbix configuration file to find out where the Zabbix ExternalScripts directory is located:

grep "ExternalScripts=" /etc/zabbix/zabbix_server.conf

The output of the command should be similar to this:

# ExternalScripts=/usr/lib/zabbix/externalscripts   # this is default

Now download my script to that directory:

curl -o /usr/lib/zabbix/externalscripts/webmonitoring.py https://bestmonitoringtools.com/dl/webmonitoring.py

And set executable permissions on the script:

chmod +x /usr/lib/zabbix/externalscripts/webmonitoring.py

c) Update Zabbix server configuration file

When monitoring a large number of websites Zabbix will generate an alarm “Zabbix HTTP poller processes more than 75% busy“. As a result, some polling data will be lost. To avoid this, you need to increase the number of HTTP pollers in the Zabbix server configuration file.

Open “zabbix_server.conf” file with command: “nano /etc/zabbix/zabbix_server.conf” and add (or change) these parameteres:

StartHTTPPollers=10

Also, the script will need more time to complete, therefore add (or change) the Timeout parameter and set it to 30 seconds:

Timeout=30

Save and exit file (ctrl+x, followed by y and enter). Learn more about the timeout parameter and what impact its change has on Zabbix in my old blog post.

In order to apply the new settings you need to restart the Zabbix server, so let’s do that:

systemctl restart zabbix-server

d) Download and import template

Now that we have the script in place, we need to set up a template that will use our script. Download the template “Web monitoring with discovery” and import it to Zabbix via frontend.

To do this, navigate to the “Templates” under the “Configuration” section and click the “Import” button, then select the downloaded template file and import it.

e) Create a host and configure macros (API credential, source of URLs etc.)

Almost done! Add a new host to Zabbix that will use our template and script to create all the triggers and graphs for each monitored website.

Create a host, link the “Web monitoring with discovery” template and use a similar setting as in the image below.

Don’t click the “Add” button yet! Now comes the important part – configure macros according to the instructions in the macro description. See my example in the image below.

A few notes, if you want to monitor a few web pages, manually add URLs in the “{$WEBMON.URLS.LIST}” macro. However, if you monitor more than 3 URLs, then define the sitemap URL using “{$WEBMON.URLS.SITEMAPS}” macro or the full DIR path of the text file that contains a list of URLs using {$WEBMON.URLS.FILES} macro. Rember that you can combine any number of options or use just one!

Also don’t forget to set the correct user permissions if you are using the text file option, for example: “chown zabbix.zabbix /usr/lib/zabbix/externalscripts/sites.txt”.

f) Check that the monitoring is working properly

To verify that monitoring is working properly, go to “Latest Information” in the “Monitoring” section and select your host in the “Hosts” field.

Want to receive email alerts when your website is down? Learn how to setup Zabbix email notifications and escalations!