Install Prometheus Network Monitoring Server on CentOS 7 - CentLinux

Latest

Sunday, 3 November 2019

Install Prometheus Network Monitoring Server on CentOS 7

Install Prometheus Monitoring Server on CentOS 7

 

Prometheus is a free and open source network monitoring tool. In this article, you will learn how to install Prometheus network monitoring server on CentOS 7.

 

This article provides:

 

 

What is Prometheus? :

Prometheus is a free and open source software application used for event monitoring and alerting. It collects and records real-time metrics in a time series database and alert the users based on custom defined thresholds. Prometheus is written in Go programming language and distributed under Apache License 2.0.

Prometheus requires multiple exporters that must be installed on the target nodes. The exporter generates and publish the metrics on a relevant port on the target node. The Prometheus monitoring server will pull these metrics and store in a time series database. Following architecture diagram from Prometheus Documentation clarifies the concept and flow of the system.

01-prometheus-architecture-diagram

In this guide, we are installing Prometheus network monitoring server on CentOS 7 along with node_exporter.

If you want to know more about Prometheus software than you should read Prometheus: Up & Running by O'Reilly Media.

 

Prometheus Features:

Following are the core features of Prometheus.

  • Multi-dimensional data model
  • Write powerful queries with PromQL
  • Grafana integration for great visualization
  • Efficient storage (in memory and disk) of time series data
  • Precise alerts using Alertmanager

 

Environment Specification:

We have configured a CentOS 7 based Linux virtual machine for this tutorial.

  • CPU - 3.4 Ghz (2 cores)
  • Memory - 2 GB
  • Storage - 20 GB
  • Operating System - CentOS 7.7
  • Hostname - prometheus-01.example.com
  • IP Address - 192.168.116.213 /24

 

Prerequisite configurations for Prometheus on CentOS 7:

Connect with prometheus-01.example.com using ssh as root user.

As a best practice, we are creating a Linux user to own Prometheus software and processes.

[root@prometheus-01 ~]# useradd --no-create-home -s /bin/false prometheus

Create required directories for Prometheus software.

[root@prometheus-01 ~]# mkdir /etc/prometheus [root@prometheus-01 ~]# mkdir /var/lib/prometheus

Change owner and group of these directories to Prometheus.

[root@prometheus-01 ~]# chown prometheus:prometheus /etc/prometheus [root@prometheus-01 ~]# chown prometheus:prometheus /var/lib/prometheus

 

Installing Prometheus Network Monitoring Server on CentOS 7:

You can download Prometheus network monitoring software from their official website or from it’s GitHub repository.

Here, we are downloading latest version of Prometheus with the help of wget Linux command.

[root@prometheus-01 ~]# cd /tmp [root@prometheus-01 tmp]# wget https://github.com/prometheus/prometheus/releases/download/v2.13.1/prometheus-2.13.1.linux-amd64.tar.gz --2019-10-30 18:35:51-- https://github.com/prometheus/prometheus/releases/download/v2.13.1/prometheus-2.13.1.linux-amd64.tar.gz Resolving github.com (github.com)... 192.30.253.112 Connecting to github.com (github.com)|192.30.253.112|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/6838921/c0040780-f109-11e9-946c-40b97454e08d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191030%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191030T133550Z&X-Amz-Expires=300&X-Amz-Signature=81f194c6a9222eee87bbb56c67c266703be220ddcdc754b9efb49f525b2b1338&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dprometheus-2.13.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream [following] --2019-10-30 18:35:53-- https://github-production-release-asset-2e65be.s3.amazonaws.com/6838921/c0040780-f109-11e9-946c-40b97454e08d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191030%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191030T133550Z&X-Amz-Expires=300&X-Amz-Signature=81f194c6a9222eee87bbb56c67c266703be220ddcdc754b9efb49f525b2b1338&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dprometheus-2.13.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.24.44 Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.24.44|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 57079650 (54M) [application/octet-stream] Saving to: âprometheus-2.13.1.linux-amd64.tar.gzâ 100%[======================================>] 57,079,650 240KB/s in 4m 24s 2019-10-30 18:40:19 (211 KB/s) - âprometheus-2.13.1.linux-amd64.tar.gzâ saved [57079650/57079650]

Extract downloaded zip file by using tar Linux command.

[root@prometheus-01 tmp]# tar xvzf prometheus-2.13.1.linux-amd64.tar.gz prometheus-2.13.1.linux-amd64/ prometheus-2.13.1.linux-amd64/consoles/ prometheus-2.13.1.linux-amd64/consoles/node-overview.html prometheus-2.13.1.linux-amd64/consoles/node-cpu.html prometheus-2.13.1.linux-amd64/consoles/node.html prometheus-2.13.1.linux-amd64/consoles/prometheus.html prometheus-2.13.1.linux-amd64/consoles/index.html.example prometheus-2.13.1.linux-amd64/consoles/node-disk.html prometheus-2.13.1.linux-amd64/consoles/prometheus-overview.html prometheus-2.13.1.linux-amd64/LICENSE prometheus-2.13.1.linux-amd64/prometheus.yml prometheus-2.13.1.linux-amd64/tsdb prometheus-2.13.1.linux-amd64/NOTICE prometheus-2.13.1.linux-amd64/promtool prometheus-2.13.1.linux-amd64/prometheus prometheus-2.13.1.linux-amd64/console_libraries/ prometheus-2.13.1.linux-amd64/console_libraries/prom.lib prometheus-2.13.1.linux-amd64/console_libraries/menu.lib

Move Prometheus files to /var/lib/prometheus directory.

[root@prometheus-01 tmp]# mv prometheus-2.13.1.linux-amd64/* /var/lib/prometheus/

Adjust owner and group of /var/lib/prometheus directories.

[root@prometheus-01 tmp]# chown -R prometheus:prometheus /var/lib/prometheus

Prometheus zip file also contains a configuration file with default settings. We can use this file to configure our Prometheus server.

With the help of mv Linux command, move Prometheus configuration file to /etc/prometheus directory.

[root@prometheus-01 tmp]# cd [root@prometheus-01 ~]# mv /var/lib/prometheus/prometheus.yml /etc/prometheus/

Create a soft link of prometheus executable file, so we can execute the prometheus command from CLI.

[root@prometheus-01 ~]# ln -s /var/lib/prometheus/prometheus /usr/local/bin/prometheus

To setup autostart of Prometheus at the time of Linux server startup, we need to create a Systemd service for Prometheus network monitoring tool as follows.

[root@prometheus-01 ~]# vi /usr/lib/systemd/system/prometheus.service

And add following directives in that file.

[Unit] Description=Prometheus Wants=network-online.target After=network-online.target [Service] User=prometheus Group=prometheus Type=simple ExecStart=/usr/local/bin/prometheus \ --config.file /etc/prometheus/prometheus.yml \ --storage.tsdb.path /var/lib/prometheus/ \ --web.console.templates=/var/lib/prometheus/consoles \ --web.console.libraries=/var/lib/prometheus/console_libraries [Install] WantedBy=multi-user.target

Enable and start Prometheus network service.

[root@prometheus-01 ~]# systemctl enable --now prometheus.service Created symlink from /etc/systemd/system/multi-user.target.wants/prometheus.service to /usr/lib/systemd/system/prometheus.service.

Prometheus uses default service port 9090/tcp. We need to allow this port in Linux firewal, so the network traffic can reach at Prometheus service.

[root@prometheus-01 ~]# firewall-cmd --permanent --add-port=9090/tcp success [root@prometheus-01 ~]# firewall-cmd --reload success

Open URL http://prometheus-01.example.com using a client's browser.

02-prometheus-query-metrics

We are now at the Expression Browser web page. Here, we can search for the metrics, for which Prometheus is collecting data.

To see the graph of the data click on Graph.

03-prometheus-draw-graph

 

Installing Node_Exporter on CentOS 7:

Prometheus receives metrics from many sources like machines, databases, applications, etc.

For this purpose, Prometheus requires an Agent (termed as Exporter) to be installed on each node, for which we want to collect metrics.

There are different types of exporters available at Prometheus website. We can download and use one, according to our requirement. But the most common Exporter that we prefer to install on each node to collect machine metrics is the node_exporter.

In this section, we are installing node_exporter on the same CentOS 7 based Prometheus network monitoring server.

Download Node_Exporter from it’s official download page.

[root@prometheus-01 ~]# cd /tmp [root@prometheus-01 tmp]# wget https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz --2019-11-01 23:12:19-- https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz Resolving github.com (github.com)... 140.82.113.3 Connecting to github.com (github.com)|140.82.113.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/9524057/5dc5df80-86f1-11e9-924c-ef392e7300e3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191101%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191101T181218Z&X-Amz-Expires=300&X-Amz-Signature=f5eae2939a77b2939f4299312aa5228aaee04471e15bde050c55db2a928212b2&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dnode_exporter-0.18.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream [following] --2019-11-01 23:12:21-- https://github-production-release-asset-2e65be.s3.amazonaws.com/9524057/5dc5df80-86f1-11e9-924c-ef392e7300e3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191101%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191101T181218Z&X-Amz-Expires=300&X-Amz-Signature=f5eae2939a77b2939f4299312aa5228aaee04471e15bde050c55db2a928212b2&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dnode_exporter-0.18.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.100.91 Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.100.91|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8083296 (7.7M) [application/octet-stream] Saving to: ânode_exporter-0.18.1.linux-amd64.tar.gzâ 100%[======================================>] 8,083,296 188KB/s in 84s 2019-11-01 23:13:46 (94.3 KB/s) - ânode_exporter-0.18.1.linux-amd64.tar.gzâ saved [8083296/8083296]

Extract downloaded files as follows.

[root@prometheus-01 tmp]# tar xvf node_exporter-0.18.1.linux-amd64.tar.gz node_exporter-0.18.1.linux-amd64/ node_exporter-0.18.1.linux-amd64/node_exporter node_exporter-0.18.1.linux-amd64/NOTICE node_exporter-0.18.1.linux-amd64/LICENSE

Create a directory for node_exporter software.

[root@prometheus-01 tmp]# mkdir -p /var/lib/prometheus/node_exporter

Move extracted files to /var/lib/prometheus/node_exporter directory.

[root@prometheus-01 tmp]# mv node_exporter-0.18.1.linux-amd64/* /var/lib/prometheus/node_exporter [root@prometheus-01 tmp]# cd

Change ownership of node_exporter directory.

[root@prometheus-01 ~]# chown -R prometheus:prometheus /var/lib/prometheus/node_exporter/

Create a systemd service for node_exporter, so it can be started automically during Linux server startup.

[root@prometheus-01 ~]# vi /usr/lib/systemd/system/node_exporter.service

Add following lines of code in this file.

[Unit] Description=Node Exporter Wants=network-online.target After=network-online.target [Service] User=prometheus ExecStart=/var/lib/prometheus/node_exporter/node_exporter [Install] WantedBy=default.target

Enable and start node_exporter service.

[root@prometheus-01 ~]# systemctl enable --now node_exporter.service Created symlink from /etc/systemd/system/default.target.wants/node_exporter.service to /usr/lib/systemd/system/node_exporter.service.

Node Exporter used default service port 9100/tcp. Therefore, we need to allow this service port in Linux firewall.

[root@prometheus-01 ~]# firewall-cmd --permanent --add-port=9100/tcp success [root@prometheus-01 ~]# firewall-cmd --reload success

 

Add a Node_Exporter Target in Prometheus Server:

We have successfully installed node_exporter, now we have to add it as a target in Prometheus server.

Edit Prometheus configuration file.

[root@prometheus-01 ~]# vi /etc/prometheus/prometheus.yml

and add following directives at the end of file.

- job_name: 'node_exporter' static_configs: - targets: ['localhost:9100']

Restart Prometheus service to take changes into effect.

[root@prometheus-01 ~]# systemctl restart prometheus.service

Browse URL http://prometheus-01.example.com:9090/ and search for some node metrics there.

We are searching node_memory_MemAvailable_bytes metric here.

04-prometheus-query-node-metrics

To see the graph of this metric click on Graph button.

05-prometheus-draw-graph-node-metrics

Since, we have recently added this node to Prometheus Server, therefore, there isn't much time series data available yet. Hence, we have adjusted time to 1m to fill the graph completely.

We have successfully installed Prometheus network monitoring server on CentOS 7 along with node_exporter.

No comments:

Post a comment