Configure Varnish Cache as Load Balancer on CentOS 7 - CentLinux

Latest

Tuesday, 15 October 2019

Configure Varnish Cache as Load Balancer on CentOS 7

Configure Varnish Cache as Load Balancer on CentOS 7

Varnish Cache is an HTTP Accelerator designed for content-heavy dynamic websites and APIs. Varnish Cache is usually installed on the same web server, where Varnish Cache acts as the front-end and accelerator for the hosted websites. Varnish Cache is free and open source software which is distributed under two-clause BSD license.

Varnish cache supports multiple back-end hosts, therefore we can also used Varnish Cache as the Reverse Proxy for load balancing of a cluster of web servers.

In this article, we are installing and configuring Varnish Cache as Load Balancer on CentOS 7.

 

This Article Provides:

     

    Environment Specification:

    We have configured a CentOS 7 virtual machine with following specifications:

    • CPU - 3.4 Ghz (1 Core)
    • Memory - 1 GB
    • Storage - 20 GB
    • Operating System - CentOS 7.7
    • Hostname - varnish-cache-01.example.com
    • IP Address - 192.168.116.213 /24

     

    Installing Apache HTTP Server on CentOS 7:

    Connect with varnish-cache-01.example.com using ssh as root user.

    Build yum cache for standard CentOS 7 repositories.

    [root@varnish-cache-01 ~]# yum makecache fast Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.ges.net.pk * extras: mirrors.ges.net.pk * updates: mirrors.ges.net.pk base | 3.6 kB 00:00 extras | 2.9 kB 00:00 updates | 2.9 kB 00:00 Metadata Cache Created

    Update CentOS 7 server packages.

    [root@varnish-cache-01 ~]# yum update Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.ges.net.pk * extras: mirrors.ges.net.pk * updates: mirrors.ges.net.pk No packages marked for update

    Our CentOS 7 server is already up-to-date.

    Install Apache HTTP server using yum command.

    [root@varnish-cache-01 ~]# yum install -y httpd Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.ges.net.pk * extras: mirrors.ges.net.pk * updates: mirrors.ges.net.pk Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-90.el7.centos will be installed --> Processing Dependency: httpd-tools = 2.4.6-90.el7.centos for package: httpd-2.4.6-90.el7.centos.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-90.el7.centos.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-90.el7.centos.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-90.el7.centos.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-5.el7 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-90.el7.centos will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: httpd x86_64 2.4.6-90.el7.centos base 2.7 M Installing for dependencies: apr x86_64 1.4.8-5.el7 base 103 k apr-util x86_64 1.5.2-6.el7 base 92 k httpd-tools x86_64 2.4.6-90.el7.centos base 91 k mailcap noarch 2.1.41-2.el7 base 31 k Transaction Summary ================================================================================ Install 1 Package (+4 Dependent packages) Total download size: 3.0 M Installed size: 10 M Downloading packages: (1/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:01 (2/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:01 (3/5): httpd-tools-2.4.6-90.el7.centos.x86_64.rpm | 91 kB 00:02 (4/5): apr-1.4.8-5.el7.x86_64.rpm | 103 kB 00:04 (5/5): httpd-2.4.6-90.el7.centos.x86_64.rpm | 2.7 MB 00:08 -------------------------------------------------------------------------------- Total 379 kB/s | 3.0 MB 00:08 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-5.el7.x86_64 1/5 Installing : apr-util-1.5.2-6.el7.x86_64 2/5 Installing : httpd-tools-2.4.6-90.el7.centos.x86_64 3/5 Installing : mailcap-2.1.41-2.el7.noarch 4/5 Installing : httpd-2.4.6-90.el7.centos.x86_64 5/5 Verifying : apr-1.4.8-5.el7.x86_64 1/5 Verifying : mailcap-2.1.41-2.el7.noarch 2/5 Verifying : httpd-tools-2.4.6-90.el7.centos.x86_64 3/5 Verifying : apr-util-1.5.2-6.el7.x86_64 4/5 Verifying : httpd-2.4.6-90.el7.centos.x86_64 5/5 Installed: httpd.x86_64 0:2.4.6-90.el7.centos Dependency Installed: apr.x86_64 0:1.4.8-5.el7 apr-util.x86_64 0:1.5.2-6.el7 httpd-tools.x86_64 0:2.4.6-90.el7.centos mailcap.noarch 0:2.1.41-2.el7 Complete!

    Start and enable Apache web service.

    [root@varnish-cache-01 ~]# systemctl enable --now httpd.service Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

    Allow HTTP service in CentOS 7 firewall.

    [root@varnish-cache-01 ~]# firewall-cmd --permanent --add-service=http success [root@varnish-cache-01 ~]# firewall-cmd --reload success

    Browse URL http://varnish-cache-01.example.com in a client's browser.

    01-varnish-cache-centos-7-apache-default-page

    Apache HTTP server is successfully installed and it is serving the default test page.

     

    Configure Apache Virtual Hosts on CentOS 7:

    We are configure two virtual hosts here, that will run on two different ports.

    Create document root directories for virtual hosts.

    [root@varnish-cache-01 ~]# mkdir /var/www/html/{vhost1,vhost2}

    Create default index page for Virtual Host 1.

    [root@varnish-cache-01 ~]# cat > /var/www/html/vhost1/index.html << EOF > <html> > <head><title>Virtual Host1</title></head> > <body><h1>This is the default page of Virtual Host 1...</h1></body> > </html> > EOF

    Similarly, create default index page for Virtual Host 2.

    [root@varnish-cache-01 ~]# cat > /var/www/html/vhost2/index.html << EOF > <html> > <head><title>Virtual Host2</title></head> > <body><h1>This is the default page of Virtual Host 2...</h1></body> > </html> > EOF

    Create configuration file for Virtual Host1.

    [root@varnish-cache-01 ~]# vi /etc/httpd/conf.d/vhost1.conf

    add following directives therein.

    Listen 8081 <VirtualHost *:8081> DocumentRoot "/var/www/html/vhost1" ServerName vhost1.example.com </VirtualHost>

    Similarly, create configuration file for Virtual Host2.

    [root@varnish-cache-01 ~]# vi /etc/httpd/conf.d/vhost2.conf

    add following directives therein.

    Listen 8082 <VirtualHost *:8082> DocumentRoot "/var/www/html/vhost2" ServerName vhost2.example.com </VirtualHost>

    Check Apache configurations for syntax errors.

    [root@varnish-cache-01 ~]# httpd -t Syntax OK

    Since, we are running Apache websites on non-default ports, therefore, we have to add these ports to SELinux port labeling.

    Check, if these ports are already added in SELinux.

    [root@varnish-cache-01 ~]# semanage port -l | grep ^http_port_t http_port_t tcp 80, 81, 443, 488, 8008, 8009, 8443, 9000

    Add ports 8081 and 8082 to type http_port_t SELinux context.

    [root@varnish-cache-01 ~]# semanage port -m -t http_port_t -p tcp 8081 [root@varnish-cache-01 ~]# semanage port -m -t http_port_t -p tcp 8082

    Verify if these ports are added in SELinux port labeling.

    [root@varnish-cache-01 ~]# semanage port -l | grep ^http_port_t http_port_t tcp 8082, 8081, 80, 81, 443, 488, 8008, 8009, 8443, 9000

    Now, we can safely load our Apache configurations.

    [root@varnish-cache-01 ~]# systemctl reload httpd.service

    Allow 8081/tcp and 8082/tcp service ports in CentOS 7 firewall.

    [root@varnish-cache-01 ~]# firewall-cmd --permanent --add-port={8081,8082}/tcp success [root@varnish-cache-01 ~]# firewall-cmd --reload success

    Browse URL http://varnish-cache-01.example.com:8081/ in a client's browser.

    02-varnish-cache-centos-7-apache-virtual-host-1

    Browse URL http://varnish-cache-01.example.com:8082/ in a client's browser.

    03-varnish-cache-centos-7-apache-virtual-host-2

    Both of our Apache virtual hosts has been configured successfully.

     

    Installing Varnish Cache on CentOS 7:

    Varnish Cache software is available in EPEL (Extra Packages for Enterprise Linux) yum repository.

    Therefore, first we have to enable EPEL yum repository as follows.

    [root@varnish-cache-01 ~]# yum install -y epel-release Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.ges.net.pk * extras: mirrors.ges.net.pk * updates: mirrors.ges.net.pk Resolving Dependencies --> Running transaction check ---> Package epel-release.noarch 0:7-11 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: epel-release noarch 7-11 extras 15 k Transaction Summary ================================================================================ Install 1 Package Total download size: 15 k Installed size: 24 k Downloading packages: epel-release-7-11.noarch.rpm | 15 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : epel-release-7-11.noarch 1/1 Verifying : epel-release-7-11.noarch 1/1 Installed: epel-release.noarch 0:7-11 Complete!

    Build cache for EPEL yum repository.

    [root@varnish-cache-01 ~]# yum makecache Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile epel/x86_64/metalink | 7.0 kB 00:00 * base: mirrors.ges.net.pk * epel: sg.fedora.ipserverone.com * extras: mirrors.ges.net.pk * updates: mirrors.ges.net.pk base | 3.6 kB 00:00 epel | 5.3 kB 00:00 extras | 2.9 kB 00:00 updates | 2.9 kB 00:00 (1/13): epel/x86_64/group_gz | 88 kB 00:01 (2/13): epel/x86_64/prestodelta | 1.6 kB 00:00 (3/13): epel/x86_64/updateinfo | 1.0 MB 00:24 (4/13): base/7/x86_64/other_db | 2.6 MB 00:34 (5/13): base/7/x86_64/filelists_db | 7.3 MB 00:35 (6/13): extras/7/x86_64/filelists_db | 207 kB 00:01 (7/13): extras/7/x86_64/other_db | 100 kB 00:01 (8/13): updates/7/x86_64/filelists_db | 1.4 MB 00:05 (9/13): epel/x86_64/primary_db | 6.9 MB 00:43 (10/13): epel/x86_64/other_db | 3.3 MB 00:25 (11/13): updates/7/x86_64/other_db | 134 kB 00:18 (12/13): epel/x86_64/filelists_db | 11 MB 01:02 (13/13): epel/x86_64/updateinfo_zck | 1.5 MB 00:19 Metadata Cache Created

    Now, we can install Varnish Cache software using yum command.

    [root@varnish-cache-01 ~]# yum install -y varnish ... Installed: varnish.x86_64 0:4.0.5-1.el7 Dependency Installed: cpp.x86_64 0:4.8.5-39.el7 dwz.x86_64 0:0.11-3.el7 gcc.x86_64 0:4.8.5-39.el7 glibc-devel.x86_64 0:2.17-292.el7 glibc-headers.x86_64 0:2.17-292.el7 jemalloc.x86_64 0:3.6.0-1.el7 kernel-headers.x86_64 0:3.10.0-1062.1.2.el7 libmpc.x86_64 0:1.0.1-3.el7 mpfr.x86_64 0:3.1.1-4.el7 perl.x86_64 4:5.16.3-294.el7_6 perl-Carp.noarch 0:1.26-244.el7 perl-Encode.x86_64 0:2.51-7.el7 perl-Exporter.noarch 0:5.68-3.el7 perl-File-Path.noarch 0:2.09-2.el7 perl-File-Temp.noarch 0:0.23.01-3.el7 perl-Filter.x86_64 0:1.49-3.el7 perl-Getopt-Long.noarch 0:2.40-3.el7 perl-HTTP-Tiny.noarch 0:0.033-3.el7 perl-PathTools.x86_64 0:3.40-5.el7 perl-Pod-Escapes.noarch 1:1.04-294.el7_6 perl-Pod-Perldoc.noarch 0:3.20-4.el7 perl-Pod-Simple.noarch 1:3.28-4.el7 perl-Pod-Usage.noarch 0:1.63-3.el7 perl-Scalar-List-Utils.x86_64 0:1.27-248.el7 perl-Socket.x86_64 0:2.010-4.el7 perl-Storable.x86_64 0:2.45-3.el7 perl-Text-ParseWords.noarch 0:3.29-4.el7 perl-Time-HiRes.x86_64 4:1.9725-3.el7 perl-Time-Local.noarch 0:1.2300-2.el7 perl-constant.noarch 0:1.27-2.el7 perl-libs.x86_64 4:5.16.3-294.el7_6 perl-macros.x86_64 4:5.16.3-294.el7_6 perl-parent.noarch 1:0.225-244.el7 perl-podlators.noarch 0:2.5.1-3.el7 perl-srpm-macros.noarch 0:1-8.el7 perl-threads.x86_64 0:1.87-4.el7 perl-threads-shared.x86_64 0:1.43-6.el7 python-srpm-macros.noarch 0:3-32.el7 redhat-rpm-config.noarch 0:9.1.0-88.el7.centos varnish-libs.x86_64 0:4.0.5-1.el7 zip.x86_64 0:3.0-11.el7 Complete!

    We have installed the default version of Varnish Cache that is available in EPEL yum repository. However, you can always download and install a latest version of Varnish Cache from their Official Download Page.

     

    Configure Varnish Cache as Load Balancer on CentOS 7:

    Before configuring Varnish Cache, we are required to free the port 80 that is currently used by Apache HTTP server.

    The directive that controls the service port 80 is defined in /etc/httpd/conf/httpd.conf file.

    We can change it using a sed command.

    [root@varnish-cache-01 ~]# sed -i "s/Listen 80/Listen 8080/" /etc/httpd/conf/httpd.conf

    Restart the Apache service to take changes into effect.

    [root@varnish-cache-01 ~]# systemctl restart httpd.service

    Now, port 80 is available and we can use it for Vanish Cache service.

    Edit Varnish Cache configuration file.

    [root@varnish-cache-01 ~]# vi /etc/varnish/varnish.params

    Locate and set following directive therein.

    VARNISH_LISTEN_PORT=80 #Default Port 6081

    We have changed the Varnish Cache default port 6081 with 80.

    It's time to configure the backend for Varnish Cache server.

    These settings are located in /etc/varnish/default.vcl file. We can easily replace this file with our custom configurations.

    Rename the existing default.vcl file using mv command.

    [root@varnish-cache-01 ~]# mv /etc/varnish/default.vcl /etc/varnish/default.vcl.org

    Create a custom backend configuration file.

    [root@varnish-cache-01 ~]# vi /etc/varnish/default.vcl

    and add following lines of codes.

    vcl 4.0; import directors; # Load the directors backend vhost1 { .host = "192.168.116.213"; .port = "8081"; .probe = { .url = "/"; .timeout = 1s; .interval = 5s; .window = 5; .threshold = 3; } } backend vhost2 { .host = "192.168.116.213"; .port = "8082"; .probe = { .url = "/"; .timeout = 1s; .interval = 5s; .window = 5; .threshold = 3; } } sub vcl_init { new lb = directors.round_robin(); # Creating a Load Balancer lb.add_backend(vhost1); # Add Virtual Host 1 lb.add_backend(vhost2); # Add Virtual Host 2 } sub vcl_recv { # send all traffic to the lb director: set req.backend_hint = lb.backend(); }

    Enable and start Varnish Cache service.

    [root@varnish-cache-01 ~]# systemctl enable --now varnish.service Created symlink from /etc/systemd/system/multi-user.target.wants/varnish.service to /usr/lib/systemd/system/varnish.service.

    Enable and start Varnish Cache logging service.

    [root@varnish-cache-01 ~]# systemctl enable --now varnishlog.service Created symlink from /etc/systemd/system/multi-user.target.wants/varnishlog.service to /usr/lib/systemd/system/varnishlog.service.

    Verify the backend list using following command.

    [root@varnish-cache-01 ~]# varnishadm backend.list Backend name Refs Admin Probe vhost1(192.168.116.213,,8081) 1 probe Healthy 5/5 vhost2(192.168.116.213,,8082) 1 probe Healthy 5/5

    Check our website’s response header.

    [root@varnish-cache-01 ~]# curl -I http://varnish-cache-01.example.com

    HTTP/1.1 200 OK Date: Sun, 13 Oct 2019 16:24:07 GMT Server: Apache/2.4.6 (CentOS) Last-Modified: Sun, 13 Oct 2019 09:35:58 GMT ETag: "7d-594c77a7e0839" Content-Length: 125 Content-Type: text/html; charset=UTF-8 X-Varnish: 32770 Age: 0 Via: 1.1 varnish-v4 Connection: keep-alive

    Browse URL http://varnish-cache-01.example.com in a client's browser.

    02-varnish-cache-centos-7-apache-virtual-host-1

    03-varnish-cache-centos-7-apache-virtual-host-2

    The Varnish Cache load balancer redirects user requests to Virtual Host 1 and Virtual Host 2 in a round robin way.

    We have successfully configured Varnish Cache as load balancer on CentOS 7.

    No comments:

    Post a comment