Configure Virtual Data Optimizer (VDO) on CentOS 8 - CentLinux

Latest

Sunday, 22 December 2019

Configure Virtual Data Optimizer (VDO) on CentOS 8

Configure Virtual Data Optimizer (VDO) on CentOS 8

VDO (Virtual Data Optimizer) is a storage software that can be used as an additional storage layer under the local filesystems, iSCSI or Ceph storages. VDO provides inline data reduction for Linux in the form of deduplication, compression and thin provisioning.

In this article, we will configure Virtual Data Optimizer (VDO) on CentOS 8 and then we will create filesystem and persistenly mount the volume.

 

This Article Provides:

     

    Virtual Data Optimizer (VDO) Features:

    The core features of Virtual Data Optimizer (VDO) are:

    • Supports Local and Remote Storages
    • Deduplication
    • Compression
    • Thin Provisioning

     

    Virtual Data Optimizer (VDO) Usage:

    Although we can use Virtual Data Optimizer (VDO) in various situations to save disk space, but two major applications of VDO are:

    • VM storage
    • BLOB storage backends

     

    Environment Specification:

    We have provisioned virtual machine with minimally installed CentOS 8 with following specification.

    • CPU - 3.4 Ghz (2 cores)
    • Memory - 1 GB
    • Storage - 20 GB
    • Operating System - CentOS 8.0
    • Hostname - vdo-storage-01.recipes.com
    • IP Address - 192.168.116.206 /24

     

    Installing Virtual Data Optimizer (VDO) on CentOS 8:

    Connect with vdo-storage-01.recipes.com using ssh as root user.

    Build dnf cache using following command.

    [root@vdo-storage-01 ~]# dnf makecache CentOS-8 - AppStream 3.0 kB/s | 4.3 kB 00:01 CentOS-8 - Base 2.8 kB/s | 3.9 kB 00:01 CentOS-8 - Extras 730 B/s | 1.5 kB 00:02 Metadata cache created.

    Update CentOS 8 packages as follows.

    [root@vdo-storage-01 ~]# dnf update -y ... Upgraded: bash-4.4.19-8.el8_0.x86_64 bind-export-libs-32:9.11.4-17.P2.el8_0.1.x86_64 dracut-049-10.git20190115.el8_0.1.x86_64 dracut-config-rescue-049-10.git20190115.el8_0.1.x86_64 dracut-network-049-10.git20190115.el8_0.1.x86_64 dracut-squash-049-10.git20190115.el8_0.1.x86_64 grub2-common-1:2.02-66.el8_0.1.noarch grub2-pc-1:2.02-66.el8_0.1.x86_64 grub2-pc-modules-1:2.02-66.el8_0.1.noarch grub2-tools-1:2.02-66.el8_0.1.x86_64 grub2-tools-extra-1:2.02-66.el8_0.1.x86_64 grub2-tools-minimal-1:2.02-66.el8_0.1.x86_64 initscripts-10.00.1-1.el8_0.1.x86_64 iptables-1.8.2-9.el8_0.1.x86_64 iptables-ebtables-1.8.2-9.el8_0.1.x86_64 iptables-libs-1.8.2-9.el8_0.1.x86_64 kernel-tools-4.18.0-80.11.2.el8_0.x86_64 kernel-tools-libs-4.18.0-80.11.2.el8_0.x86_64 kmod-25-11.el8_0.2.x86_64 kmod-libs-25-11.el8_0.2.x86_64 kpartx-0.7.8-7.el8_0.2.x86_64 libdnf-0.22.5-5.el8_0.x86_64 libnfsidmap-1:2.3.3-14.el8_0.2.x86_64 libsss_autofs-2.0.0-43.el8_0.3.x86_64 libsss_certmap-2.0.0-43.el8_0.3.x86_64 libsss_idmap-2.0.0-43.el8_0.3.x86_64 libsss_nss_idmap-2.0.0-43.el8_0.3.x86_64 libsss_sudo-2.0.0-43.el8_0.3.x86_64 microcode_ctl-4:20180807a-2.20190618.1.el8_0.x86_64 p11-kit-0.23.14-5.el8_0.x86_64 p11-kit-trust-0.23.14-5.el8_0.x86_64 platform-python-3.6.8-4.el8_0.x86_64 python3-hawkey-0.22.5-5.el8_0.x86_64 python3-libdnf-0.22.5-5.el8_0.x86_64 python3-libs-3.6.8-4.el8_0.x86_64 python3-perf-4.18.0-80.11.2.el8_0.x86_64 python3-rpm-4.14.2-11.el8_0.x86_64 rpm-4.14.2-11.el8_0.x86_64 rpm-build-libs-4.14.2-11.el8_0.x86_64 rpm-libs-4.14.2-11.el8_0.x86_64 rpm-plugin-selinux-4.14.2-11.el8_0.x86_64 rpm-plugin-systemd-inhibit-4.14.2-11.el8_0.x86_64 selinux-policy-3.14.1-61.el8_0.2.noarch selinux-policy-targeted-3.14.1-61.el8_0.2.noarch setup-2.12.2-2.el8.noarch sssd-client-2.0.0-43.el8_0.3.x86_64 sssd-common-2.0.0-43.el8_0.3.x86_64 sssd-kcm-2.0.0-43.el8_0.3.x86_64 sssd-nfs-idmap-2.0.0-43.el8_0.3.x86_64 systemd-239-13.el8_0.5.x86_64 systemd-libs-239-13.el8_0.5.x86_64 systemd-pam-239-13.el8_0.5.x86_64 systemd-udev-239-13.el8_0.5.x86_64 Installed: kernel-4.18.0-80.11.2.el8_0.x86_64 kernel-modules-4.18.0-80.11.2.el8_0.x86_64 grub2-tools-efi-1:2.02-66.el8_0.1.x86_64 Complete!

    To setup Virtual Data Optimizer on CentOS 8, we have to install following two packages.

    • kmod-kvdo - Kernel Modules for Virtual Data Optimizer
    • vdo - Management tools for Virtual Data Optimizer

    We are installing both of these packages using a single dnf command.

    [root@vdo-storage-01 ~]# dnf install -y kmod-kvdo vdo Last metadata expiration check: 0:03:17 ago on Sun 22 Dec 2019 12:05:21 PM PKT. Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: kmod-kvdo x86_64 6.2.0.293-53.el8_0 BaseOS 349 k vdo x86_64 6.2.0.298-10.el8_0 BaseOS 682 k Installing dependencies: kernel-core x86_64 4.18.0-80.11.2.el8_0 BaseOS 24 M python3-pyyaml x86_64 3.12-12.el8 BaseOS 193 k Transaction Summary ================================================================================ Install 4 Packages Total download size: 25 M Installed size: 63 M Downloading Packages: (1/4): python3-pyyaml-3.12-12.el8.x86_64.rpm 51 kB/s | 193 kB 00:03 (2/4): kmod-kvdo-6.2.0.293-53.el8_0.x86_64.rpm 65 kB/s | 349 kB 00:05 (3/4): vdo-6.2.0.298-10.el8_0.x86_64.rpm 143 kB/s | 682 kB 00:04 (4/4): kernel-core-4.18.0-80.11.2.el8_0.x86_64. 266 kB/s | 24 MB 01:30 -------------------------------------------------------------------------------- Total 276 kB/s | 25 MB 01:32 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Running scriptlet: kmod-kvdo-6.2.0.293-53.el8_0.x86_64 1/1 Preparing : 1/1 Installing : python3-pyyaml-3.12-12.el8.x86_64 1/4 Installing : kernel-core-4.18.0-80.11.2.el8_0.x86_64 2/4 Running scriptlet: kernel-core-4.18.0-80.11.2.el8_0.x86_64 2/4 Running scriptlet: kmod-kvdo-6.2.0.293-53.el8_0.x86_64 3/4 Installing : kmod-kvdo-6.2.0.293-53.el8_0.x86_64 3/4 Running scriptlet: kmod-kvdo-6.2.0.293-53.el8_0.x86_64 3/4 Installing : vdo-6.2.0.298-10.el8_0.x86_64 4/4 Running scriptlet: vdo-6.2.0.298-10.el8_0.x86_64 4/4 Running scriptlet: kernel-core-4.18.0-80.11.2.el8_0.x86_64 4/4 Running scriptlet: kmod-kvdo-6.2.0.293-53.el8_0.x86_64 4/4 Running scriptlet: vdo-6.2.0.298-10.el8_0.x86_64 4/4 Verifying : kernel-core-4.18.0-80.11.2.el8_0.x86_64 1/4 Verifying : kmod-kvdo-6.2.0.293-53.el8_0.x86_64 2/4 Verifying : python3-pyyaml-3.12-12.el8.x86_64 3/4 Verifying : vdo-6.2.0.298-10.el8_0.x86_64 4/4 Installed: kmod-kvdo-6.2.0.293-53.el8_0.x86_64 vdo-6.2.0.298-10.el8_0.x86_64 kernel-core-4.18.0-80.11.2.el8_0.x86_64 python3-pyyaml-3.12-12.el8.x86_64 Complete!

    The Virtual Data Optimizer (VDO) service is automatically enabled and started by the installer.

    [root@vdo-storage-01 ~]# systemctl status vdo.service â vdo.service - VDO volume services Loaded: loaded (/usr/lib/systemd/system/vdo.service; enabled; vendor preset:> Active: active (exited) since Sun 2019-12-22 12:54:27 PKT; 20min ago Main PID: 801 (code=exited, status=0/SUCCESS) Tasks: 0 (limit: 5948) Memory: 0B CGroup: /system.slice/vdo.service Dec 22 12:54:12 vdo-storage-01.recipes.com systemd[1]: Starting VDO volume serv> Dec 22 12:54:27 vdo-storage-01.recipes.com systemd[1]: Started VDO volume servi>

    Virtual Data Optimizer (VDO) has been installed on CentOS 8.

     

    Create a VDO Volume in CentOS 8:

    We have added an extra disk as /dev/sdb in our CentOS 8 virtual machine. We will use /dev/sdb as VDO data store and to create VDO volumes.

    While we try to create a VDO volume, we encounter following error.

    [root@vdo-storage-01 ~]# vdo create --name=vdo1 --device=/dev/sdb --vdoLogicalSize=1T Creating VDO vdo1 vdo: ERROR - Kernel module kvdo not installed vdo: ERROR - modprobe: FATAL: Module kvdo not found in directory /lib/modules/4.18.0-80.el8.x86_64

    It is a known bug and as a workaround, we restarted the CentOS 8 server.

    [root@vdo-storage-01 ~]# systemctl reboot

    After restart, we try to create the VDO volume again, and it is working now.

    [root@vdo-storage-01 ~]# vdo create --name=vdo1 --device=/dev/sdb --vdoLogicalSize=1T Creating VDO vdo1 Starting VDO vdo1 Starting compression on VDO vdo1 VDO instance 0 volume is ready at /dev/mapper/vdo1

    Since we are using VDO volume as local filesystem, therefore, we need to create a filesystem in VDO volume.

    [root@vdo-storage-01 ~]# mkfs.xfs -K /dev/mapper/vdo1 meta-data=/dev/mapper/vdo1 isize=512 agcount=4, agsize=67108864 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=268435456, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=131072, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0

     

    Persistently Mount VDO Volume in CentOS 8:

    Create a directory to mount our VDO volume.

    [root@vdo-storage-01 ~]# mkdir /mnt/vdo1

    To persistently mount our VDO volume, we must add an entry in /etc/fstab file.

    [root@vdo-storage-01 ~]# vi /etc/fstab

    Add following line in this file.

    /dev/mapper/vdo1 /mnt/vdo1 xfs defaults,x-systemd.req=vdo.service,discard 0 0

    Execute following command to inform Systemd about the new entry in /etc/fstab file.

    [root@vdo-storage-01 ~]# systemctl daemon-reload

    Mount entries in /etc/fstab file.

    [root@vdo-storage-01 ~]# mount -a

    Check list of block devices.

    [root@vdo-storage-01 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 60G 0 disk ââsda1 8:1 0 1G 0 part /boot ââsda2 8:2 0 59G 0 part ââcl-root 253:0 0 38.2G 0 lvm / ââcl-swap 253:1 0 2.1G 0 lvm [SWAP] ââcl-home 253:2 0 18.7G 0 lvm /home sdb 8:16 0 10G 0 disk ââvdo1 253:3 0 1T 0 vdo /mnt/vdo1 sr0 11:0 1 6.7G 0 rom

    Check status of VDO volumes.

    [root@vdo-storage-01 ~]# vdostats Device 1K-blocks Used Available Use% Space saving% /dev/mapper/vdo1 10485760 4200156 6285604 40% 99%

    We have successfully configured Virtual Data Optimizer (VDO) on CentOS 8.

    1 comment: