Using Molecule to testing the deployment of a Veritas Netbackup Master Server with DR Process on Azure Cloud
Introduction
Molecule is a tool used for testing Ansible roles on different kind of testing environments. You can learn more about the main purpose and concepts of Molecule in the following link:
Setting up Molecule for testing Ansible roles with Vagrant and Testinfra
In a previous post we saw how it works Molecule to testing the deployment of NBU in a local environment using VMs with Vagrant and VirtualBox.
Using Molecule to testing the deployment of a Veritas Netbackup Master Server with DR Process
On this post we are going to learn how to deploy the same infrastructure but this time running on a VM inside the Azure Cloud with a free account.
Prerequisites
The prerequisites for this testing are the following:
Molecule installed with all the components that was detailed in the other post.
And the following:
And finally a fresh copy of the Master branch of the NBU_DRAAS repo:
After that you need to configure your credentials file on your Ansible Machine.
mkdir $HOME/.azure
vim $HOME/.azure/credentialsThe credentials file must contain the following vars for authenticate on your Azure Account from your CLI.
[default]
subscription_id="id"
client_id="appId"
secret="password"
tenant="tenant"To obtain the last three values you must open and Azure CLI on your machine or in the portal web. Then, you must create the following object name whose name is “App Registration”:
az ad sp create-for-rbac --name AnsibleTestingTo obtain the first value you must execute a command for watch the details of your Azure account:
az account showNow your Ansible controller should be able to connect to your Azure Account with connection: local method.
How it works
This repo contain two roles:
- install_master: NBU Master Server installation
- dr_process: Disaster Recovery process
We are going to do the process with the DR role because this role has been developed to install the Master Server and do the Disaster Recovery process after that.
Take a look to the main config file of Molecule for doing the DR process (molecule.yml).
---
# roles/molecule/azure/molecule.yml
dependency:
name: galaxy
driver:
name: azure
platforms:
# IMPORTANT: Set to the value corresponding to {{ azure_resource_group }}.{{ azure_resource_location }}.cloudapp.azure.com
- name: nbu-deploy.westeurope.cloudapp.azure.com
provisioner:
name: ansible
options:
v: True
inventory:
hosts:
all:
vars:
# Azure Testing Environment Variables
azure_vm_name: backup # Recommended to left backup name for DR process testing
azure_vm_type: Standard_D2s_v3 # Azure VM instance type. Default 2 CPUs and 8GB RAM
azure_vm_user: azurenbu # Azure VM admin user
azure_resource_group: nbu-deploy # Azure resource group name (Object Container). Change this to other name to avoid problems of duplicate entries on Azure Cloud DNS if more members of Sorint are testing at the same time than you.
azure_resource_location: westeurope # Azure Data Center region for store VM
local_public_key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" # Change the value if your public key has not the default name id_rsa.pub
# Install_master role inventory variables
nbu_license: KJXZ-9YZW-LOJT-CZCP-3NPN-PJW4-LGO2-LM7M-LMPP-XC
nbu_smart: /software/veritas_customer_registration_key.json
nbu_smart_dest: /software/veritas_customer_registration_key.json
nbu_sw_path: /software
nbu_unarch_path: /software
nbu_version: 8.2
install_master__reinstall: n
dr_process__catalog_packet: /software/DRPKG/Catalog_1594296652_FULL.drpkg
dr_process__catalog_file: /software/DRPKG/Catalog_1594296652_FULL
dr_process__contradr: P@ssw0rd
dr_process__with_install: n
lint: |
set -e
yamllint .
ansible-lint
scenario:
test_sequence:
- dependency
- syntax
- lint
- create
- prepare
- converge
- verify
- cleanup
- destroy
verifier:
name: testinfra
enabled: True
options:
v: 1This file is used for tell to Molecule how to deploy the environment and test the roles inside them.
We need to configure the driver name to Azure. This feature has been installed previously with the installation of the Molecule-Azure plugin. On platforms we need to configure the name of the DNS that our machine is going to use. We could also configure the public IP of the machine but with the Azure Free Account the public IP is dynamic so the best choice is put the DNS.
Next, on the provisioner (Always Ansible) we are going to put all the inventory vars needed for the role execution plus all the vars needed for the Azure deployment. We can change the VM name, the VM instance type, the VM admin user, the resource-group name and the resources region location.
For linting code we are going to use yamllint for checking the good practises of YAML language and Ansible Lint for checking the Ansible Best practises.
The final tests will be performed with testinfra python library.
Also the final test sequence execution are defined to tell molecule which steps are needed for the correct deployment of the testing environment.
Execution
For make the testing all that you need is type the following command inside your Ansible role path.
molecule test -s azureThis command execute the test execution sequence defined on molecule.yml file.
-
Syntax checking of the role code.
-
Linting of the code. The rules are defined on the root path of the role on .yamllint and .ansible-lint files.
-
Deployment of the VM on the Azure Cloud. This task execute the create.yml playbook. This playbook uses the Azure modules developed for Ansible and create all the resources needed for have a VM ready for testing. As you can see Ansible use the connection local method to use your .azure/credentials file to deploy the VM on your account.
The resources created are a resource group which is the container for all the remaining elements of the VM. A storage account which define the storage type of the VM. A virtual network and subnet defined as a private network for the VM. The public IP exposed on the Internet. A group of firewall rules needed for the proper operation of the NBU and take to us access via SSH or test the GUI via web. A virtual NIC. Finally on the playbook is defined a tasks which use all the resources created previously for create the VM with a CentOS7 image created on the Azure Cloud by the community. A tuning.sh script is passed to the machine as an initial config to prepare the VM to gain access to us and Ansible via SSH. You can check the script on the repo.
---
# roles/dr_process/molecule/default/create.yml
- name: Deploy CentOS7 Instance on Azure Cloud for Netbackup
hosts: 127.0.0.1
connection: local
tasks:
- name: install_master | molecule | create.yml | Create provisioning script for Azure VM
template:
src: templates/tuning.j2
dest: provisioning/tuning.sh
- name: install_master | molecule | create.yml | Create a resource group
azure_rm_resourcegroup:
name: "{{ azure_resource_group }}"
location: "{{ azure_resource_location }}"
- name: install_master | molecule | create.yml | Create storage account
azure_rm_storageaccount:
resource_group: "{{ azure_resource_group }}"
name: nbusa001
account_type: Standard_LRS
- name: install_master | molecule | create.yml | Create virtual network
azure_rm_virtualnetwork:
resource_group: "{{ azure_resource_group }}"
name: nbuvn001
address_prefixes: "192.168.0.0/16"
- name: install_master | molecule | create.yml | Add subnet
azure_rm_subnet:
resource_group: "{{ azure_resource_group }}"
name: nbusn001
address_prefix: "192.168.1.0/24"
virtual_network: nbuvn001
- name: install_master | molecule | create.yml | Create public ip
azure_rm_publicipaddress:
resource_group: "{{ azure_resource_group }}"
allocation_method: Dynamic
domain_name: "{{ azure_resource_group }}"
name: nbupi001
- name: install_master | molecule | create.yml | Create security group that allows SSH, HTTPS and NBUping
azure_rm_securitygroup:
resource_group: "{{ azure_resource_group }}"
name: nbusg001
rules:
- name: SSH
protocol: Tcp
destination_port_range: 22
access: Allow
priority: 101
direction: Inbound
- name: HTTPS
protocol: Tcp
destination_port_range: 443
access: Allow
priority: 102
direction: Inbound
- name: NBUping
protocol: Tcp
destination_port_range: 1556
access: Allow
priority: 103
direction: Inbound
- name: install_master | molecule | create.yml | Create NIC
azure_rm_networkinterface:
resource_group: "{{ azure_resource_group }}"
name: nbuni001
virtual_network: nbuvn001
subnet: nbusn001
public_ip_name: nbupi001
security_group: nbusg001
- name: install_master | molecule | create.yml | Create virtual machine
azure_rm_virtualmachine:
resource_group: "{{ azure_resource_group }}"
name: "{{ azure_vm_name }}"
vm_size: "{{ azure_vm_type }}"
storage_account_name: nbusa001
storage_blob_name: nbubd001.vhd
storage_container_name: nbusc001
os_type: Linux
admin_username: "{{ azure_vm_user }}"
ssh_password_enabled: False
ssh_public_keys:
- path: "/home/{{ azure_vm_user }}/.ssh/authorized_keys"
key_data: "{{ local_public_key }}"
network_interfaces: nbuni001
image:
offer: CentOS
publisher: OpenLogic
sku: '7.7'
version: latest
custom_data: "{{ lookup('file', 'provisioning/tuning.sh') }}"- Execution of the prepare.yml playbook which defined tasks to prepare machine for the correct role execution. This tasks copy from our localhost the files needed for the dr process on the VM in addition to other things.
---
---
# roles/dr_process/molecule/default/prepare.yml
- name: Prepare instance for role execution
hosts: all
become: True
vars:
dr_process__nbu_basic_path: /BASIC
dr_process__nbu_backup_src_path: /software/backup_image.tar
dr_process__nbu_backup_dest_path: /BASIC/backup_image.tar
dr_process__backup_files:
- "backup_1594296620_C1_F1.1594296620.img"
- "backup_1594296620_C1_F1.1594296620.info"
- "backup_1594296620_C1_HDR.1594296620.img"
- "backup_1594296620_C1_HDR.1594296620.info"
- "backup_1594296652_C1_F1.1594296652.img"
- "backup_1594296652_C1_F1.1594296652.info"
- "backup_1594296652_C1_HDR.1594296652.img"
- "backup_1594296652_C1_HDR.1594296652.info"
- "backup_1594296652_C1_TIR.1594296652.img"
- "backup_1594296652_C1_TIR.1594296652.info"
tasks:
- name: dr_process | molecule | prepare.yml | Create NBU bin path
file:
path: "{{ nbu_unarch_path }}"
state: directory
owner: root
group: root
- name: dr_process | molecule | prepare.yml | Copy Veritas Registration Key to NBU bin path
copy:
src: "{{ nbu_smart }}"
dest: "{{ nbu_smart_dest }}"
owner: root
group: root
- name: dr_process | molecule | prepare.yml | Create BASIC path
file:
path: "{{ dr_process__nbu_basic_path }}"
state: directory
owner: root
group: root
- name: dr_process | molecule | prepare.yml | Copy backup_image.tar to BASIC path on VM
copy:
src: "{{ dr_process__nbu_backup_src_path }}"
dest: "{{ dr_process__nbu_backup_dest_path }}"
- name: dr_process | molecule | prepare.yml | Extract backup_image.tar in BASIC path
unarchive:
src: "{{ dr_process__nbu_backup_dest_path }}"
dest: "{{ dr_process__nbu_basic_path }}"
remote_src: True
- name: dr_process | molecule | prepare.yml | Copy files from backup to correct location
copy:
src: "{{ dr_process__nbu_basic_path }}/MSDP/DR/{{ item }}"
dest: "{{ dr_process__nbu_basic_path }}"
remote_src: True
loop: "{{ dr_process__backup_files }}"
- name: dr_process | molecule | prepare.yml | Install expect package # noqa 403
yum:
name: expect
state: latest5.- Execution of the converge.yml playbook which have the tasks used for execute the roles on the VM. First the install_master role and after that the dr_process.
---
# roles/molecule/azure/converge.yml
- name: Converge
hosts: all
become: True
tasks:
- name: dr_process | molecule | converge.yml | Include install_master
include_role:
name: "install_master"
- name: dr_process | molecule | converge.yml | Include dr_process
include_role:
name: "dr_process"6.- Then the test-default.py Python script is executed on the VM to run the tests defined on file. We are checking some things like NB processes, users, groups, sockets and packages. If all it's fine Testinfra show a message which say that 100% of tests are passed succesfully.
"""PyTest Fixtures."""
from __future__ import absolute_import
import os
import pytest
import testinfra.utils.ansible_runner
def test_nbu_packages_are_installed(host):
pkgs_list = ["VRTSnbpck","VRTSnbjava","VRTSnetbp","VRTSpbx","VRTSnbjre","VRTSpddea","VRTSnbcfg","VRTSnbclt","VRTSpddes"]
for package in pkgs_list:
nbu_packages = host.package(package)
assert nbu_packages.is_installed
def test_nbu_processes_are_running(host):
proc_nbu_list = ["vnetd","bpclntcmd","nbaudit","bpcd","nbdisco","nbevtmgr","nbemm","nbrb","bprd","bpdbm","bpcompatd","nbjm","bpjobd","nbpem","nbstserv","nbrmms","nbproxy","nbsl","nbim","nbvault","nbsvcmon","bpdbm","nbatd"]
proc_nbu_running = str([host.process.filter()])
for proc in proc_nbu_list:
assert proc in proc_nbu_running
def test_nbu_user_nbwebsvc_exists(host):
nbu_user = host.user("nbwebsvc")
assert nbu_user.exists
def test_nbu_group_nbwebgrp_exists(host):
nbu_group = host.group("nbwebgrp")
assert nbu_group.exists
def test_nbu_ports_are_listenting(host):
ports_list = ["1556","13724","443"]
for port in ports_list:
port_check = host.socket("tcp://0.0.0.0:"+port)
assert port_check.is_listening7.- After that a playbook called cleanup.yml is executed for doing tasks to clean our local environment for prepare them for the next time execution. This playbook remove the known_key of the Azure VM from our known_hosts file.
---
# roles/molecule/azure/cleanup.yml
- name: Cleanup testing environment for next time execution
hosts: 127.0.0.1
connection: local
tasks:
- name: dr_process | molecule | cleanup.yml | Get Domain Name of Azure VM
azure_rm_publicipaddress_info:
resource_group: "{{ azure_resource_group }}"
name: nbupi001
register: vm_public_data
changed_when: False
- name: dr_process | molecule | cleanup.yml | Delete known key from Azure VM on your localhost
shell: "ssh-keygen -f ~/.ssh/known_hosts -R {{ vm_public_data.publicipaddresses[0].dns_settings.fqdn }}"
delegate_to: localhost8.- Finally the destroy.yml playbook is executed. This playbook is used for destroy the VM and all the resources created previously on our Azure Account.
---
# roles/molecule/azure/destroy.yml
- name: Destroying instance of CentOS7 on Azure Cloud
hosts: 127.0.0.1
connection: local
tasks:
- name: dr_process | molecule | destroy.yml | Destroy resource-group items
azure_rm_resourcegroup:
name: "{{ azure_resource_group }}"
force_delete_nonempty: True
state: absentIf any error occurs during the execution of any of the steps, the destruction sequence will be executed and the VM created in the cloud will be automatically destroyed. This gives you the ability to fix the error in the repository code and re-run the entire role again in a totally clean environment.
Summary
In this post we have seen how to deploy a simple virtual machine in the cloud of Azure and test our Ansible code inside it. And we have seen how Molecule facilitates the execution of the testing performing all the necessary tasks in a totally autonomous way. Also consuming a minimum amount of resources in our PC or server since the testing machine is in the cloud of Azure.
This is actually a very basic configuration of Molecule, but there are infinite possibilities to configure and automate our entire testing environment helping us very efficiently when performing the repetitive testing tasks that we would otherwise have to do manually.