Sign in Subscribe
ansible

Testing an Ansible Collection - Part 1 - Introduction and Morpheus Test Appliance

2bit Coder

• 4 min read

I wrote an Ansible collection for Morpheus mainly to house the inventory plugin that sources and filters inventory targets using Morpheus as a source of truth.  The inventory plugin had various issues over time, and I didn't have any sort of continuous testing in place.  I also wanted to have it available in Ansible Galaxy to make it easier to install for users.  

There should be an easier way to test an Ansible collection, and there may be.  Unfortunately, At a certain point, you have to press forward with the knowledge you have, or you may be looking for it forever.

The initial issues that needed to be addressed were:

  • Ansible Galaxy issues warnings and errors on flake8 and ansible-lint scanning, and will block your collection from importing
  • If your collection successfully imports into galaxy, there is no way to alter or remove it.  You need to release and import a higher version.
  • Testing against an application can require a running instance of that application.
  • Ansible has many versions and runs against every version of Python >=2.7.  Testing against all the permutations is advised.

The most time consuming thing was getting the environment stood up.  Morpheus is a large application and takes a while to spin up the first time, in contrast to a docker container for testing.  I used terraform to create the infrastructure and the VM on AWS using a spare domain and a wildcard certificate I created there.  After that, installation of Morpheus is very easy and works well with the playbook and role here: https://github.com/tryfan/ansible-morpheus-examples/tree/master/all-in-one  Just make sure to enable the initial morpheus_setup variable with some additional config in the group_vars to have Morpheus ready to go before configuration.

morpheus_setup: true
morpheus_setup_appliance_name: MorpheusTest
morpheus_setup_appliance_url: "{{ morpheus_appliance_url }}"
morpheus_setup_master_tenant: MasterTenant
morpheus_setup_admin_username: admin
morpheus_setup_admin_password: morphPass1@
morpheus_setup_admin_email: test@example.com
morpheus_setup_admin_firstname: admin
morpheus_setup_admin_lastname: admin
morpheus_setup_enable_backups: false
morpheus_setup_enable_monitoring: false
morpheus_setup_enable_logs: false
morpheus_setup_hub_mode: skip
group_vars/<group>

Waiting for the appliance to come up takes a while since I am using an AWS ALB to make my life easier, but it's not very quick.  All of this was done in Jenkins so I used lots of environment variables to make it portable.  The Ansible role starts Morpheus, but doesn't test access through the ALB.  The next script does that:

from pymorpheus import MorpheusClient
import os
import time

#print(os.environ)
# Setup vars from env
morpheusUrl = os.environ['morpheus_url']
morpheusUsername = os.environ['morpheus_user']
morpheusPassword = os.environ['morpheus_pass']
morpheusLicense = os.environ['morpheus_license']

morpheusAlive = False
while not morpheusAlive:
    try:
        morpheus = MorpheusClient(morpheusUrl, username=morpheusUsername, password=morpheusPassword)
        result = morpheus.call("get","/ping")
        print(result)
        if result['success'] == True:
            print("it is true")
            morpheusAlive = True
    except:
        print("Checking...")
        time.sleep(4)
    
licenseJson = '{"license": "%s"}' % morpheusLicense
putLicense = morpheus.call("post", "/license", jsonpayload=licenseJson)

I'm using the pymorpheus python module to make things easier.  Once this completes, the setup_ansible.sh script runs to prepare the Ansible environment.  I've already created and entered a virtualenv to install the other requirements.  The AWS credentials are coming out of Jenkins.  I'm using a static version of the collection just so I'm sure everything will work.

ansible-galaxy collection install -p ./collections https://github.com/tryfan/ansible-collection-morpheus-core/releases/download/v0.2.2/morpheus-core-0.2.2.tar.gz
sed -i -r "s@MORPHEUSURL@$morpheus_url@" group_vars/all
sed -i -r "s@AWSACCESSKEY@$AWS_ACCESS_KEY_ID@" group_vars/all
sed -i -r "s@AWSSECRETKEY@$AWS_SECRET_ACCESS_KEY@" group_vars/all
echo morpheus_token: $(curl -XPOST "${morpheus_url}/oauth/token?grant_type=password&scope=write&client_id=morph-api" --data-urlencode 'username=${morpheus_username}' --data-urlencode 'password=${morpheus_password} | jq -r '.access_token') >> group_vars/all

For the Morpheus configuration, I used quite a few roles from the morpheus.core collection:

- hosts: morpheus
  roles:
    - morpheus.core.settings
    - morpheus.core.groups
    - morpheus.core.clouds
    - morpheus.core.keyscerts
    - morpheus.core.virtualimages
    - morpheus.core.nodetypes
    - morpheus.core.instancetypes
    - morpheus.core.layouts

I wanted to test with CentOS, Ubuntu, and a Windows VM to verify the platforms were getting detected.  CentOS and Ubuntu were fine, but I needed to add a Windows 2019 AMI to build.  windowsPassword is required to provision, but not important since I won't be logging into the instance. It ended up looking like this:

morpheus_url: MORPHEUSURL
morpheus_token: MORPHEUSTOKEN
morpheus_settings:
  windowsPassword: "morphSecret2@"
morpheus_groups:
  - name: maingroup
    code: maingroup
morpheus_clouds:
  - name: awscloud
    code: awscloud
    location: useast1
    visibility: private
    inventoryLevel: none
    groupName: maingroup
    zoneType:
      code: amazon
    config:
      importExisting: "off"
      isVpc: "true"
      endpoint: "ec2.us-east-1.amazonaws.com"
      accessKey: AWSACCESSKEY
      secretKey: AWSSECRETKEY
morpheus_keyscerts:
  - name: jenkinskey
    publicKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxuQdW2P1ZP6gi6gJ/EBwkDzzPQhHxAl+mPTsEuaHrQV9MZb2Hu8X/qzDT5hgIE/0R5rQDnPnwKfRVgX7WAItUQxBu06FYW22CeXB4puJa2s9tuUtKTUSVr4nsyAUUh0ZFtIUoSR8RiNAhLpUBQ6UDZRqfE1vQPZ8hoHZwm6Whw56VFQu9ki2vP401qMSV+vRtCp+d2bAegPIwpU47CnimHhKu781R2lz7r8LOTHjiOrvVLovg/6DdyUfXAayF4IciQpLhioSTXYzYYkDspjP9gw99AaVvyN2rLX+wO6voXBg1OxQkb5Cm4HKfZ6nKPL6v6sFYoRFdGXEV6R7Y/Szb
morpheus_virtualimages:
  - name: "Windows 2019 AMI New"
    imageType: ami
    isCloudInit: "true"
    installAgent: "true"
    externalId: "ami-0aad84f764a2bd39a"
morpheus_nodetypes:
  - name: "Win2019 Node"
    shortName: win2019node
    provisionTypeCode: amazon
    imageType: ami
    containerVersion: 2
    virtual_image_name: "Windows 2019 AMI New"
morpheus_instancetypes:
  - name: "New Builds"
    code: newbuilds
    category: OS
    featured: "true"
morpheus_layouts:
  - name: Win2019
    instanceVersion: 1
    memoryRequirement: 4096
    provisionTypeCode: amazon
    nodetype_name: "Win2019 Node"
    instancetypecode: newbuilds

Now that all that is done, I wait 180 seconds and make the instances.  I've got JSON templates which I modify because plan IDs are all dynamically generated, so there's a few substitutions in the code.  The code is too long for inclusion here, so it's in the git repository. The JSON payload source files for creating the instances are also in the repository.  

A note about the make_instances.py script: At first I used the plan IDs from my first run and thought they were static.  They are not.  I ended up creating a m4a.16xlarge or similar and had a machine with 384GB of RAM waiting for me.  That's why I rewrote for an ID lookup.

Part 2 will go into actually testing the collection against the real Morpheus test appliance, as well as preparations to submit to Ansible Galaxy.

Sign up for more like this.

Enter your email
Subscribe