Add new CLI sub command to create community validations

Presently, the operator(s) can only execute the official and supported
validations coming from tripleo-validations and validations-common.
Community validations enable a sysadmin to create and execute
validations unique to their environment.

This patch introduces the new Command Line Interface sub command to
create a new community validation skeleton. First, this latter will
check if there is an existing role or a playbook either in the community
validations catalog or the official validations catalog.  And it will
create an Ansible role (with ansible-galaxy[1]) and a playbook in the
~/community-validations directory.

By default, the community validations feature is enabled but may be
disabled by setting [DEFAULT].enable_community_validations to ``False``
in the validation configuration file.

Example:

[stack@localhost]$ validation init my-new-validation
Validation config file found: /etc/validation.cfg
New role created successfully in /home/stack/community-validations/roles/my_new_validation
New playbook created successfully in /home/stack/community-validations/playbooks/my-new-validation.yaml

For a full demo of this new CLI sub command, please take a look at this
asciinema[2].

[1] - https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html
[2] - https://asciinema.org/a/445105

Change-Id: I8fb16e3456696187d4a9d3820740a7639a96e315
Signed-off-by: Gael Chamoulaud (Strider) <gchamoul@redhat.com>

README.rst

@@ -86,4 +86,150 @@ If no hosts key is provided for a given validation, it will be considered as ``h
     The ``reason`` and ``lp`` key are for tracking and documentation purposes,
     the framework won't use those keys.
 
+Community Validations
+=====================
+
+Community Validations enable a sysadmin to create and execute validations unique
+to their environment through the ``validation`` CLI.
+
+The Community Validations will be created and stored in an unique, standardized
+and known place, called ``'community-validations/'``, in the home directory of the
+non-root user which is running the CLI.
+
+.. note::
+   The Community Validations are enabled by default. If you want to disable
+   them, please set ``[DEFAULT].enable_community_validations`` to ``False`` in the
+   validation configuration file located by default in ``/etc/validation.cfg``
+
+The first level of the mandatory structure will be the following (assuming the
+operator uses the ``pennywise`` user):
+
+.. code-block:: console
+
+    /home/pennywise/community-validations
+    ├── library
+    ├── lookup_plugins
+    ├── playbooks
+    └── roles
+
+.. note::
+   The ``community-validations`` directory and its sub directories will be
+   created at the first CLI use and will be checked everytime a new community
+   validation will be created through the CLI.
+
+How To Create A New Community Validation
+----------------------------------------
+
+.. code-block:: console
+
+   [pennywise@localhost]$ validation init my-new-validation
+   Validation config file found: /etc/validation.cfg
+   New role created successfully in /home/pennywise/community-validations/roles/my_new_validation
+   New playbook created successfully in /home/pennywise/community-validations/playbooks/my-new-validation.yaml
+
+The ``community-validations/`` directory should have been created in the home
+directory of the ``pennywise`` user.
+
+.. code-block:: console
+
+    [pennywise@localhost ~]$ cd && tree community-validations/
+    community-validations/
+    ├── library
+    ├── lookup_plugins
+    ├── playbooks
+    │   └── my-new-validation.yaml
+    └── roles
+        └── my_new_validation
+            ├── defaults
+            │   └── main.yml
+            ├── files
+            ├── handlers
+            │   └── main.yml
+            ├── meta
+            │   └── main.yml
+            ├── README.md
+            ├── tasks
+            │   └── main.yml
+            ├── templates
+            ├── tests
+            │   ├── inventory
+            │   └── test.yml
+            └── vars
+                └── main.yml
+
+    13 directories, 9 files
+
+Your new community validation should also be available when listing all the
+validations available on your system.
+
+.. code-block:: console
+
+    [pennywise@localhost ~]$ validation list
+    Validation config file found: /etc/validation.cfg
+    +-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
+    | ID                            | Name                           | Groups                         | Categories                        | Products      |
+    +-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
+    | 512e                          | Advanced Format 512e Support   | ['prep', 'pre-deployment']     | ['storage', 'disk', 'system']     | ['common']    |
+    | check-cpu                     | Verify if the server fits the  | ['prep', 'backup-and-restore', | ['system', 'cpu', 'core', 'os']   | ['common']    |
+    |                               | CPU core requirements          | 'pre-introspection']           |                                   |               |
+    | check-disk-space-pre-upgrade  | Verify server fits the disk    | ['pre-upgrade']                | ['system', 'disk', 'upgrade']     | ['common']    |
+    |                               | space requirements to perform  |                                |                                   |               |
+    |                               | an upgrade                     |                                |                                   |               |
+    | check-disk-space              | Verify server fits the disk    | ['prep', 'pre-introspection']  | ['system', 'disk', 'upgrade']     | ['common']    |
+    |                               | space requirements             |                                |                                   |               |
+    | check-ftype                   | XFS ftype check                | ['pre-upgrade']                | ['storage', 'xfs', 'disk']        | ['common']    |
+    | check-latest-packages-version | Check if latest version of     | ['pre-upgrade']                | ['packages', 'rpm', 'upgrade']    | ['common']    |
+    |                               | packages is installed          |                                |                                   |               |
+    | check-ram                     | Verify the server fits the RAM | ['prep', 'pre-introspection',  | ['system', 'ram', 'memory', 'os'] | ['common']    |
+    |                               | requirements                   | 'pre-upgrade']                 |                                   |               |
+    | check-selinux-mode            | SELinux Enforcing Mode Check   | ['prep', 'pre-introspection']  | ['security', 'selinux']           | ['common']    |
+    | dns                           | Verify DNS                     | ['pre-deployment']             | ['networking', 'dns']             | ['common']    |
+    | no-op                         | NO-OP validation               | ['no-op']                      | ['noop', 'dummy', 'test']         | ['common']    |
+    | ntp                           | Verify all deployed servers    | ['post-deployment']            | ['networking', 'time', 'os']      | ['common']    |
+    |                               | have their clock synchronised  |                                |                                   |               |
+    | service-status                | Ensure services state          | ['prep', 'backup-and-restore', | ['systemd', 'container',          | ['common']    |
+    |                               |                                | 'pre-deployment', 'pre-        | 'docker', 'podman']               |               |
+    |                               |                                | upgrade', 'post-deployment',   |                                   |               |
+    |                               |                                | 'post-upgrade']                |                                   |               |
+    | validate-selinux              | validate-selinux               | ['backup-and-restore', 'pre-   | ['security', 'selinux', 'audit']  | ['common']    |
+    |                               |                                | deployment', 'post-            |                                   |               |
+    |                               |                                | deployment', 'pre-upgrade',    |                                   |               |
+    |                               |                                | 'post-upgrade']                |                                   |               |
+    | my-new-validation             | Brief and general description  | ['prep', 'pre-deployment']     | ['networking', 'security', 'os',  | ['community'] |
+    |                               | of the validation              |                                | 'system']                         |               |
+    +-------------------------------+--------------------------------+--------------------------------+-----------------------------------+---------------+
+
+To get only the list of your community validations, you can filter by products:
+
+.. code-block:: console
+
+   [pennywise@localhost]$ validation list --product community
+   Validation config file found: /etc/validation.cfg
+   +-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
+   | ID                | Name                                     | Groups                     | Categories                               | Products      |
+   +-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
+   | my-new-validation | Brief and general description of the     | ['prep', 'pre-deployment'] | ['networking', 'security', 'os',         | ['community'] |
+   |                   | validation                               |                            | 'system']                                |               |
+   +-------------------+------------------------------------------+----------------------------+------------------------------------------+---------------+
+
+How To Develop Your New Community Validation
+--------------------------------------------
+
+As you can see above, the ``validation init`` CLI sub command has generated a
+new Ansible role by using `ansible-galaxy
+<https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html>`_
+and a new Ansible playbook in the ``community-validations/`` directory.
+
+.. warning::
+   The community validations won't be supported at all. We won't be responsible
+   as well for potential use of malignant code in their validations.  Only the
+   creation of a community validation structure through the new Validation CLI sub
+   command will be supported.
+
+You are now able to implement your own validation by editing the generated
+playbook and adding your ansible tasks in the associated role.
+
+For people not familiar with how to write a validation, get started with this
+`documentation <https://docs.openstack.org/tripleo-validations/latest/contributing/developer_guide.html#writing-validations>`_.
+
 .. _Apache_license: http://www.apache.org/licenses/LICENSE-2.0

setup.cfg

@@ -52,3 +52,4 @@ validation.cli:
     run = validations_libs.cli.run:Run
     history_list = validations_libs.cli.history:ListHistory
     history_get = validations_libs.cli.history:GetHistory
+    init = validations_libs.cli.community:CommunityValidationInit

validation.cfg

@@ -6,6 +6,10 @@
 # Location where the Validation playbooks are stored.
 validation_dir = /usr/share/ansible/validation-playbooks
 
+# Whether to enable the creation and running of Community Validations
+# (boolean value)
+enable_community_validations = True
+
 # Path where the framework is supposed to write logs and results.
 # Note: this should not be a relative path.
 # By default the framework log in $HOME/validations.

validations_libs/cli/community.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python
+
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.
+
+import logging
+
+from validations_libs import constants, utils
+from validations_libs.cli.base import BaseCommand
+from validations_libs.community.init_validation import \
+    CommunityValidation as com_val
+
+LOG = logging.getLogger(__name__)
+
+
+class CommunityValidationInit(BaseCommand):
+    """Initialize Community Validation Skeleton"""
+
+    def get_parser(self, parser):
+        """Argument parser for Community Validation Init"""
+        parser = super(CommunityValidationInit, self).get_parser(parser)
+
+        parser.add_argument(
+            'validation_name',
+            metavar="<validation_name>",
+            type=str,
+            help=(
+                "The name of the Community Validation:\n"
+                "Validation name is limited to contain only lowercase "
+                "alphanumeric characters, plus '_' or '-' and starts "
+                "with an alpha character. \n"
+                "Ex: my-val, my_val2. \n"
+                "This will generate an Ansible role and a playbook in "
+                f"{constants.COMMUNITY_VALIDATIONS_BASEDIR}. "
+                "Note that the structure of this directory will be created at "
+                "the first use."
+            )
+        )
+
+        if self.app:
+            # Merge config and CLI args:
+            return self.base.set_argument_parser(parser)
+        return parser
+
+    def take_action(self, parsed_args):
+        """Take Community Validation Action"""
+
+        co_validation = com_val(parsed_args.validation_name)
+
+        if co_validation.is_community_validations_enabled(self.base.config):
+            LOG.debug(
+                (
+                    "Checking the presence of the community validations "
+                    f"{constants.COMMUNITY_VALIDATIONS_BASEDIR} directory..."
+                )
+            )
+
+            utils.check_community_validations_dir()
+
+            if co_validation.is_role_exists():
+                raise RuntimeError(
+                    (
+                    f"An Ansible role called {co_validation.role_name} "
+                    "already exist in: \n"
+                    f" - {constants.COMMUNITY_ROLES_DIR}\n"
+                    f" - {constants.ANSIBLE_ROLES_DIR}"
+                    )
+                )
+
+            if co_validation.is_playbook_exists():
+                raise RuntimeError(
+                    (
+                    f"An Ansible playbook called {co_validation.playbook_name} "
+                    "already exist in: \n"
+                    f" - {constants.COMMUNITY_PLAYBOOKS_DIR}\n"
+                    f" - {constants.ANSIBLE_VALIDATION_DIR}"
+                    )
+                )
+
+            co_validation.execute()
+        else:
+            raise RuntimeError(
+                "The Community Validations are disabled:\n"
+                "To enable them, set [DEFAULT].enable_community_validations "
+                "to 'True' in the configuration file."
+            )

validations_libs/community/__init__.py

@@ -0,0 +1,15 @@
+#!/usr/bin/env python
+
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.

validations_libs/community/init_validation.py

@@ -0,0 +1,207 @@
+#!/usr/bin/env python
+
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.
+#
+
+import logging
+import re
+from pathlib import Path
+
+from validations_libs import constants, utils
+
+LOG = logging.getLogger(__name__)
+
+
+class CommunityValidation:
+    """Init Community Validation Role and Playbook Command Class
+
+    Initialize a new community role using ansible-galaxy and create a playboook
+    from a template.
+    """
+
+    def __init__(self, validation_name):
+        """Construct Role and Playbook."""
+        self._validation_name = validation_name
+
+    def execute(self):
+        """Execute the actions necessary to create a new community validation
+
+        Check if the role name is compliant with Ansible specification
+        Initializing the new role using ansible-galaxy
+        Creating the validation playbook from a template on disk
+
+        :rtype: ``NoneType``
+        """
+        if not self.is_role_name_compliant:
+            raise RuntimeError(
+                "Role Name are limited to contain only lowercase "
+                "alphanumeric characters, plus '_', '-' and start with an "
+                "alpha character."
+            )
+
+        cmd = ['ansible-galaxy', 'init', '-v',
+               '--offline', self.role_name,
+               '--init-path', self.role_basedir]
+
+        result = utils.run_command_and_log(LOG, cmd)
+
+        if result != 0:
+            raise RuntimeError(
+                (
+                    f"Ansible Galaxy failed to create the role "
+                    f"{self.role_name}, returned {result}."
+                )
+            )
+
+        LOG.info(f"New role created successfully in {self.role_dir_path}")
+
+        try:
+            self.create_playbook()
+        except (PermissionError, OSError) as error:
+            raise RuntimeError(
+                (
+                    f"Exception {error} encountered while trying to write "
+                    f"the community validation playbook file {self.playbook_path}."
+                )
+            )
+
+        LOG.info(f"New playbook created successfully in {self.playbook_path}")
+
+    def create_playbook(self):
+        """Create the playbook for the new community validation"""
+        playbook = constants.COMMUNITY_PLAYBOOK_TEMPLATE.format(self.role_name)
+        with open(self.playbook_path, 'w') as playbook_file:
+            playbook_file.write(playbook)
+
+    def is_role_exists(self):
+        """New role existence check
+
+        This class method checks if the new role name is already existing
+        in the official validations catalog and in the current community
+        validations directory.
+
+        First, it gets the list of the role names available in
+        ``constants.ANSIBLE_ROLES_DIR``. If there is a match in at least one
+        of the directories, it returns ``True``, otherwise ``False``.
+
+        :rtype: ``Boolean``
+        """
+        non_community_roles = [
+            Path(x).name
+            for x in Path(constants.ANSIBLE_ROLES_DIR).iterdir()
+            if x.is_dir()
+        ]
+
+        return Path(self.role_dir_path).exists() or \
+            self.role_name in non_community_roles
+
+    def is_playbook_exists(self):
+        """New playbook existence check
+
+        This class method checks if the new playbook file is already existing
+        in the official validations catalog and in the current community
+        validations directory.
+
+        First, it gets the list of the playbooks yaml file available in
+        ``constants.ANSIBLE_VALIDATIONS_DIR``. If there is a match in at least
+        one of the directories, it returns ``True``, otherwise ``False``.
+
+        :rtype: ``Boolean``
+        """
+        non_community_playbooks = [
+            Path(x).name
+            for x in Path(constants.ANSIBLE_VALIDATION_DIR).iterdir()
+            if x.is_file()
+        ]
+        return Path(self.playbook_path).exists() or \
+            self.playbook_name in non_community_playbooks
+
+    def is_community_validations_enabled(self, base_config):
+        """Checks if the community validations are enabled in the config file
+
+        :param base_config: Contents of the configuration file
+        :type base_config: ``Dict``
+
+        :rtype: ``Boolean``
+        """
+        config = base_config
+        default_conf = (config.get('default', {})
+                        if isinstance(config, dict) else {})
+        return default_conf.get('enable_community_validations', True)
+
+    @property
+    def role_name(self):
+        """Returns the community validation role name
+
+        :rtype: ``str``
+        """
+        if re.match(r'^[a-z][a-z0-9_-]+$', self._validation_name) and \
+           '-' in self._validation_name:
+            return self._validation_name.replace('-', '_')
+        return self._validation_name
+
+    @property
+    def role_basedir(self):
+        """Returns the absolute path of the community validations roles
+
+        :rtype: ``pathlib.PosixPath``
+        """
+        return constants.COMMUNITY_ROLES_DIR
+
+    @property
+    def role_dir_path(self):
+        """Returns the community validation role directory name
+
+        :rtype: ``pathlib.PosixPath``
+        """
+        return Path.joinpath(self.role_basedir, self.role_name)
+
+    @property
+    def is_role_name_compliant(self):
+        """Check if the role name is compliant with Ansible Rules
+
+        Roles Name are limited to contain only lowercase
+        alphanumeric characters, plus '_' and start with an
+        alpha character.
+
+        :rtype: ``Boolean``
+        """
+        if not re.match(r'^[a-z][a-z0-9_]+$', self.role_name):
+            return False
+        return True
+
+    @property
+    def playbook_name(self):
+        """Return the new playbook name with the yaml extension
+
+        :rtype: ``str``
+        """
+        return self._validation_name.replace('_', '-') + ".yaml"
+
+    @property
+    def playbook_basedir(self):
+        """Returns the absolute path of the community playbooks directory
+
+        :rtype: ``pathlib.PosixPath``
+        """
+        return constants.COMMUNITY_PLAYBOOKS_DIR
+
+    @property
+    def playbook_path(self):
+        """Returns the absolute path of the new community playbook yaml file
+
+        :rtype: ``pathlib.PosixPath``
+        """
+        return Path.joinpath(self.playbook_basedir, self.playbook_name)

validations_libs/constants.py

@@ -22,12 +22,17 @@ or as a fallback, when custom locations fail.
 
 import os
 
+from pathlib import Path
+
 DEFAULT_VALIDATIONS_BASEDIR = '/usr/share/ansible'
 
 ANSIBLE_VALIDATION_DIR = os.path.join(
     DEFAULT_VALIDATIONS_BASEDIR,
     'validation-playbooks')
 
+ANSIBLE_ROLES_DIR = Path.joinpath(Path(DEFAULT_VALIDATIONS_BASEDIR),
+                                  'roles')
+
 VALIDATION_GROUPS_INFO = os.path.join(
     DEFAULT_VALIDATIONS_BASEDIR,
     'groups.yaml')
@@ -42,3 +47,82 @@ VALIDATION_ANSIBLE_ARTIFACT_PATH = os.path.join(
 ANSIBLE_RUNNER_CONFIG_PARAMETERS = ['verbosity', 'extravars', 'fact_cache',
                                     'fact_cache_type', 'inventory', 'playbook',
                                     'project_dir', 'quiet', 'rotate_artifacts']
+
+# Community Validations paths
+COMMUNITY_VALIDATIONS_BASEDIR = Path.home().joinpath('community-validations')
+
+COMMUNITY_ROLES_DIR = Path.joinpath(COMMUNITY_VALIDATIONS_BASEDIR, 'roles')
+
+COMMUNITY_PLAYBOOKS_DIR = Path.joinpath(
+    COMMUNITY_VALIDATIONS_BASEDIR, 'playbooks')
+
+COMMUNITY_LIBRARY_DIR = Path.joinpath(
+    COMMUNITY_VALIDATIONS_BASEDIR, 'library')
+
+COMMUNITY_LOOKUP_DIR = Path.joinpath(
+    COMMUNITY_VALIDATIONS_BASEDIR, 'lookup_plugins')
+
+COMMUNITY_VALIDATIONS_SUBDIR = [COMMUNITY_ROLES_DIR,
+                                COMMUNITY_PLAYBOOKS_DIR,
+                                COMMUNITY_LIBRARY_DIR,
+                                COMMUNITY_LOOKUP_DIR]
+
+COMMUNITY_PLAYBOOK_TEMPLATE = \
+"""---
+# This playbook has been generated by the `validation init` CLI.
+#
+# As shown here in this template, the validation playbook requires three
+# top-level directive:
+#   ``hosts``, ``vars -> metadata`` and ``roles``.
+#
+# ``hosts``: specifies which nodes to run the validation on. The options can
+#            be ``all`` (run on all nodes), or you could use the hosts defined
+#            in the inventory.
+# ``vars``: this section serves for storing variables that are going to be
+#           available to the Ansible playbook. The validations API uses the
+#           ``metadata`` section to read each validation's name and description
+#           These values are then reported by the API.
+#
+# The validations can be grouped together by specyfying a ``groups`` metadata.
+# Groups function similar to tags and a validation can thus be part of many
+# groups. To get a full list of the groups available and their description,
+# please run the following command on your Ansible Controller host:
+#
+#    $ validation show group
+#
+# The validations can also be categorized by technical domain and acan belong to
+# one or multiple ``categories``. For example, if your validation checks some
+# networking related configuration, you may want to put ``networking`` as a
+# category.  Note that this section is open and you are free to categorize your
+# validations as you like.
+#
+# The ``products`` section refers to the product on which you would like to run
+# the validation. It's another way to categorized your community validations.
+# Note that, by default, ``community`` is set in the ``products`` section to
+# help you list your validations by filtering by products:
+#
+#    $ validation list --product community
+#
+- hosts: hostname
+  gather_facts: false
+  vars:
+    metadata:
+      name: Brief and general description of the validation
+      description: |
+        The complete description of this validation should be here
+# GROUPS:
+# Run ``validation show group`` to get the list of groups
+# :type group: `list`
+# If you don't want to add groups for your validation, just
+# set an empty list to the groups key
+      groups: []
+# CATEGORIES:
+# :type group: `list`
+# If you don't want to categorize your validation, just
+# set an empty list to the categories key
+      categories: []
+      products:
+        - community
+  roles:
+    - {}
+"""

validations_libs/tests/cli/test_community.py

@@ -0,0 +1,92 @@
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.
+#
+try:
+    from unittest import mock
+except ImportError:
+    import mock
+
+from validations_libs.cli import community
+from validations_libs.cli import base
+from validations_libs.community.init_validation import \
+    CommunityValidation as cv
+from validations_libs.tests import fakes
+from validations_libs.tests.cli.fakes import BaseCommand
+
+
+class TestCommunityValidationInit(BaseCommand):
+
+    def setUp(self):
+        super(TestCommunityValidationInit, self).setUp()
+        self.cmd = community.CommunityValidationInit(self.app, None)
+        self.base = base.Base()
+
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.execute')
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
+        return_value=False)
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
+        return_value=False)
+    def test_validation_init(self,
+                             mock_role_exists,
+                             mock_play_exists,
+                             mock_execute):
+        args = self._set_args(['my_new_community_val'])
+        verifylist = [('validation_name', 'my_new_community_val')]
+
+        parsed_args = self.check_parser(self.cmd, args, verifylist)
+        self.cmd.take_action(parsed_args)
+
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_community_validations_enabled',
+        return_value=False)
+    def test_validation_init_with_com_val_disabled(self, mock_config):
+        args = self._set_args(['my_new_community_val'])
+        verifylist = [('validation_name', 'my_new_community_val')]
+
+        parsed_args = self.check_parser(self.cmd, args, verifylist)
+        self.assertRaises(RuntimeError, self.cmd.take_action,
+                          parsed_args)
+
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
+        return_value=True)
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
+        return_value=False)
+    def test_validation_init_with_role_existing(self, mock_playbook_exists,
+                                                mock_role_exists):
+        args = self._set_args(['my_new_community_val'])
+        verifylist = [('validation_name', 'my_new_community_val')]
+
+        parsed_args = self.check_parser(self.cmd, args, verifylist)
+        self.assertRaises(RuntimeError, self.cmd.take_action,
+                          parsed_args)
+
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_role_exists',
+        return_value=False)
+    @mock.patch(
+        'validations_libs.community.init_validation.CommunityValidation.is_playbook_exists',
+        return_value=True)
+    def test_validation_with_playbook_existing(self, mock_playbook_exists,
+                                               mock_role_exists):
+        args = self._set_args(['my_new_community_val'])
+        verifylist = [('validation_name', 'my_new_community_val')]
+
+        parsed_args = self.check_parser(self.cmd, args, verifylist)
+        self.assertRaises(RuntimeError, self.cmd.take_action,
+                          parsed_args)

validations_libs/tests/community/__init__.py

@@ -0,0 +1,14 @@
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.
+#

validations_libs/tests/community/test_init_validation.py

@@ -0,0 +1,172 @@
+#   Copyright 2021 Red Hat, Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License"); you may
+#   not use this file except in compliance with the License. You may obtain
+#   a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#   License for the specific language governing permissions and limitations
+#   under the License.
+#
+
+try:
+    from unittest import mock
+except ImportError:
+    import mock
+
+from pathlib import PosixPath
+from unittest import TestCase
+
+from validations_libs import constants
+from validations_libs.community.init_validation import \
+    CommunityValidation as cv
+from validations_libs.tests import fakes
+
+
+class TestCommunityValidation(TestCase):
+
+    def setUp(self):
+        super(TestCommunityValidation, self).setUp()
+
+    def test_role_name_underscored(self):
+        validation_name = "my_new_validation"
+        co_val = cv(validation_name)
+        role_name = co_val.role_name
+        self.assertEqual(role_name, validation_name)
+
+    def test_role_name_with_underscores_and_dashes(self):
+        validation_name = "my_new-validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.role_name, "my_new_validation")
+
+    def test_role_name_with_dashes_only(self):
+        validation_name = "my-new-validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.role_name,
+                         "my_new_validation")
+
+    def test_role_name_compliant(self):
+        validation_name = "my_new_validation"
+        co_val = cv(validation_name)
+        self.assertTrue(co_val.is_role_name_compliant)
+
+    def test_role_name_not_compliant(self):
+        validation_name = "123_my_new-validation"
+        co_val = cv(validation_name)
+        self.assertFalse(co_val.is_role_name_compliant)
+
+    def test_role_basedir(self):
+        validation_name = "my_new-validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.role_basedir,
+                         constants.COMMUNITY_ROLES_DIR)
+
+    def test_playbook_name_with_underscores(self):
+        validation_name = "my_new_validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.playbook_name,
+                         "my-new-validation.yaml")
+
+    def test_playbook_name_with_underscores_and_dashes(self):
+        validation_name = "my_new-validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.playbook_name,
+                         "my-new-validation.yaml")
+
+    def test_playbook_basedir(self):
+        validation_name = "my_new-validation"
+        co_val = cv(validation_name)
+        self.assertEqual(co_val.playbook_basedir,
+                         constants.COMMUNITY_PLAYBOOKS_DIR)
+
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=fakes.FAKE_ROLES_ITERDIR1)
+    @mock.patch('pathlib.Path.is_dir')
+    @mock.patch('pathlib.Path.exists', return_value=False)
+    def test_role_already_exists(self,
+                                 mock_path_exists,
+                                 mock_path_is_dir,
+                                 mock_path_iterdir):
+        validation_name = "my-val"
+        co_val = cv(validation_name)
+        self.assertTrue(co_val.is_role_exists())
+
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=fakes.FAKE_ROLES_ITERDIR2)
+    @mock.patch('pathlib.Path.is_dir')
+    @mock.patch('pathlib.Path.exists', return_value=False)
+    def test_role_not_exists(self,
+                             mock_path_exists,
+                             mock_path_is_dir,
+                             mock_path_iterdir):
+        validation_name = "my-val"
+        co_val = cv(validation_name)
+        self.assertFalse(co_val.is_role_exists())
+
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=fakes.FAKE_PLAYBOOKS_ITERDIR1)
+    @mock.patch('pathlib.Path.is_file')
+    @mock.patch('pathlib.Path.exists', return_value=True)
+    def test_playbook_already_exists(self,
+                                     mock_path_exists,
+                                     mock_path_is_file,
+                                     mock_path_iterdir):
+        validation_name = "my_val"
+        co_val = cv(validation_name)
+        self.assertTrue(co_val.is_playbook_exists())
+
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=fakes.FAKE_PLAYBOOKS_ITERDIR2)
+    @mock.patch('pathlib.Path.is_file')
+    @mock.patch('pathlib.Path.exists', return_value=False)
+    def test_playbook_not_exists(self,
+                                 mock_path_exists,
+                                 mock_path_is_file,
+                                 mock_path_iterdir):
+        validation_name = "my_val"
+        co_val = cv(validation_name)
+        self.assertFalse(co_val.is_playbook_exists())
+
+    def test_execute_with_role_name_not_compliant(self):
+        validation_name = "3_my-val"
+        co_val = cv(validation_name)
+        self.assertRaises(RuntimeError, co_val.execute)
+
+    @mock.patch('validations_libs.utils.run_command_and_log',
+                return_value=0)
+    @mock.patch('validations_libs.community.init_validation.CommunityValidation.role_basedir',
+                return_value=PosixPath("/foo/bar/roles"))
+    @mock.patch('validations_libs.community.init_validation.LOG',
+                autospec=True)
+    def test_exec_new_role_with_galaxy(self,
+                                       mock_log,
+                                       mock_role_basedir,
+                                       mock_run):
+        validation_name = "my_val"
+        cmd = ['ansible-galaxy', 'init', '-v',
+               '--offline', validation_name,
+               '--init-path', mock_role_basedir]
+        co_val = cv(validation_name)
+        co_val.execute()
+        mock_run.assert_called_once_with(mock_log, cmd)
+
+    @mock.patch('validations_libs.utils.run_command_and_log',
+                return_value=1)
+    @mock.patch('validations_libs.community.init_validation.CommunityValidation.role_basedir',
+                return_value=PosixPath("/foo/bar/roles"))
+    @mock.patch('validations_libs.community.init_validation.LOG',
+                autospec=True)
+    def test_exec_new_role_with_galaxy_and_error(self,
+                                                 mock_log,
+                                                 mock_role_basedir,
+                                                 mock_run):
+        validation_name = "my_val"
+        cmd = ['ansible-galaxy', 'init', '-v',
+               '--offline', validation_name,
+               '--init-path', mock_role_basedir]
+        co_val = cv(validation_name)
+        self.assertRaises(RuntimeError, co_val.execute)

validations_libs/tests/fakes.py

@@ -13,6 +13,7 @@
 #   under the License.
 #
 
+from pathlib import PosixPath
 from validations_libs import constants
 
 VALIDATIONS_LIST = [{
@@ -319,6 +320,15 @@ FAKE_FAILED_RUN = [{'Duration': '0:00:01.761',
 FAKE_VALIDATIONS_PATH = '/usr/share/ansible/validation-playbooks'
 
 DEFAULT_CONFIG = {'validation_dir': '/usr/share/ansible/validation-playbooks',
+                  'enable_community_validations': True,
+                  'ansible_base_dir': '/usr/share/ansible/',
+                  'output_log': 'output.log',
+                  'history_limit': 15,
+                  'fit_width': True}
+
+CONFIG_WITH_COMMUNITY_VAL_DISABLED = {
+    'validation_dir': '/usr/share/ansible/validation-playbooks',
+    'enable_community_validations': False,
     'ansible_base_dir': '/usr/share/ansible/',
     'output_log': 'output.log',
     'history_limit': 15,
@@ -335,6 +345,46 @@ ANSIBLE_ENVIRONNMENT_CONFIG = {'ANSIBLE_CALLBACK_WHITELIST':
                                'profile_tasks',
                                'ANSIBLE_STDOUT_CALLBACK': 'validation_stdout'}
 
+COVAL_SUBDIR = [PosixPath("/foo/bar/community-validations/roles"),
+                PosixPath("/foo/bar/community-validations/playbooks"),
+                PosixPath("/foo/bar/community-validations/library"),
+                PosixPath("/foo/bar/community-validations/lookup_plugins")]
+
+COVAL_MISSING_SUBDIR = [PosixPath("/foo/bar/community-validations/roles"),
+                        PosixPath("/foo/bar/community-validations/playbooks")]
+
+FAKE_COVAL_ITERDIR1 = iter(COVAL_SUBDIR)
+
+FAKE_COVAL_MISSING_SUBDIR_ITERDIR1 = iter(COVAL_MISSING_SUBDIR)
+
+FAKE_ROLES_ITERDIR1 = iter([PosixPath("/u/s/a/roles/role_1"),
+                            PosixPath("/u/s/a/roles/role_2"),
+                            PosixPath("/u/s/a/roles/role_3"),
+                            PosixPath("/u/s/a/roles/role_4"),
+                            PosixPath("/u/s/a/roles/role_5"),
+                            PosixPath("/u/s/a/roles/my_val")])
+
+FAKE_ROLES_ITERDIR2 = iter([PosixPath("/u/s/a/roles/role_1"),
+                            PosixPath("/u/s/a/roles/role_2"),
+                            PosixPath("/u/s/a/roles/role_3"),
+                            PosixPath("/u/s/a/roles/role_4"),
+                            PosixPath("/u/s/a/roles/role_5"),
+                            PosixPath("/u/s/a/roles/role_6")])
+
+FAKE_PLAYBOOKS_ITERDIR1 = iter([PosixPath("/u/s/a/plays/play_1.yaml"),
+                                PosixPath("/u/s/a/plays/play_2.yaml"),
+                                PosixPath("/u/s/a/plays/play_3.yaml"),
+                                PosixPath("/u/s/a/plays/play_4.yaml"),
+                                PosixPath("/u/s/a/plays/play_5.yaml"),
+                                PosixPath("/u/s/a/plays/my-val.yaml")])
+
+FAKE_PLAYBOOKS_ITERDIR2 = iter([PosixPath("/u/s/a/plays/play_1.yaml"),
+                                PosixPath("/u/s/a/plays/play_2.yaml"),
+                                PosixPath("/u/s/a/plays/play_3.yaml"),
+                                PosixPath("/u/s/a/plays/play_4.yaml"),
+                                PosixPath("/u/s/a/plays/play_5.yaml"),
+                                PosixPath("/u/s/a/plays/play_6.yaml")])
+
 
 def fake_ansible_runner_run_return(status='successful', rc=0):
     return status, rc

validations_libs/tests/test_utils.py

@@ -13,11 +13,16 @@
 #   under the License.
 #
 
+import logging
+import os
+import subprocess
+
 try:
     from unittest import mock
 except ImportError:
     import mock
 
+from pathlib import PosixPath
 from unittest import TestCase
 
 from validations_libs import utils, constants
@@ -426,3 +431,132 @@ class TestUtils(TestCase):
         self.assertEqual(
             results['ansible_environment']['ANSIBLE_STDOUT_CALLBACK'],
             fakes.ANSIBLE_ENVIRONNMENT_CONFIG['ANSIBLE_STDOUT_CALLBACK'])
+
+    @mock.patch('validations_libs.utils.LOG', autospec=True)
+    @mock.patch('pathlib.Path.exists',
+                return_value=False)
+    @mock.patch('pathlib.Path.is_dir',
+                return_value=False)
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=iter([]))
+    @mock.patch('pathlib.Path.mkdir')
+    def test_check_creation_community_validations_dir(self, mock_mkdir,
+                                                      mock_iterdir,
+                                                      mock_isdir,
+                                                      mock_exists,
+                                                      mock_log):
+        basedir = PosixPath('/foo/bar/community-validations')
+        subdir = fakes.COVAL_SUBDIR
+        result = utils.check_community_validations_dir(basedir, subdir)
+        self.assertEqual(result,
+                         [PosixPath('/foo/bar/community-validations'),
+                          PosixPath("/foo/bar/community-validations/roles"),
+                          PosixPath("/foo/bar/community-validations/playbooks"),
+                          PosixPath("/foo/bar/community-validations/library"),
+                          PosixPath("/foo/bar/community-validations/lookup_plugins")]
+                         )
+
+    @mock.patch('validations_libs.utils.LOG', autospec=True)
+    @mock.patch('pathlib.Path.is_dir', return_value=True)
+    @mock.patch('pathlib.Path.exists', return_value=True)
+    @mock.patch('pathlib.Path.iterdir',
+                return_value=fakes.FAKE_COVAL_MISSING_SUBDIR_ITERDIR1)
+    @mock.patch('pathlib.Path.mkdir')
+    def test_check_community_validations_dir_with_missing_subdir(self,
+                                                                 mock_mkdir,
+                                                                 mock_iterdir,
+                                                                 mock_exists,
+                                                                 mock_isdir,
+                                                                 mock_log):
+        basedir = PosixPath('/foo/bar/community-validations')
+        subdir = fakes.COVAL_SUBDIR
+        result = utils.check_community_validations_dir(basedir, subdir)
+        self.assertEqual(result,
+                         [PosixPath('/foo/bar/community-validations/library'),
+                          PosixPath('/foo/bar/community-validations/lookup_plugins')])
+
+
+class TestRunCommandAndLog(TestCase):
+    def setUp(self):
+        self.mock_logger = mock.Mock(spec=logging.Logger)
+
+        self.mock_process = mock.Mock()
+        self.mock_process.stdout.readline.side_effect = ['foo\n', 'bar\n']
+        self.mock_process.wait.side_effect = [0]
+        self.mock_process.returncode = 0
+
+        mock_sub = mock.patch('subprocess.Popen',
+                              return_value=self.mock_process)
+        self.mock_popen = mock_sub.start()
+        self.addCleanup(mock_sub.stop)
+
+        self.cmd = ['exit', '0']
+        self.e_cmd = ['exit', '1']
+        self.log_calls = [mock.call('foo'),
+                          mock.call('bar')]
+
+    def test_success_default(self):
+        retcode = utils.run_command_and_log(self.mock_logger, self.cmd)
+        self.mock_popen.assert_called_once_with(self.cmd,
+                                                stdout=subprocess.PIPE,
+                                                stderr=subprocess.STDOUT,
+                                                shell=False,
+                                                cwd=None, env=None)
+        self.assertEqual(retcode, 0)
+        self.mock_logger.debug.assert_has_calls(self.log_calls,
+                                                any_order=False)
+
+    @mock.patch('subprocess.Popen')
+    def test_error_subprocess(self, mock_popen):
+        mock_process = mock.Mock()
+        mock_process.stdout.readline.side_effect = ['Error\n']
+        mock_process.wait.side_effect = [1]
+        mock_process.returncode = 1
+
+        mock_popen.return_value = mock_process
+
+        retcode = utils.run_command_and_log(self.mock_logger, self.e_cmd)
+        mock_popen.assert_called_once_with(self.e_cmd, stdout=subprocess.PIPE,
+                                           stderr=subprocess.STDOUT,
+                                           shell=False, cwd=None,
+                                           env=None)
+
+        self.assertEqual(retcode, 1)
+        self.mock_logger.debug.assert_called_once_with('Error')
+
+    def test_success_env(self):
+        test_env = os.environ.copy()
+        retcode = utils.run_command_and_log(self.mock_logger, self.cmd,
+                                            env=test_env)
+        self.mock_popen.assert_called_once_with(self.cmd,
+                                                stdout=subprocess.PIPE,
+                                                stderr=subprocess.STDOUT,
+                                                shell=False,
+                                                cwd=None, env=test_env)
+        self.assertEqual(retcode, 0)
+        self.mock_logger.debug.assert_has_calls(self.log_calls,
+                                                any_order=False)
+
+    def test_success_cwd(self):
+        test_cwd = '/usr/local/bin'
+        retcode = utils.run_command_and_log(self.mock_logger, self.cmd,
+                                            cwd=test_cwd)
+        self.mock_popen.assert_called_once_with(self.cmd,
+                                                stdout=subprocess.PIPE,
+                                                stderr=subprocess.STDOUT,
+                                                shell=False,
+                                                cwd=test_cwd, env=None)
+        self.assertEqual(retcode, 0)
+        self.mock_logger.debug.assert_has_calls(self.log_calls,
+                                                any_order=False)
+
+    def test_success_no_retcode(self):
+        run = utils.run_command_and_log(self.mock_logger, self.cmd,
+                                        retcode_only=False)
+        self.mock_popen.assert_called_once_with(self.cmd,
+                                                stdout=subprocess.PIPE,
+                                                stderr=subprocess.STDOUT,
+                                                shell=False,
+                                                cwd=None, env=None)
+        self.assertEqual(run, self.mock_process)
+        self.mock_logger.debug.assert_not_called()

validations_libs/utils.py

@@ -20,10 +20,11 @@ import logging
 import os
 import site
 import six
-import sys
+import subprocess
 import uuid
 
 from os.path import join
+from pathlib import Path
 from validations_libs import constants
 from validations_libs.group import Group
 from validations_libs.validation import Validation
@@ -583,3 +584,114 @@ def find_config_file(config_file_name='validation.cfg'):
         if _check_path(current_path):
             return current_path
     return current_path
+
+
+def run_command_and_log(log, cmd, cwd=None,
+                        env=None, retcode_only=True):
+    """Run command and log output
+
+    :param log: Logger instance for logging
+    :type log: `Logger`
+
+    :param cmd: Command to run in list form
+    :type cmd: ``List``
+
+    :param cwd: Current working directory for execution
+    :type cmd: ``String``
+
+    :param env: Modified environment for command run
+    :type env: ``List``
+
+    :param retcode_only: Returns only retcode instead or proc object
+    :type retcdode_only: ``Boolean``
+    """
+    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
+                            stderr=subprocess.STDOUT, shell=False,
+                            cwd=cwd, env=env)
+    if retcode_only:
+        while True:
+            try:
+                line = proc.stdout.readline()
+            except StopIteration:
+                break
+            if line != b'':
+                if isinstance(line, bytes):
+                    line = line.decode('utf-8')
+                log.debug(line.rstrip())
+            else:
+                break
+        proc.stdout.close()
+        return proc.wait()
+    return proc
+
+
+def check_community_validations_dir(
+        basedir=constants.COMMUNITY_VALIDATIONS_BASEDIR,
+        subdirs=constants.COMMUNITY_VALIDATIONS_SUBDIR):
+    """Check presence of the community validations directory structure
+
+    The community validations are stored and located in:
+
+    .. code-block:: console
+
+        /home/<username>/community-validations
+        ├── library
+        ├── lookup_plugins
+        ├── playbooks
+        └── roles
+
+    This function checks for the presence of the community-validations directory
+    in the $HOME of the user running the validation CLI. If the primary
+    directory doesn't exist, this function will create it and will check if the
+    four subdirectories are present and will create them otherwise.
+
+    :param basedir: Absolute path of the community validations
+    :type basedir: ``pathlib.PosixPath``
+
+    :param subdirs: List of Absolute path of the community validations subdirs
+    :type subdirs: ``list`` of ``pathlib.PosixPath``
+
+    :rtype: ``NoneType``
+    """
+    recreated_comval_dir = []
+
+    def create_subdir(subdir):
+        for _dir in subdir:
+            LOG.debug(
+                f"Missing {Path(_dir).name} directory in {basedir}:"
+            )
+            Path.mkdir(_dir)
+            recreated_comval_dir.append(_dir)
+            LOG.debug(
+                f"└── {_dir} directory created successfully..."
+            )
+
+    if Path(basedir).exists and Path(basedir).is_dir():
+        _subdirectories = [x for x in basedir.iterdir() if x.is_dir()]
+        missing_dirs = [
+            _dir for _dir in subdirs
+            if _dir not in _subdirectories
+        ]
+
+        create_subdir(missing_dirs)
+    else:
+        LOG.debug(
+            f"The community validations {basedir} directory is not present:"
+        )
+        Path.mkdir(basedir)
+        recreated_comval_dir.append(basedir)
+        LOG.debug(f"└── {basedir} directory created...")
+        create_subdir(subdirs)
+
+    LOG.debug(
+        (
+            f"The {basedir} directory and its required subtree are present "
+            f"and correct:\n"
+            f"{basedir}/\n"
+            "├── library            OK\n"
+            "├── lookup_plugins     OK\n"
+            "├── playbooks          OK\n"
+            "└── roles              OK\n"
+        )
+    )
+    return recreated_comval_dir