Using Ansible inside an Operator

    Operators use the Kubernetes extension mechanism, custom resource definitions (CRDs), so your custom resource (CR) looks and acts just like the built-in, native Kubernetes objects.

    The CR file format is a Kubernetes resource file. The object has mandatory and optional fields:

    The following list of CR annotations modify the behavior of the Operator:

    Table 2. Ansible-based Operator annotations
    AnnotationDescription

    ansible.operator-sdk/reconcile-period

    Specifies the reconciliation interval for the CR. This value is parsed using the standard Golang package . Specifically, ParseDuration is used which applies the default suffix of s, giving the value in seconds.

    Example Ansible-based Operator annotation

    You can test the logic inside of an Ansible-based Operator running locally by using the make run command from the top-level directory of your Operator project. The make run Makefile target runs the ansible-operator binary locally, which reads from the watches.yaml file and uses your ~/.kube/config file to communicate with a Kubernetes cluster just as the k8s modules do.

    Prerequisites

    • Ansible Runner v2.0.2+

    • v1.0.0+

    • Performed the previous steps for testing the Kubernetes Collection locally

    Procedure

    1. Install your custom resource definition (CRD) and proper role-based access control (RBAC) definitions for your custom resource (CR):

      1. $ make install

      Example output

      1. /usr/bin/kustomize build config/crd | kubectl apply -f -
      2. customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created

    2. Run the make run command:

      1. $ make run

      Example output

      1. /home/user/memcached-operator/bin/ansible-operator run
      2. {"level":"info","ts":1612739145.2871568,"logger":"cmd","msg":"Version","Go Version":"go1.15.5","GOOS":"linux","GOARCH":"amd64","ansible-operator":"v1.10.1","commit":"1abf57985b43bf6a59dcd18147b3c574fa57d3f6"}
      3. ...
      4. {"level":"info","ts":1612739148.347306,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
      5. {"level":"info","ts":1612739148.3488882,"logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_MEMCACHED_CACHE_EXAMPLE_COM","default":2}
      6. {"level":"info","ts":1612739148.3490262,"logger":"cmd","msg":"Environment variable not set; using default value","Namespace":"","envVar":"ANSIBLE_DEBUG_LOGS","ANSIBLE_DEBUG_LOGS":false}
      7. {"level":"info","ts":1612739148.3490646,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"cache.example.com","Options.Version":"v1","Options.Kind":"Memcached"}
      8. {"level":"info","ts":1612739148.350217,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
      9. {"level":"info","ts":1612739148.3506632,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
      10. {"level":"info","ts":1612739148.350784,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting EventSource","source":"kind source: cache.example.com/v1, Kind=Memcached"}
      11. {"level":"info","ts":1612739148.5511978,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting Controller"}

      With the Operator now watching your CR for events, the creation of a CR will trigger your Ansible role to run.

      Consider an example config/samples/<gvk>.yaml CR manifest:

      1. apiVersion: <group>.example.com/v1alpha1
      2. kind: <kind>
      3.   name: “<kind>-sample”

      Because the spec field is not set, Ansible is invoked with no extra variables. Passing extra variables from a CR to Ansible is covered in another section. It is important to set reasonable defaults for the Operator.

    3. Create an instance of your CR with the default variable state set to present:

      1. $ oc apply -f config/samples/<gvk>.yaml

    4. Check that the example-config config map was created:

      Example output

      1. NAME                    STATUS    AGE
      2. example-config          Active    3s

    5. Modify your config/samples/<gvk>.yaml file to set the state field to absent. For example:

      1. apiVersion: cache.example.com/v1
      2. kind: Memcached
      3. metadata:
      4.   name: memcached-sample
      5. spec:
      6.   state: absent

    6. Apply the changes:

      1. $ oc apply -f config/samples/<gvk>.yaml

    7. Confirm that the config map is deleted:

      1. $ oc get configmap

    After you have tested your custom Ansible logic locally inside of an Operator, you can test the Operator inside of a pod on an OKD cluster, which is preferred for production use.

    Procedure

    1. Run the following make commands to build and push the Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

      1. Build the image:

        1. $ make docker-build IMG=<registry>/<user>/<image_name>:<tag>

      2. Push the image to a repository:

        1. $ make docker-push IMG=<registry>/<user>/<image_name>:<tag>

        The name and tag of the image, for example IMG=<registry>/<user>/<image_name>:<tag>, in both the commands can also be set in your Makefile. Modify the IMG ?= controller:latest value to set your default image name.

    2. Run the following command to deploy the Operator:

      By default, this command creates a namespace with the name of your Operator project in the form <project_name>-system and is used for the deployment. This command also installs the RBAC manifests from config/rbac.

    3. Run the following command to verify that the Operator is running:

      1. $ oc get deployment -n <project_name>-system

      Example output

      2. <project_name>-controller-manager       1/1     1            1           8m

    Ansible-based Operators provide logs about the Ansible run, which can be useful for debugging your Ansible tasks. The logs can also contain detailed information about the internals of the Operator and its interactions with Kubernetes.

    Prerequisites

    • Ansible-based Operator running as a deployment on a cluster

    Procedure

    • To view logs from an Ansible-based Operator, run the following command:

      1. $ oc logs deployment/<project_name>-controller-manager \
      2.     -n <namespace> (2)

      Example output

      1. {"level":"info","ts":1612732105.0579333,"logger":"cmd","msg":"Version","Go Version":"go1.15.5","GOOS":"linux","GOARCH":"amd64","ansible-operator":"v1.10.1","commit":"1abf57985b43bf6a59dcd18147b3c574fa57d3f6"}
      2. {"level":"info","ts":1612732105.0587437,"logger":"cmd","msg":"WATCH_NAMESPACE environment variable not set. Watching all namespaces.","Namespace":""}
      3. I0207 21:08:26.110949       7 request.go:645] Throttling request took 1.035521578s, request: GET:https://172.30.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1alpha1?timeout=32s
      4. {"level":"info","ts":1612732107.768025,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"127.0.0.1:8080"}
      5. {"level":"info","ts":1612732107.768796,"logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_MEMCACHED_CACHE_EXAMPLE_COM","default":2}
      6. {"level":"info","ts":1612732107.7688773,"logger":"cmd","msg":"Environment variable not set; using default value","Namespace":"","envVar":"ANSIBLE_DEBUG_LOGS","ANSIBLE_DEBUG_LOGS":false}
      7. {"level":"info","ts":1612732107.7688901,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"cache.example.com","Options.Version":"v1","Options.Kind":"Memcached"}
      8. {"level":"info","ts":1612732107.770032,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
      9. I0207 21:08:27.770185       7 leaderelection.go:243] attempting to acquire leader lease  memcached-operator-system/memcached-operator...
      10. {"level":"info","ts":1612732107.770202,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
      11. I0207 21:08:27.784854       7 leaderelection.go:253] successfully acquired lease memcached-operator-system/memcached-operator
      12. {"level":"info","ts":1612732107.7850506,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting EventSource","source":"kind source: cache.example.com/v1, Kind=Memcached"}
      13. {"level":"info","ts":1612732107.8853772,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting Controller"}
      14. {"level":"info","ts":1612732107.8854098,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting workers","worker count":4}

    You can set the environment variable ANSIBLE_DEBUG_LOGS to True to enable checking the full Ansible result in logs, which can be helpful when debugging.

    Procedure

    • Edit the config/manager/manager.yaml and config/default/manager_auth_proxy_patch.yaml files to include the following configuration:

      1.       containers:
      2.       - name: manager
      3.         env:
      4.         - name: ANSIBLE_DEBUG_LOGS
      5.           value: "True"

    While developing an Ansible-based Operator, it can be helpful to enable additional debugging in logs.

    • Add the ansible.sdk.operatorframework.io/verbosity annotation to your custom resource to enable the verbosity level that you want. For example:

      1. apiVersion: "cache.example.com/v1alpha1"
      2. kind: "Memcached"
      3. metadata:
      4.   name: "example-memcached"
      5.   annotations:
      6.     "ansible.sdk.operatorframework.io/verbosity": "4"
      8.   size: 4