Windows module development walkthrough

    Because Windows modules are written in Powershell and need to be run on aWindows host, this guide differs from the usual development walkthrough guide.

    What’s covered in this section:

    Unlike Python module development which can be run on the host that runsAnsible, Windows modules need to be written and tested for Windows hosts.While evaluation editions of Windows can be downloaded fromMicrosoft, these images are usually not ready to be used by Ansible withoutfurther modification. The easiest way to set up a Windows host so that it isready to by used by Ansible is to set up a virtual machine using Vagrant.Vagrant can be used to download existing OS images called boxes that are thendeployed to a hypervisor like VirtualBox. These boxes can either be created andstored offline or they can be downloaded from a central repository calledVagrant Cloud.

    This guide will use the Vagrant boxes created by the packer-windozerepository which have also been uploaded to .To find out more info on how these images are created, please go to the Githubrepo and look at the file.

    Before you can get started, the following programs must be installed (please consult the Vagrant andVirtualBox documentation for installation instructions):

    • Vagrant
    • VirtualBox

    Create a Windows server in a VM

    To create a single Windows Server 2016 instance, run the following:

    This will download the Vagrant box from Vagrant Cloud and add it to the localboxes on your host and then start up that instance in VirtualBox. When startingfor the first time, the Windows VM will run through the sysprep process andthen create a HTTP and HTTPS WinRM listener automatically. Vagrant will finishits process once the listeners are onlinem, after which the VM can be used by Ansible.

    Create an Ansible inventory

    The following Ansible inventory file can be used to connect to the newlycreated Windows VM:

    1. [windows]
    2. WindowsServer  ansible_host=127.0.0.1
    3.  
    4. [windows:vars]
    5. ansible_user=vagrant
    6. ansible_password=vagrant
    7. ansible_port=55986
    8. ansible_connection=winrm
    9. ansible_winrm_transport=ntlm
    10. ansible_winrm_server_cert_validation=ignore

    Note

    The port 55986 is automatically forwarded by Vagrant to theWindows host that was created, if this conflicts with an existing localport then Vagrant will automatically use another one at random and displayshow that in the output.

    The OS that is created is based on the image set. The followingimages can be used:

    When the host is online, it can accessible by RDP on 127.0.0.1:3389 but theport may differ depending if there was a conflict. To get rid of the host, runvagrant destroy —force and Vagrant will automatically remove the VM andany other files associated with that VM.

    While this is useful when testing modules on a single Windows instance, thesehost won’t work without modification with domain based modules. The Vagrantfileat ansible-windowscan be used to create a test domain environment to be used in Ansible. Thisrepo contains three files which are used by both Ansible and Vagrant to createmultiple Windows hosts in a domain environment. These files are:

    • Vagrantfile: The Vagrant file that reads the inventory setup of inventory.yml and provisions the hosts that are required
    • inventory.yml: Contains the hosts that are required and other connection information such as IP addresses and forwarded ports
    • main.yml: Ansible playbook called by Vagrant to provision the domain controller and join the child hosts to the domain

    By default, these files will create the following environment:

    • A single domain controller running on Windows Server 2016
    • Five child hosts for each major Windows Server version joined to that domain
    • A domain with the DNS name domain.local
    • A local administrator account on each host with the username vagrant and password vagrant
    • A domain admin account vagrant-domain@domain.local with the password VagrantPass1

    The domain name and accounts can be modified by changing the variablesdomain_* in the inventory.yml file if it is required. The inventoryfile can also be modified to provision more or less servers by changing thehosts that are defined under the domain_children key. The host variableansible_host is the private IP that will be assigned to the VirtualBox hostonly network adapter while vagrant_box is the box that will be used tocreate the VM.

    To provision the environment as is, run the following:

    1. git clone https://github.com/jborean93/ansible-windows.git
    2. cd vagrant
    3. vagrant up

    Note

    Vagrant provisions each host sequentially so this can take some timeto complete. If any errors occur during the Ansible phase of setting up thedomain, run vagrant provision to rerun just that step.

    • RDP: 295xx
    • SSH: 296xx
    • WinRM HTTP: 297xx
    • WinRM HTTPS: 298xx
    • SMB: 299xx

    Replace xx with the entry number in the inventory file where the domaincontroller started with 00 and is incremented from there. For example, inthe default inventory.yml file, WinRM over HTTPS for SERVER2012R2 isforwarded over port 29804 as it’s the fourth entry in .

    Note

    While an SSH server is available on all Windows hosts but Server2008 (non R2), it is not a support connection for Ansible managing Windowshosts and should not be used with Ansible.

    Windows new module development

    When creating a new module there are a few things to keep in mind:

    • Module code is in Powershell (.ps1) files while the documentation is contained in Python (.py) files of the same name
    • Avoid using Write-Host/Debug/Verbose/Error in the module and add what needs to be returned to the $result variable
    • When trying an exception use Fail-Json -obj $result -message "exception message here" instead
    • Most new modules require check mode and integration tests before they are merged into the main Ansible codebase
    • Avoid using try/catch statements over a large code block, rather use them for individual calls so the error message can be more descriptive
    • Try and catch specific exceptions when using try/catch statements
    • Avoid using PSCustomObjects unless necessary
    • Look for common functions in ./lib/ansible/module_utils/powershell/ and use the code there instead of duplicating work. These can be imported by adding the line #Requires -Module where is the filename to import, and will be automatically included with the module code sent to the Windows target when run via Ansible
    • Ensure the code runs under Powershell v3 and higher on Windows Server 2008 and higher; if higher minimum Powershell or OS versions are required, ensure the documentation reflects this clearly
    • Ansible runs modules under strictmode version 2.0. Be sure to test with that enabled by putting Set-StrictMode -Version 2.0 at the top of your dev script
    • Favour native Powershell cmdlets over executable calls if possible
    • If adding an object to $result, ensure any trailing slashes are removed or escaped, as ConvertTo-Json will fail to convert it
    • Use the full cmdlet name instead of aliases, e.g. Remove-Item over rm
    • Use named parameters with cmdlets, e.g. Remove-Item -Path C:\temp over Remove-Item C:\temp

    A very basic powershell module is included below. It demonstrates how to implement check-mode and diff-support, and also shows a warning to the user when a specific condition is met.

    1. #!powershell
    2.  
    3. # Copyright: (c) 2015, Jon Hawkesworth (@jhawkesworth) <[email protected]>
    4. # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
    5.  
    6. #Requires -Module Ansible.ModuleUtils.Legacy
    7.  
    8. $ErrorActionPreference = "Stop"
    9.  
    10. $params = Parse-Args -arguments $args -supports_check_mode $true
    11. $check_mode = Get-AnsibleParam -obj $params -name "_ansible_check_mode" -type "bool" -default $false
    12.  
    13. $name = Get-AnsibleParam -obj $params -name "name" -type "str" -failifempty $true
    14. $state = Get-AnsibleParam -obj $params -name "state" -type "str" -default "present" -validateset "absent","present"
    15. $value = Get-AnsibleParam -obj $params -name "value" -type "str"
    16. $level = Get-AnsibleParam -obj $params -name "level" -type "str" -validateSet "machine","user","process" -failifempty $true
    17.  
    18. $before_value = [Environment]::GetEnvironmentVariable($name, $level)
    19.  
    20. $result = @{
    21.     before_value = $before_value
    22.     changed = $false
    23.     value = $value
    24. }
    25.  
    26. # When removing environment, set value to $null if set
    27. if ($state -eq "absent" -and $value) {
    28.     Add-Warning -obj $result -message "When removing environment variable '$name' it should not have a value '$value' set"
    29.     $value = $null
    30. } elseif ($state -eq "present" -and (-not $value)) {
    31.     Fail-Json -obj $result -message "When state=present, value must be defined and not an empty string, if you wish to remove the envvar, set state=absent"
    32. }
    33.  
    34. if ($state -eq "present" -and $before_value -ne $value) {
    35.  
    36.     if (-not $check_mode) {
    37.         [Environment]::SetEnvironmentVariable($name, $value, $level)
    38.     }
    39.     $result.changed = $true
    40.  
    41.     if ($diff_mode) {
    42.         if ($before_value -eq $null) {
    43.             $result.diff = @{
    44.                 prepared = " [$level]`n+$name = $value`n"
    45.             }
    46.         } else {
    47.             $result.diff = @{
    48.                 prepared = " [$level]`n-$name = $before_value`n+$name = $value`n"
    49.             }
    50.         }
    51.     }
    52.  
    53. } elseif ($state -eq "absent" -and $before_value -ne $null) {
    54.  
    55.     if (-not $check_mode) {
    56.         [Environment]::SetEnvironmentVariable($name, $null, $level)
    57.     }
    58.     $result.changed = $true
    59.  
    60.     if ($diff_mode) {
    61.         $result.diff = @{
    62.             prepared = " [$level]`n-$name = $before_value`n"
    63.         }
    64.     }
    65.  
    66. }
    67.  
    68. Exit-Json -obj $result

    A slightly more advanced module is which additionally shows how to use different parameter types (bool, str, int, list, dict, path) and a selection of choices for parameters, how to fail a module and how to handle exceptions.

    When in doubt, look at some of the other core modules and see how things have beenimplemented there.

    Sometimes there are multiple ways that Windows offers to complete a task; thisis the order to favour when writing modules:

    • Native Powershell cmdlets like Remove-Item -Path C:\temp -Recurse
    • .NET classes like [System.IO.Path]::GetRandomFileName()
    • WMI objects through the New-CimInstance cmdlet
    • COM objects through New-Object -ComObject cmdlet
    • Calls to native executables like Secedit.exe

    PowerShell modules support a small subset of the #Requires options builtinto PowerShell as well as some Ansible-specific requirements specified by#AnsibleRequires. These statements can be placed at any point in the script,but are most commonly near the top. They are used to make it easier to state therequirements of the module without writing any of the checks. Each requiresstatement must be on its own line, but there can be multiple requires statementsin one script.

    These are the checks that can be used within Ansible modules:

    • #Requires -Module Ansible.ModuleUtils.<module_util>: Added in Ansible 2.4, specifies a module_util to load in for the module execution.
    • #Requires -Version x.y: Added in Ansible 2.5, specifies the version of PowerShell that is required by the module. The module will fail if this requirement is not met.
    • #AnsibleRequires -OSVersion x.y: Added in Ansible 2.5, specifies the OS build version that is required by the module and will fail if this requirement is not met. The actual OS version is derived from [Environment]::OSVersion.Version.
    • #AnsibleRequires -Become: Added in Ansible 2.5, forces the exec runner to run the module with become, which is primarily used to bypass WinRM restrictions. If ansible_become_user is not specified then the SYSTEM account is used instead.

    Windows module utilities

    Like Python modules, PowerShell modules also provide a number of moduleutilities that provide helper functions within PowerShell. These module_utilscan be imported by adding the following line to a PowerShell module:

    This will import the module_util at ./lib/ansible/module_utils/powershell/Ansible.ModuleUtils.Legacy.psm1and enable calling all of its functions.

    The following is a list of module_utils that are packaged with Ansible and a general description of whatthey do:

    • ArgvParser: Utiliy used to convert a list of arguments to an escaped string compliant with the Windows argument parsing rules.
    • CamelConversion: Utility used to convert camelCase strings/lists/dicts to snake_case.
    • CommandUtil: Utility used to execute a Windows process and return the stdout/stderr and rc as separate objects.
    • FileUtil: Utility that expands on the Get-ChildItem and to work with special files like C:\pagefile.sys.
    • Legacy: General definitions and helper utilities for Ansible module.
    • LinkUtil: Utility to create, remove, and get information about symbolic links, junction points and hard inks.
    • SID: Utilities used to convert a user or group to a Windows SID and vice versa.

    For more details on any specific module utility and their requirements, please see the Ansiblemodule utilities source code.

    PowerShell module utilities can be stored outside of the standard Ansibledistribution for use with custom modules. Custom module_utils are placed in afolder called module_utils located in the root folder of the playbook or roledirectory.

    The below example is a role structure that contains two custom module_utilscalled Ansible.ModuleUtils.ModuleUtil1 andAnsible.ModuleUtils.ModuleUtil2:

    1. meta/
    2.   main.yml
    3. defaults/
    4.   main.yml
    5. module_utils/
    6.   Ansible.ModuleUtils.ModuleUtil1.psm1
    7.   Ansible.ModuleUtils.ModuleUtil2.psm1
    8. tasks/

    Each module_util must contain at least one function, and a list of functions, aliases and cmdlets to export for usein a module. This can be a blanket export by using *. For example:

    1. Export-ModuleMember -Alias * -Function * -Cmdlet *

    You can test a module with an Ansible playbook. For example:

    • Create an inventory file in the same directory touch hosts.

    • Populate the inventory file with the variables required to connect to a Windows host(s).

    • Add the following to the new playbook file:

    1. ---
    2. - name: test out windows module
    3.   hosts: windows
    4.   tasks:
    5.   - name: test out module
    6.     win_module:
    7.       name: test name
    • Run the playbook ansible-playbook -i hosts testmodule.yml

    This can be useful for seeing how Ansible runs withthe new module end to end. Other possible ways to test the module areshown below.

    Windows debugging

    Debugging a module currently can only be done on a Windows host. This can beuseful when developing a new module or implementing bug fixes. Theseare some steps that need to be followed to set this up:

    • Copy the module script to the Windows server

    • Copy ./lib/ansible/module_utils/powershell/Ansible.ModuleUtils.Legacy.psm1 to the same directory as the script above

    • To stop the script from exiting the editor on a successful run, in Ansible.ModuleUtils.Legacy.psm1 under the function Exit-Json, replace the last two lines of the function with:

    • To stop the script from exiting the editor on a failed run, in Ansible.ModuleUtils.Legacy.psm1 under the function Fail-Json, replace the last two lines of the function with:
    1. Write-Error -Message (ConvertTo-Json -InputObject $obj -Depth 99)
    • Add the following to the start of the module script that was copied to the server:
    1. ### start setup code
    2. $complex_args = @{
    3.     "_ansible_check_mode" = $false
    4.     "_ansible_diff" = $false
    5.     "path" = "C:\temp"
    6.     "state" = "present"
    7. }
    8.  
    9. Import-Module -Name .\Ansible.ModuleUtils.Legacy.psm1
    10. ### end setup code

    You can add more args to $complex_args as required by the module. Themodule can now be run on the Windows host either directly through Powershellor through an IDE.

    There are multiple IDEs that can be used to debug a Powershell script, two ofthe most popular are

    To be able to view the arguments as passed by Ansible to the module followthese steps.

    • Prefix the Ansible command with to specify that Ansible should keep the exec files on the server.
    • Log onto the Windows server using the same user account that Ansible used to execute the module.
    • Navigate to %TEMP%... It should contain a folder starting with ansible-tmp-.
    • Inside this folder, open the PowerShell script for the module.
    • In this script is a raw JSON script under $json_raw which contains the module arguments under module_args. These args can be assigned manually to the $complex_args variable that is defined on your debug script.

    Windows unit testing

    Currently there is no mechanism to run unit tests for Powershell modules under Ansible CI.

    Integration tests for Ansible modules are typically written as Ansible roles. These testroles are located in ./test/integration/targets. You must first set up your testingenvironment, and configure a test inventory for Ansible to connect to.

    In this example we will set up a test inventory to connect to two hosts and run the integrationtests for win_stat:

    • Create a copy of ./test/integration/inventory.winrm.template and name it inventory.winrm.
    • Fill in entries under [windows] and set the required variables that are needed to connect to the host.
    • To execute the integration tests, run ansible-test windows-integration win_stat; you can replace win_stat with the role you wish to test.

    This will execute all the tests currently defined for that role. You can setthe verbosity level using the -v argument just as you would withansible-playbook.

    When developing tests for a new module, it is recommended to test a scenario once incheck mode and twice not in check mode. This ensures that check modedoes not make any changes but reports a change, as well as that the second run isidempotent and does not report changes. For example:

    1. - name: remove a file (check mode)
    2.   win_file:
    3.     path: C:\temp
    4.     state: absent
    5.   register: remove_file_check
    6.   check_mode: yes
    7.  
    8. - name: get result of remove a file (check mode)
    9.   win_command: powershell.exe "if (Test-Path -Path 'C:\temp') { 'true' } else { 'false' }"
    10.   register: remove_file_actual_check
    11.  
    12. - name: assert remove a file (check mode)
    13.   assert:
    14.     that:
    15.     - remove_file_check is changed
    16.     - remove_file_actual_check.stdout == 'true\r\n'
    17.  
    18. - name: remove a file
    19.   win_file:
    20.     path: C:\temp
    21.     state: absent
    22.   register: remove_file
    23.  
    24. - name: get result of remove a file
    25.   win_command: powershell.exe "if (Test-Path -Path 'C:\temp') { 'true' } else { 'false' }"
    26.   register: remove_file_actual
    27.  
    28. - name: assert remove a file
    29.   assert:
    30.     that:
    31.     - remove_file is changed
    32.     - remove_file_actual.stdout == 'false\r\n'
    33.  
    34. - name: remove a file (idempotent)
    35.   win_file:
    36.     path: C:\temp
    37.     state: absent
    38.   register: remove_file_again
    39.  
    40. - name: assert remove a file (idempotent)
    41.   assert:
    42.     that:

    Windows communication and development support

    Join the IRC channel #ansible-devel or #ansible-windows on freenode fordiscussions about Ansible development for Windows.

    For questions and discussions pertaining to using the Ansible product,use the #ansible channel.