Previously we went through how to use the ARM Test Toolkit (ARM TTK) in Azure DevOps to test your ARM templates, which involved downloading the ARM TTK PowerShell module and storing it in your code repository.

Today we are going to go through how to use the ARM Test Toolkit without storing any modules in your repository, plus publish the test results without requiring any third party Azure DevOps extensions or tools.

The first thing we need to do is create a PowerShell script that executes the Test-AzTemplate command with the -Pester switch. Pester provides a framework for running Unit Tests to execute and validate PowerShell commands (more information on pester can be found here).

Create a PS script as per below (for this example lets call it Test.ARMTemplate.ps1).

param (
    [Parameter()]
    [String]$TemplatePath
)
Test-AzTemplate -TemplatePath $TemplatePath -Pester

In our pipeline that will be running the ARM template tests we need to create two tasks.

1. A PowerShell task to import the ARM TTK and Pester modules, plus run the Pester test
2. Publish the test results

NOTE: In order to be able to run Pester successfully with ARM TTK we will need to run Pester version 4, as version 5 currently isn’t supported.

As part of PowerShell task we will conduct the following:

• Download the ARM TTK module (required to run Test-AzTemplate commands)
• Import Pester module version 4 (Pester version 4 will get installed if it is not found)
• Run Pester on script Test.ARMTemplate.ps1 and output the test results to a file
• Review the results and if any tests failed write an error to the error stream. This is to ensure the Azure DevOps pipeline reports a error when executed (otherwise the Azure pipeline will report a success even though a test failed)

The Azure DevOps PowerShell task is below:

  - powershell: |                  
      $ARMTemplatePath = $env:templateFilePath      
      if((Test-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk) -eq $false){
        git clone https://github.com/Azure/arm-ttk.git --quiet $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk
      }
      import-module $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk      
      try {
        Remove-Module Pester -ErrorAction SilentlyContinue
        Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
      }
      catch {
          $errorMessage = $error[0]
          if ($errorMessage -like "*no valid module file was found*"){
              Install-Module Pester -AllowClobber -RequiredVersion 4.10.1 -Force -SkipPublisherCheck -AcceptLicense
              Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
          }
          else {
              Write-Error -Message $errorMessage
          }
      }            
      $results = Invoke-Pester -Script  @{ Path =  '.\Test.ARMTemplate.ps1'; Parameters = @{TemplatePath = ('.\' + $template)} } -OutputFormat NUnitXml -OutputFile $env:BUILD_ARTIFACTSTAGINGDIRECTORY\TEST-ARMTemplate.xml -PassThru
      if ($results.TestResult.Result -contains "Failed"){
        Write-Error -Message "Test Failed"
      }
    failOnStderr: true     
    displayName: Run Integration Tests      
    env:
      templateFilePath: ${{parameters.templateFile}}

The final task we to need to do is publish the test results, which is an out of the box Azure pipeline task. The only caveat is that we need to ensure this task will always run even if the previous task fails. This ensures the test results are always published.

The Azure DevOps Publish Test Results task is below:

  - task: PublishTestResults@2
    displayName: Publish Test Results
    condition: always()
    inputs:
      testResultsFormat: NUnit
      searchFolder: $(Build.ArtifactStagingDirectory) 

That’s it we are done!

Now when we execute our test pipeline it will run all tests on our ARM template and publish the pipeline test results, example beloW.