Azure Done Right Series: Azure DevOps and Publishing ARM Test Toolkit results

Previously we went through how to use the ARM Test Toolkit (ARM TTK) in Azure DevOps to test your ARM templates, which involved downloading the ARM TTK PowerShell module and storing it in your code repository.

Today we are going to go through how to use the ARM Test Toolkit without storing any modules in your repository, plus publish the test results without requiring any third party Azure DevOps extensions or tools.

The first thing we need to do is create a PowerShell script that executes the Test-AzTemplate command with the -Pester switch. Pester provides a framework for running Unit Tests to execute and validate PowerShell commands (more information on pester can be found here).

Create a PS script as per below (for this example lets call it Test.ARMTemplate.ps1).

param (
Test-AzTemplate -TemplatePath $TemplatePath -Pester

In our pipeline that will be running the ARM template tests we need to create two tasks.

  1. A PowerShell task to import the ARM TTK and Pester modules, plus run the Pester test
  2. Publish the test results

In order to be able to run Pester successfully with ARM TTK we will need to run Pester version 4, as version 5 currently isn’t supported.

As part of PowerShell task we will conduct the following:

  • Download the ARM TTK module (required to run Test-AzTemplate commands)
  • Import Pester module version 4 (Pester version 4 will get installed if it is not found)
  • Run Pester on script Test.ARMTemplate.ps1 and output the test results to a file
  • Review the results and if any tests failed write an error to the error stream. This is to ensure the Azure DevOps pipeline reports a error when executed (otherwise the Azure pipeline will report a success even though a test failed)

The Azure DevOps PowerShell task is below:

  - powershell: |                  
      $ARMTemplatePath = $env:templateFilePath      
      if((Test-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk) -eq $false){
        git clone --quiet $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk
      import-module $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk      
      try {
        Remove-Module Pester -ErrorAction SilentlyContinue
        Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
      catch {
          $errorMessage = $error[0]
          if ($errorMessage -like "*no valid module file was found*"){
              Install-Module Pester -AllowClobber -RequiredVersion 4.10.1 -Force -SkipPublisherCheck -AcceptLicense
              Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
          else {
              Write-Error -Message $errorMessage
      $results = Invoke-Pester -Script  @{ Path =  '.\Test.ARMTemplate.ps1'; Parameters = @{TemplatePath = ('.\' + $template)} } -OutputFormat NUnitXml -OutputFile $env:BUILD_ARTIFACTSTAGINGDIRECTORY\TEST-ARMTemplate.xml -PassThru
      if ($results.TestResult.Result -contains "Failed"){
        Write-Error -Message "Test Failed"
    failOnStderr: true     
    displayName: Run Integration Tests      
      templateFilePath: ${{parameters.templateFile}}

The final task we to need to do is publish the test results, which is an out of the box Azure pipeline task. The only caveat is that we need to ensure this task will always run even if the previous task fails. This ensures the test results are always published.

The Azure DevOps Publish Test Results task is below:

  - task: PublishTestResults@2
    displayName: Publish Test Results
    condition: always()
      testResultsFormat: NUnit
      searchFolder: $(Build.ArtifactStagingDirectory) 

That’s it we are done!

Now when we execute our test pipeline it will run all tests on our ARM template and publish the pipeline test results, example beloW.

[mailpoet_form id="1"]

Other Recent Blogs

Level 9, 360 Collins Street, 
Melbourne VIC 3000

Level 2, 24 Campbell St,
Sydney NSW 2000

200 Adelaide St,
Brisbane QLD 4000

191 St Georges Terrace
Perth WA 6000

Level 10, 41 Shortland Street

Part of

Arinco trades as Arinco (VIC) Pty Ltd and Arinco (NSW) Pty Ltd. © 2023 All Rights Reserved Arinco™ | Privacy Policy
Arinco acknowledges the Traditional Owners of the land on which our offices are situated, and pay our respects to their Elders past, present and emerging.

Get started on the right path to cloud success today. Our Crew are standing by to answer your questions and get you up and running.