Share on facebook
Share on twitter
Share on linkedin
Share on email

Azure Done Right Series: Azure DevOps and Publishing ARM Test Toolkit results

Previously we went through how to use the ARM Test Toolkit (ARM TTK) in Azure DevOps to test your ARM templates, which involved downloading the ARM TTK PowerShell module and storing it in your code repository.

Today we are going to go through how to use the ARM Test Toolkit without storing any modules in your repository, plus publish the test results without requiring any third party Azure DevOps extensions or tools.

The first thing we need to do is create a PowerShell script that executes the Test-AzTemplate command with the -Pester switch. Pester provides a framework for running Unit Tests to execute and validate PowerShell commands (more information on pester can be found here).

Create a PS script as per below (for this example lets call it Test.ARMTemplate.ps1).

param (
Test-AzTemplate -TemplatePath $TemplatePath -Pester

In our pipeline that will be running the ARM template tests we need to create two tasks.

  1. A PowerShell task to import the ARM TTK and Pester modules, plus run the Pester test
  2. Publish the test results

In order to be able to run Pester successfully with ARM TTK we will need to run Pester version 4, as version 5 currently isn’t supported.

As part of PowerShell task we will conduct the following:

  • Download the ARM TTK module (required to run Test-AzTemplate commands)
  • Import Pester module version 4 (Pester version 4 will get installed if it is not found)
  • Run Pester on script Test.ARMTemplate.ps1 and output the test results to a file
  • Review the results and if any tests failed write an error to the error stream. This is to ensure the Azure DevOps pipeline reports a error when executed (otherwise the Azure pipeline will report a success even though a test failed)

The Azure DevOps PowerShell task is below:

  - powershell: |                  
      $ARMTemplatePath = $env:templateFilePath      
      if((Test-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk) -eq $false){
        git clone --quiet $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk
      import-module $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk      
      try {
        Remove-Module Pester -ErrorAction SilentlyContinue
        Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
      catch {
          $errorMessage = $error[0]
          if ($errorMessage -like "*no valid module file was found*"){
              Install-Module Pester -AllowClobber -RequiredVersion 4.10.1 -Force -SkipPublisherCheck -AcceptLicense
              Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
          else {
              Write-Error -Message $errorMessage
      $results = Invoke-Pester -Script  @{ Path =  '.\Test.ARMTemplate.ps1'; Parameters = @{TemplatePath = ('.\' + $template)} } -OutputFormat NUnitXml -OutputFile $env:BUILD_ARTIFACTSTAGINGDIRECTORY\TEST-ARMTemplate.xml -PassThru
      if ($results.TestResult.Result -contains "Failed"){
        Write-Error -Message "Test Failed"
    failOnStderr: true     
    displayName: Run Integration Tests      
      templateFilePath: ${{parameters.templateFile}}

The final task we to need to do is publish the test results, which is an out of the box Azure pipeline task. The only caveat is that we need to ensure this task will always run even if the previous task fails. This ensures the test results are always published.

The Azure DevOps Publish Test Results task is below:

  - task: PublishTestResults@2
    displayName: Publish Test Results
    condition: always()
      testResultsFormat: NUnit
      searchFolder: $(Build.ArtifactStagingDirectory) 

That’s it we are done!

Now when we execute our test pipeline it will run all tests on our ARM template and publish the pipeline test results, example beloW.


Other Recent Blogs

Using Helm as a configuration manager for AKS

Helm is a package manager for Kubernetes that allows easier packaging, configuring and deployment of applications and services on your Kubernetes cluster, however it is not “only” a tool for application deployment. In this post I will discuss using it as a configuration and compliance management tool for AKS.

Read More »

Our Award Winning AI-Infused Solutions

Learn about the AI-infused solution we developed for Maurice Blackburn Lawyers, Australia’s leading social justice law firm. Leveraging Azure Cognitive Search and Services, the solution uses AI to review life insurance policies to assess rapidly whether a claim has merit and should be pursued. Our solution has helped the firm cut the time taken to assess a client’s entitlement to a superannuation disability insurance claim from months down to the span of the client’s initial phone. Our work with Maurice Blackburn Lawyers was recently featured across a range of Australian publications.

Read More »

Arinco PTY LTD
All Rights Reserved

Level 17, 303 Collins Street
Melbourne VIC 3000