In my previous post we went through how to deploy ARM templates with Azure DevOps using GitHub Flow. Today we are expanding on that and going through some tips and gotchas around adding the ARM Test Toolkit for testing and passing artifacts between stages.
ARM Test Toolkit
When we setup our Azure Pipeline Template we added a simple ARM template validation test. While this is a good start we should be expanding on this and adding some more comprehensive tests. A good place to start for testing ARM templates is the ARM Test Toolkit. It is a PowerShell module that provides a series of test cases to run against your ARM templates and also allows you to add your own.
The only caveat is the PowerShell module is not currently available from the PowerShellGallery and you will need to download the module and manage it yourself.
Below is the changes i made to incorporate the ARM Test Toolkit.
- Download the module and stored it in my repo (in my case I chose to store the module in a directory called “modules”)
- Updated my Azure Pipeline Template to run all tests from the ARM Test Toolkit
parameters:
serviceConnection: ""
resourceGroup: ""
location: ""
templateFile: ""
templateParametersFile: ""
overrideParameters: ""
deploymentMode: Validation
stages:
- stage: build
displayName: Build
jobs:
- job: build
displayName: Build
pool:
vmimage: windows-2019
steps:
- task: AzureResourceGroupDeployment@2
displayName: 'Validate ARM Template'
inputs:
azureSubscription: '${{parameters.serviceConnection}}'
resourceGroupName: '${{parameters.resourceGroup}}'
location: '${{parameters.location}}'
csmFile: '${{parameters.templateFile}}'
csmParametersFile: '${{parameters.templateParametersFile}}'
overrideParameters: '${{parameters.overrideParameters}}'
deploymentMode: '${{parameters.deploymentMode}}'
- task: PowerShell@1
displayName: 'Run Integration Tests'
inputs:
scriptType: inlineScript
failOnStderr: true
arguments: '-template ${{parameters.templateFile}}'
inlineScript: |
param(
[string]$template
)
$ErrorActionPreference = 'Stop'
import-module .\modules\arm-ttk\arm-ttk.psd1
Test-AzTemplate -TemplatePath .\$template
- task: PowerShell@1
displayName: 'Get ARM Template directory'
inputs:
scriptType: inlineScript
arguments: '-directory ${{parameters.templateFile}}'
inlineScript: |
param(
[string]$directory
)
$date = get-date -Format yyyy-MM-dd-hhmmss
$templateDirectory = $directory.Substring(0,$directory.LastIndexOf("/"))
write-output ("##vso[task.setvariable variable=templateDirectory;]$templateDirectory")
write-output ("##vso[task.setvariable variable=artifactDate;]$date")
- task: PublishPipelineArtifact@1
displayName: 'Publish Pipeline Artifact'
inputs:
path: $(templateDirectory)
artifact: drop_$(artifactDate)Done! Now when my StorageAccounts-CI pipeline is kicked off a lot more tests are run.
Tips and Gotchas
One gotcha i ran into is that there could be a situation where a deployment of my template has been kicked off and is awaiting approval to deploy my production environment. In the meantime someone else has committed and merged a change, if i then approve the deployment to production it will use the latest artifact resulting in different code deployed to my environments.
So how do we ensure that our deployments to each environment use the same artifacts? The answer is simple, specify the artifact to download and use throughout each stage of the deployment.
While sounds easy its a bit more complex to execute, it will involve downloading the latest artifact, retrieving the artifact name and then storing the artifact name in a variable for each stage to reference. While this still may not seem overly complex to you here comes the next gotcha, you cannot share variables between stages in Azure DevOps (at the time of writing this article). Although we currently cannot share variables between stages there is a workaround, which is to store the variable in a file and download the file in each stage of your CD pipeline. Its not pretty but it works.
Below is the updates i made to my CD pipeline;
- Added a “Build” stage to;Download the latest artifact
- Get the latest artifact name and store the name of the artifact it in a file
- Publish the file as an artifact so it will be available to download in other stages
NOTE I chose to store my file in the Build.ArtifactStagingDirectory as this directory is purged before each new build, so you don’t have to clean it up yourself.
stages:
- stage: Build
jobs:
- job: build
displayName: Build
pool:
vmimage: windows-2019
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
- powershell: |
$dir = '$(Build.ArtifactStagingDirectory)\variables\'
$file = "sharedvariables.json"
$artifactname = (get-item $(Pipeline.Workspace)\drop_*).name
new-item $dir -itemType directory -force | out-null
new-item ($dir + $file) -itemtype file -force | out-null
set-content ($dir + $file) ('{ "latestArtifactName": "' + $artifactname + '" }') -force
displayName: 'Create Shared Variables'
- task: PublishPipelineArtifact@1
displayName: 'Publish Shared Variables'
inputs:
path: $(Build.ArtifactStagingDirectory)/variables
artifact: shared_variables- Updated each stage to;Download the file from Build.ArtifactStagingDirectory
- Read the file and get the artifact name
- Download the artifact using the name I got from my file
- Deploy my ARM Template
- stage: Development
jobs:
- deployment: storage_accounts_dev
displayName: Deploy Storage Accounts
pool:
vmimage: windows-2019
environment: Development
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Shared Variables'
inputs:
artifactName: shared_variables
targetPath: $(Build.ArtifactStagingDirectory)
- powershell: |
$vars = Get-Content '$(Build.ArtifactStagingDirectory)\*' | ConvertFrom-Json
$artifactname = $vars.latestArtifactName
write-output ("##vso[task.setvariable variable=artifactName;]$artifactname")
displayName: 'Define Deployment Artifact'
- task: DownloadPipelineArtifact@2
displayName: 'Download Latest Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
artifactName: $(artifactName)
targetPath: $(Build.ArtifactStagingDirectory)/drop
- task: AzureResourceGroupDeployment@2
displayName: Deploy Dev Diagnostic Storage Account
inputs:
azureSubscription: "$(serviceConnectionNpd)"
resourceGroupName: "$(rgDiagNpd)"
location: "$(primaryLoc)"
csmFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.json'
csmParametersFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.parameters.json'
overrideParameters: '-storageName "$(diagNpdStorageAccount)" -accountType "Standard_LRS" -accessTier "Hot" -kind "StorageV2" -softDeleteRetentionDays 7'
deploymentMode: ValidationMy entire CD pipeline now looks like this and will ensure deployments to each stage use the same artifacts.
trigger:
branches:
include:
- master
paths:
include:
- New-StorageAccount/*
name: StorageAccounts-CD
variables:
- group: Variables - Storage
stages:
- stage: Build
jobs:
- job: build
displayName: Build
pool:
vmimage: windows-2019
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
- powershell: |
$dir = '$(Build.ArtifactStagingDirectory)\variables\'
$file = "sharedvariables.json"
$artifactname = (get-item $(Pipeline.Workspace)\drop_*).name
new-item $dir -itemType directory -force | out-null
new-item ($dir + $file) -itemtype file -force | out-null
set-content ($dir + $file) ('{ "latestArtifactName": "' + $artifactname + '" }') -force
displayName: 'Create Shared Variables'
- task: PublishPipelineArtifact@1
displayName: 'Publish Shared Variables'
inputs:
path: $(Build.ArtifactStagingDirectory)/variables
artifact: shared_variables
- stage: Development
jobs:
- deployment: storage_accounts_dev
displayName: Deploy Storage Accounts
pool:
vmimage: windows-2019
environment: Development
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Shared Variables'
inputs:
artifactName: shared_variables
targetPath: $(Build.ArtifactStagingDirectory)
- powershell: |
$vars = Get-Content '$(Build.ArtifactStagingDirectory)\*' | ConvertFrom-Json
$artifactname = $vars.latestArtifactName
write-output ("##vso[task.setvariable variable=artifactName;]$artifactname")
displayName: 'Define Deployment Artifact'
- task: DownloadPipelineArtifact@2
displayName: 'Download Latest Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
artifactName: $(artifactName)
targetPath: $(Build.ArtifactStagingDirectory)/drop
- task: AzureResourceGroupDeployment@2
displayName: Deploy Dev Diagnostic Storage Account
inputs:
azureSubscription: "$(serviceConnectionNpd)"
resourceGroupName: "$(rgDiagNpd)"
location: "$(primaryLoc)"
csmFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.json'
csmParametersFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.parameters.json'
overrideParameters: '-storageName "$(diagNpdStorageAccount)" -accountType "Standard_LRS" -accessTier "Hot" -kind "StorageV2" -softDeleteRetentionDays 7'
deploymentMode: Validation
- stage: NonProduction
jobs:
- deployment: storage_accounts_npd
displayName: Deploy Storage Accounts
pool:
vmimage: windows-2019
environment: Non-Production
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Shared Variables'
inputs:
artifactName: shared_variables
targetPath: $(Build.ArtifactStagingDirectory)
- powershell: |
$vars = Get-Content '$(Build.ArtifactStagingDirectory)\*' | ConvertFrom-Json
$artifactname = $vars.latestArtifactName
write-output ("##vso[task.setvariable variable=artifactName;]$artifactname")
displayName: 'Define Deployment Artifact'
- task: DownloadPipelineArtifact@2
displayName: 'Download Latest Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
artifactName: $(artifactName)
targetPath: $(Build.ArtifactStagingDirectory)/drop
- task: AzureResourceGroupDeployment@2
displayName: Deploy NonProd Diagnostic Storage Account
inputs:
azureSubscription: "$(serviceConnectionNpd)"
resourceGroupName: "$(rgDiagNpd)"
location: "$(primaryLoc)"
csmFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.json'
csmParametersFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.parameters.json'
overrideParameters: '-storageName "$(diagNpdStorageAccount)" -accountType "Standard_LRS" -accessTier "Hot" -kind "StorageV2" -softDeleteRetentionDays 7'
deploymentMode: Incremental
- stage: Production
jobs:
- deployment: storage_accounts_prd
displayName: Deploy Storage Accounts
pool:
vmimage: windows-2019
environment: Production
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Shared Variables'
inputs:
artifactName: shared_variables
targetPath: $(Build.ArtifactStagingDirectory)
- powershell: |
$vars = Get-Content '$(Build.ArtifactStagingDirectory)\*' | ConvertFrom-Json
$artifactname = $vars.latestArtifactName
write-output ("##vso[task.setvariable variable=artifactName;]$artifactname")
displayName: 'Define Deployment Artifact'
- task: DownloadPipelineArtifact@2
displayName: 'Download Latest Artifact'
inputs:
buildType: specific
project: AzureFoundations
definition: StorageAccounts-CI
artifactName: $(artifactName)
targetPath: $(Build.ArtifactStagingDirectory)/drop
- task: AzureResourceGroupDeployment@2
displayName: Deploy Prod Diagnostic Storage Account
inputs:
azureSubscription: "$(serviceConnectionPrd)"
resourceGroupName: "$(rgDiagPrd)"
location: "$(primaryLoc)"
csmFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.json'
csmParametersFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.parameters.json'
overrideParameters: '-storageName "$(diagPrdStorageAccount)" -accountType "Standard_LRS" -accessTier "Hot" -kind "StorageV2" -softDeleteRetentionDays 7'
deploymentMode: Incremental
- task: AzureResourceGroupDeployment@2
displayName: Deploy Shared Diagnostic Storage Account
inputs:
azureSubscription: "$(serviceConnectionPrd)"
resourceGroupName: "$(rgDiagShd)"
location: "$(primaryLoc)"
csmFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.json'
csmParametersFile: '$(Build.ArtifactStagingDirectory)/drop/New.StorageAccount.parameters.json'
overrideParameters: '-storageName "$(diagShdStorageAccount)" -accountType "Standard_LRS" -accessTier "Hot" -kind "StorageV2" -softDeleteRetentionDays 7'
deploymentMode: Incremental 
