Blogs

Azure MLOps Challenge Blog: Part 8

Hi there! πŸ‘‹

Welcome to the eighth and final entry of the Azure MLOps Challenge Blog! πŸŽ‰

In my previous post we finished the Azure MLOps Challenge “proper” and deployed our model to an online endpoint using GitHub Actions and validated it’s functionality with the Azure ML CLI. In this post we’ll complete a bonus module of my making to consume (inference) the Model using a functioning web app. πŸ₯³ We’ll use Github Actions to deploy Azure Web App infrastucture using Bicep then deploy a basic flask app using the Azure Web App Service Deploy action.

This is going to be a long one, so let’s get started! πŸš€

Bonus Challenge: Deploy a web app

The Objectives of the bonus challenge are as follows:

  • Create Azure resources using Bicep.
  • Deploy a web app using GitHub Actions.
  • Consume the model using the web app.

Note: As a general “theme” going forward, I’ve tried to keep the code as verbose as possible to make it easier to follow. This means that I’ve used a lot of hard coded values and comments in the code. In a production environment you would look to parameterise the values and move the parameters/variable to a paramater file or App Configuration store.

Create Azure resources using Bicep

We’ll use Bicep to create the Azure resources required to host our web app. Bicep is a Domain Specific Language (DSL) for deploying Azure resources declaratively. It’s a great alternative to ARM templates and is much easier to read and write.

To keep things simple we’ll deploy deploy the App Service Plan and App Service in the same resource group as our Azure Machine Learning workspace. In a production environment you would look to deploy the App Service Plan and App Service in a separate resource group, maybe even separate subscription and Landing Zones.

Create a new infra folder under src then create a new main.bicep file in the infra folder with the following code:

param location string = resourceGroup().location
param keyVaultName string = 'myworkspkeyvault43303873'

@description('Reference an existing Key Vault')
resource kv 'Microsoft.KeyVault/vaults@2023-02-01' existing = {
  name: keyVaultName
}

@description('Create an App Service Plan with a Basic SKU')
resource exampleAppServicePlan 'Microsoft.Web/serverfarms@2022-09-01' = {
  name: 'mlflow-ASP'
  location: location
  kind: 'Linux'
  properties: {
    reserved: true
  }
  sku: {
    tier: 'Basic'
    name: 'B1'
  }
}

@description('Create the App Service with a system-assigned identity')
resource exampleAppService 'Microsoft.Web/sites@2022-09-01' = {
  name: 'mlflow-appservice' // must be globally unique
  location: location
  kind: 'linux'
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    serverFarmId: exampleAppServicePlan.id
    reserved: true
    siteConfig: {
      linuxFxVersion: 'PYTHON|3.11'
    }
  }
}

@description('Configure the App Service with the Key Vault secrets')
resource exampleAppServiceConfig 'Microsoft.Web/sites/config@2022-09-01' = {
  name: 'web'
  parent: exampleAppService
  properties: {
    appSettings: [
      {
        name: 'API_KEY'
        value: '@Microsoft.KeyVault(SecretUri=https://${kv.name}.vault.azure.net/secrets/apiKey/)'
      }
      {
        name: 'AZURE_ML_ENDPOINT_URL'
        value: '@Microsoft.KeyVault(SecretUri=https://${kv.name}.vault.azure.net/secrets/endpointUrl/)'
      }
      {
        name: 'SCM_DO_BUILD_DURING_DEPLOYMENT'
        value: '1'
      }
    ]
  }
}

@description('This is the built-in Key Vault Secrets User role. See https://docs.microsoft.com/azure/role-based-access-control/built-in-roles')
resource keyVaultSecretsUser 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {
  name: '4633458b-17de-408a-b874-0445c86b69e6'
}

@description('Assign the Key Vault Secrets User role to the App Service')
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(kv.id, exampleAppService.id)
  properties: {
    principalId: exampleAppService.identity.principalId
    roleDefinitionId: keyVaultSecretsUser.id
    principalType: 'ServicePrincipal'
  }
  scope: kv
}

Okay, so there’s a lot going on here. Let’s break it down:

First we have some parameters. These are values that we can pass in when we deploy the Bicep template. We’re using the resourceGroup().location function to get the location of the resource group that the template is being deployed to. We’re also using a keyVaultName parameter to reference an existing Key Vault. We’ll use this Key Vault to store the API key and endpoint URL for our Azure Machine Learning workspace.

Next we have a resource declaration for the Key Vault. This is a special type of resource declaration that references an existing resource. We’re using this to reference the Key Vault created by the Azure Machine Learning workspace in the first part of the challenge.

Next we have a resource declaration for the App Service Plan. We’re using the properties property to set the reserved property to true. This will ensure that the App Service Plan is always available and won’t be scaled down to zero.

Next we have a resource declaration for the App Service. We’re using the identity property to create a system-assigned identity for the App Service. We’re also using the properties property to set the reserved property to true and the siteConfig property to set the linuxFxVersion property to PYTHON|3.11. This will ensure that the App Service is always available and will use Python 3.11.

Next we have a resource declaration for the App Service configuration. We’re using the properties property to set the appSettings property to an array of objects. Each object contains a name property and a value property. We’re using the @Microsoft.KeyVault function to reference the Key Vault and the SecretUri property to reference the API key and endpoint URL for the Azure Machine Learning workspace, these will all be created in the GitHub Actions workflow. We’re also using the SCM_DO_BUILD_DURING_DEPLOYMENT app setting to ensure that the App Service is not “re-zipped” during deployment. The GitHub Action will zip and deploy the web app code to the App Service.

Next we have a resource declaration for the Key Vault Secrets User role.

Finally we have a resource declaration for the role assignment. We’re using the properties property to set the principalId property to the Principal ID of the App Service identity, the roleDefinitionId property to the id property of the Key Vault Secrets User role. We’re using the scope property to set the scope to the id property of the Key Vault.

Create a workflow to deploy the Bicep template

Now, we’ll create a workflow to deploy the Bicep template. No fancy triggering methods this time ’round. We’ll just be triggering the workflow manually from the GitHub Portal.

Create a new deploy-infra.yml file in the .github/workflows folder with the following code:

on:
    workflow_dispatch:
  
permissions:
    id-token: write
    contents: read
  
env:
    resource-group: myResourceGroup # name of the Azure resource group
    ml-workspace: myWorkspace # name of the Azure Machine Learning workspace
    mlEndpointUrl: 'https://mlflow-endpoint-1.australiaeast.inference.ml.azure.com/score' # name of the Azure Machine Learning endpoint
    mlEndpointName: mlflow-endpoint-1 # name of the Azure Machine Learning endpoint
    keyVaultName: 'myworkspkeyvault43303873'
  
jobs:
    bicep-whatif:
      name: 'Bicep What-If'
      runs-on: ubuntu-latest
      steps:
        - name: Checkout code
          uses: actions/checkout@v3
  
        - name: 'Az CLI login'
          uses: azure/login@v1
          with:
            client-id: ${{ secrets.AZURE_CLIENT_ID }}
            tenant-id: ${{ secrets.AZURE_TENANT_ID }}
            subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
        
        - name: Install az ml extension
          run: az extension add -n ml -y

        - name: Retrieve API key
          run: |
            ENDPOINT_KEY=$(az ml online-endpoint get-credentials -n ${{ env.mlEndpointName }} -g ${{ env.resource-group }} -w ${{ env.ml-workspace }} -o tsv --query primaryKey)
            echo ::add-mask::$ENDPOINT_KEY
            echo ENDPOINT_KEY=$ENDPOINT_KEY >> $GITHUB_ENV
        
        - name: add secret to key vault
          run: |
            az keyvault secret set --vault-name ${{ env.keyVaultName }} --name 'apiKey'  --value ${{ env.ENDPOINT_KEY }}
            az keyvault secret set --vault-name ${{ env.keyVaultName }} --name 'endpointUrl' --value ${{ env.mlEndpointUrl }}
  
        # Run what-if deployment
        - name: What-If
          uses: azure/arm-deploy@v1
          with:
            scope: resourcegroup
            failOnStdErr: false
            resourceGroupName: ${{ env.resource-group }}
            template: src\infra\main.bicep
            additionalArguments: --what-if
  
    bicep-deploy:
      name: 'Bicep Deploy'
      runs-on: ubuntu-latest
      environment: prod
      needs: [bicep-whatif]
      
      steps:
        # Checkout the repository to the GitHub Actions runner
        - name: Checkout
          uses: actions/checkout@v3
  
        # Authenticate to Az CLI using OIDC
        - name: 'Az CLI login'
          uses: azure/login@v1
          with:
            client-id: ${{ secrets.AZURE_CLIENT_ID }}
            tenant-id: ${{ secrets.AZURE_TENANT_ID }}
            subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
        - name: Install az ml extension
          run: az extension add -n ml -y

        # Deploy
        - name: Deploy
          uses: azure/arm-deploy@v1
          with:
            scope: resourcegroup
            failOnStdErr: false
            resourceGroupName: ${{ env.resource-group }}
            template: src\infra\main.bicep

The main difference here is that we’re using the `–what-if` flag to run a what-if deployment. This will show us what changes will be made to our Azure resources without actually making any changes. Before that runs we’re using the `az ml online-endpoint get-credentials` command to get the API key for the Azure Machine Learning endpoint and the `az keyvault secret set` command to store the API key and endpoint URL in the Key Vault.

The pipeline will run a what-if deployment to show you what changes will be made to your Azure resources and then pause for manual approval. If you’re happy with the changes, you can approve the workflow and it will deploy the resources to Azure.

After deployment, we can validate that the API key and endpoint URL have been stored in the Key Vault and the App Service has been deployed and configured with the API key and endpoint URL.

Create a Flask web app

Next, create the following folder structure under `src`:

myapp/
β”œβ”€β”€ myapp/
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ app.py
β”‚   └── templates/
β”‚       β”œβ”€β”€ index.html
β”‚       └── result.html
β”œβ”€β”€ requirements.txt
└── web.config

myapp/ is the root directory of your Flask app.

myapp/myapp/ is the package directory that contains your Flask app code.

myapp/myapp/__init__.py is an empty file that tells Python that the myapp/myapp/ directory should be considered a Python package.

myapp/myapp/app.py is the file that contains your Flask app code.

myapp/myapp/templates/ is the directory that contains your HTML templates.

myapp/myapp/templates/index.html is the HTML file for the form where users can enter values for the specified items.

myapp/myapp/templates/result.html is the HTML file that displays the result of the diabetes prediction.

myapp/requirements.txt is a file that lists the Python packages required by your Flask app. For example, if your app uses Flask and requests, your requirements.txt file should contain:

Flask==2.2.5
requests==2.31.0

myapp/web.config is a configuration file required by Azure to run your Flask app. Here’s an example web.config file you can use:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="httpPlatformHandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" />
    </handlers>
    <httpPlatform processPath="%INTERPRETERPATH%" arguments="%INTERPRETER_ARGUMENTS%" stdoutLogEnabled="true" stdoutLogFile="\\?\%home%\LogFiles\python.log" startupTimeLimit="60" processesPerApplication="16">
      <environmentVariables>
        <environmentVariable name="PORT" value="%HTTP_PLATFORM_PORT%" />
        <environmentVariable name="PYTHONPATH" value="%HOME%\site\wwwroot" />
      </environmentVariables>
    </httpPlatform>
  </system.webServer>
</configuration>

The Azure Web App looks for app.py specifically to start Gunicorn. This file contains your Flask app code. The following code is an example of a Flask app that uses the Flask web framework to create a web app that accepts user input, sends the input to the deployed model, and displays the result. Azure provides sample code to query the model endpoint in the “Consume” tab of the model endpoint in the Azure ML portal.

from flask import Flask, request, render_template
import json
import os
import ssl
import urllib.request

app = Flask(__name__)


def allowSelfSignedHttps(allowed):
    # bypass the server certificate verification on client side
    if (
        allowed
        and not os.environ.get("PYTHONHTTPSVERIFY", "")
        and getattr(ssl, "_create_unverified_context", None)
    ):
        ssl._create_default_https_context = ssl._create_unverified_context


allowSelfSignedHttps(True)
# this line is needed if you use self-signed certificate
# in your scoring service.


@app.route("/", methods=["GET", "POST"])
def index():
    if request.method == "POST":
        # Get user input
        patient_id = request.form["PatientID"]
        pregnancies = request.form["Pregnancies"]
        plasma_glucose = request.form["PlasmaGlucose"]
        diastolic_blood_pressure = request.form["DiastolicBloodPressure"]
        triceps_thickness = request.form["TricepsThickness"]
        serum_insulin = request.form["SerumInsulin"]
        bmi = request.form["BMI"]
        diabetes_pedigree = request.form["DiabetesPedigree"]
        age = request.form["Age"]

        # Format input data
        input_data = {
            "input_data": {
                "columns": [
                    "PatientID",
                    "Pregnancies",
                    "PlasmaGlucose",
                    "DiastolicBloodPressure",
                    "TricepsThickness",
                    "SerumInsulin",
                    "BMI",
                    "DiabetesPedigree",
                    "Age",
                ],
                "index": [0],
                "data": [
                    [
                        patient_id,
                        pregnancies,
                        plasma_glucose,
                        diastolic_blood_pressure,
                        triceps_thickness,
                        serum_insulin,
                        bmi,
                        diabetes_pedigree,
                        age,
                    ]
                ],
            }
        }

        body = str.encode(json.dumps(input_data))
        # Get the AZURE_ML_ENDPOINT_URL environment variable passed from App Service Application settings
        url = os.environ["AZURE_ML_ENDPOINT_URL"]
        # Get the API_KEY environment variable passed from App Service Application settings
        api_key = os.environ["API_KEY"]
        if not api_key:
            raise Exception("A key should be provided to invoke the endpoint")

        # The azureml-model-deployment header will force the request
        # to go to a specific deployment.
        # Remove this header to have the request observe the endpoint
        # traffic rules
        headers = {
            "Content-Type": "application/json",
            "Authorization": ("Bearer " + api_key),
            "azureml-model-deployment": "mlflow-deployment",
        }

        req = urllib.request.Request(url, body, headers)

        try:
            response = urllib.request.urlopen(req)

            result = json.loads(response.read())

            # Display result
            if result[0] == 1:
                return render_template("result.html", result="The patient is diabetic.")
            else:
                return render_template(
                    "result.html", result="The patient is not diabetic."
                )
        except urllib.error.HTTPError as error:
            print("The request failed with status code: " + str(error.code))

            # Print the headers - they include the requert ID and the
            # timestamp, which are useful for debugging the failure
            print(error.info())
            print(error.read().decode("utf8", "ignore"))
    else:
        return render_template("index.html")


if __name__ == "__main__":
    app.run()

index.html is the HTML file for the form where users can enter values for the specified items. The following code is an example of an index.html file that contains a form with the specified items:

<!DOCTYPE html>
<html>
    <head>
        <title>Diabetes Prediction</title>
    </head>
    <body>
        <h1>Diabetes Prediction</h1>
        <form method="post">
            <label for="PatientID">Patient ID:</label><br>
            <input type="text" id="PatientID" name="PatientID"><br><br>
            <label for="Pregnancies">Pregnancies:</label><br>
            <input type="text" id="Pregnancies" name="Pregnancies"><br><br>
            <label for="PlasmaGlucose">Plasma Glucose:</label><br>
            <input type="text" id="PlasmaGlucose" name="PlasmaGlucose"><br><br>
            <label for="DiastolicBloodPressure">Diastolic Blood Pressure:</label><br>
            <input type="text" id="DiastolicBloodPressure" name="DiastolicBloodPressure"><br><br>
            <label for="TricepsThickness">Triceps Thickness:</label><br>
            <input type="text" id="TricepsThickness" name="TricepsThickness"><br><br>
            <label for="SerumInsulin">Serum Insulin:</label><br>
            <input type="text" id="SerumInsulin" name="SerumInsulin"><br><br>
            <label for="BMI">BMI:</label><br>
            <input type="text" id="BMI" name="BMI"><br><br>
            <label for="DiabetesPedigree">Diabetes Pedigree:</label><br>
            <input type="text" id="DiabetesPedigree" name="DiabetesPedigree"><br><br>
            <label for="Age">Age:</label><br>
            <input type="text" id="Age" name="Age"><br><br>
            <input type="submit" value="Submit">
        </form> 
    </body>
</html>

result.html is the HTML file that displays the result of the diabetes prediction. The following code is an example of a result.html file that displays the result of the diabetes prediction:

<!DOCTYPE html>
<html>
    <head>
        <title>Diabetes Prediction Result</title>
    </head>
    <body>
        <h1>Diabetes Prediction Result</h1>
        <p>{{ result }}</p>
    </body>
</html>

Create a workflow to deploy the Flask app

Now create a workflow to deploy the Flask app. Create a new deploy-webapp.yml file in the .github/workflows folder with the following code:

on:
  workflow_dispatch:

permissions:
  id-token: write
  contents: read

env:
  AZURE_WEBAPP_NAME: mlflow-appservice # set this to your web application's name
  AZURE_WEBAPP_PACKAGE_PATH: 'src/web/myapp' # set this to the path to your web app project, defaults to the repository root

jobs:
  build:
    environment:
      name: 'prod'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: azure/login@v1
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Set up Python 3.x
        uses: actions/setup-python@v4
        with:
          python-version: 3.x
      - name: Install dependencies
        run: |
          pip install -r src/web/myapp/requirements.txt
      - name: Deploy App
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          package: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}
      - name: logout
        run: |
          az logout

This workflow will deploy the Flask app to the App Service using the webapps-deploy action. The webapps-deploy action will zip the Flask app code and deploy it to the App Service.

After committing and pushing the code to GitHub, manually run the workflow to deploy the Flask app to the App Service.

We’re not going to win any awards for style here 🀣

but it works! πŸŽ‰

MLOps

Conclusion

And that’s it! We’ve successfully deployed a working Web App that consumes our model. πŸ₯³ Not only that but we’ve also created a CI/CD pipeline to store secrets in a Key Vault and used a managed identity to authenticate the web app to the Key Vault, completely removing any human interaction in the secret handling process. πŸŽ‰

This is the simplest minimum viable product (MVP) that I could come up with to demonstrate the functionality of consuming the model. For example you could:

  • Add authentication to the web app.
  • Add a database to the web app to store the results of the predictions.
  • Develop a mobile app that consumes the model.
  • Instead of having a user input the values for the specified items, you could use a camera to take a picture of the specified items and use Azure Cognitive Services to extract the values from the picture and send them to the deployed model.
  • The possibilities are endless! 🀯

I hope you’ve enjoyed this series of blog posts and learned something along the way. I know I have! πŸ€“

If you have any questions or feedback, please feel free to reach out to me on LinkedIn

Thanks for reading! πŸ‘‹

[mailpoet_form id="1"]

Other Recent Blogs

Level 9, 360 Collins Street,Β 
Melbourne VIC 3000

Level 2, 24 Campbell St,
Sydney NSW 2000

200 Adelaide St,
Brisbane QLD 4000

191 St Georges Terrace
Perth WA 6000

Level 10, 41 Shortland Street
Auckland

Part of

Arinco trades as Arinco (VIC) Pty Ltd and Arinco (NSW) Pty Ltd. Β© 2023 All Rights Reserved Arincoβ„’ | Privacy Policy
Arinco acknowledges the Traditional Owners of the land on which our offices are situated, and pay our respects to their Elders past, present and emerging.

Get started on the right path to cloud success today. Our Crew are standing by to answer your questions and get you up and running.