Blogs

Share on facebook
Share on twitter
Share on linkedin
Share on email

Using Helm as a configuration manager for AKS

Helm is a package manager for Kubernetes that allows easier packaging, configuring and deployment of applications and services on your Kubernetes cluster, however it is not “only” a tool for application deployment. In this post I will discuss using it as a configuration and compliance management tool for AKS.

It is good practice to manage items such as resource quotas and limits and role bindings at the AKS Namespace level, with the namespace being the virtual cluster within the physical cluster used to group application or environment specific resources. 

Today, we will look at how to create a new namespace using Helm that configures the below for us:

  • CPU and memory limits
  • CPU and memory quotas
  • RBAC rolebindings for namespace access

More information on Kubernetes limits and quotas can be found here.

Prerequisites

The requirements for this exercise are:

  • Create an AKS Administrators AAD Group and add your user ID to this group
  • The above groups Object ID
  • Azure CLI installed on your workstation
  • kubectl installed on your workstation
  • Helm v3 installed on your workstation
  • An AKS Cluster with RBAC and AKS Managed Azure Active Directory enabled
Process

Now that the requirements are ready, we can create our Helm charts for our namespace creation and configuration. Browse to a local folder on your workstation where you want to store the Helm charts, I am using a folder called ‘Charts’.

Usually we would use the Helm create command to create the Helm chart, however in this instance we will manually create the folders/files as we do not need the standard structure that is created.

Folder Structure

Create the below folder/file structure on your local workstation, for this demo I am using C:\Charts\namespace-create\.
Note: In a real-world scenario you would have these files and folders checked in to source control.

Helm Files

Add the following contents to each file. As these are YAML files be careful with your indentation.

.helmignore

# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

 Chart.yaml

apiVersion: v2
name: namespace-create
description: A Helm chart used to scaffold the creation and configuration of a namespace with foundational elements
type: application
version: 0.1.0
appVersion: 0.1.0

values.yaml

enableBuiltinClusterRoleBindings: true
adminGroupId: [AKS Admins AAD Group Object ID] # Replace this value with your AKS Admin AAD Group Object ID – xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
# Resource Quotas for Namespace
enableResourceQuota: true
requestCpuQuota: 2
requestMemoryQuota: 2Gi
limitCpuQuota: 5
limitMemoryQuota: 5Gi
# Default Limit Ranges for Pods in Namespace
enableDefaultLimitRange: true
defaultMemRequest: 128Mi
defaultMemLimit: 512Mi
defaultCpuRequest: 0.5
defaultCpuLimit: 1.0

limitRange.yaml

apiVersion: v1
kind: LimitRange
metadata:
name: namespace-default-limit-range
spec:
limits:
- default:
memory: {{ .Values.defaultMemLimit }}
cpu: {{ .Values.defaultCpuLimit }}
defaultRequest:
memory: {{ .Values.defaultMemRequest }}
cpu: {{ .Values.defaultCpuRequest }}
type: Container

resourceQuota.yaml

{{- if .Values.enableResourceQuota -}}
kind: ResourceQuota
apiVersion: v1
metadata:
name: namespace-resourcequota
namespace: {{ .Release.Namespace }}
spec:
hard:
requests.cpu: {{ .Values.requestCpuQuota }}
requests.memory: {{ .Values.requestMemoryQuota }}
limits.cpu: {{ .Values.limitCpuQuota }}
limits.memory: {{ .Values.limitMemoryQuota }}
{{- end }}

roleBinding.yaml

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: namespace-admin
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- kind: Group
namespace: {{ .Release.Namespace }}
name: {{ .Values.adminGroupId }}

Let’s take a quick look at what each of these files is used for.

  • .helmignore 
    Is used to specify files you don’t want to include in your helm chart.
  • Chart.yaml
    This file is required and includes the API version, version, application version and name of the chart.
  • values.yaml
    This file is used to pass values into the chart. The values are a key value pair like below.
    favouriteDrink: coffeename
    In our chart files we can then access these values using the below.
    data:
    myvalue: "Hello World"
    drink: {{ .Values.favoriteDrink }}
  • limitRange.yaml
    This file sets our limit values for CPU and memory as well as request limits for CPU and memory for each pod that is deployed to the namespace.
  • resourceQuota.yaml
    This file sets our resource quota limits for CPU and memory as well as request limits for CPU and memory for each pod that is deployed to the namespace.
  • roleBinding.yaml
    This file configures the role bindings for namespace RBAC and assigns the AAD Group accordingly. In this example we are only setting the Admin role.
 
Connect to your Cluster

Now we need to set our context to our cluster in the .kube/config file. We can do this by entering the below command, replacing the values with ones relevant for your environment.

az aks get-credentials --name --resource-group

You will see a message like below.

Merged "aks-blog-cluster" as current context in C:\Users\gary_\.kube\config

Now we can run commands on our AKS Cluster, lets run the below command to see the existing namespaces. As this is the first time connecting you may be prompted to login.

kubectl get namespace

If your cluster is new and default you should see the below for the namespace list.

NAME            STATUS      AGE
default         Active      3h24m
kube-node-lease Active      3h24m
kube-public     Active      3h24m
kube-system     Active      3h24m

Make sure that your command line session is set to the directory that is storing your Helm chart file, for me this is C:\Charts\.

 
Apply the config with Helm Install

Now we can create our new namespace with our resource limits, quotas and role bindings configured automatically using the below command.

helm install {releasename} {chart folder} --namespace {namespace} --create-namespace

For my example this is.

helm install aks-blog namespace-create --namespace demo-namespace --create-namespace

If successful you will see the below.

NAME: aks-blog
LAST DEPLOYED: Wed Oct 21 13:20:33 2020
NAMESPACE: demo-namespace
STATUS: deployed
REVISION: 1
TEST SUITE: None

Now let run our kubectl get namespace command again.

NAME            STATUS      AGE
default         Active      3h24m
demo-namespace  Active      3h24m
kube-public     Active      3h24m
kube-system     Active      3h24m

We now have our new demo-namespace, we can check that our configuration has applied by running the below command.

kubectl get limitrange,ResourceQuota,roleBindings --namespace demo-namespace

We should see our namespace limits and quotas along with our rolebinding.

NAME                                   AGE         REQUEST                               resourcequota/namespace-resourcequota  15m         requests.cpu: 0/2, requests.memory: 0/2Gi  
Summary

This post has shown how to effectively use helm as a Kubernetes manifest engine and provides a structured, repeatable method to create namespaces with pre-determined configurations.

If you would like to know more feel free to reach out to discuss our AKS Foundations Accelerator offering.

 

Subscribe

Other Recent Blogs

Using Helm as a configuration manager for AKS

Helm is a package manager for Kubernetes that allows easier packaging, configuring and deployment of applications and services on your Kubernetes cluster, however it is not “only” a tool for application deployment. In this post I will discuss using it as a configuration and compliance management tool for AKS.

Read More »

Our Award Winning AI-Infused Solutions

Learn about the AI-infused solution we developed for Maurice Blackburn Lawyers, Australia’s leading social justice law firm. Leveraging Azure Cognitive Search and Services, the solution uses AI to review life insurance policies to assess rapidly whether a claim has merit and should be pursued. Our solution has helped the firm cut the time taken to assess a client’s entitlement to a superannuation disability insurance claim from months down to the span of the client’s initial phone. Our work with Maurice Blackburn Lawyers was recently featured across a range of Australian publications.

Read More »

Arinco PTY LTD
All Rights Reserved

Level 17, 303 Collins Street
Melbourne VIC 3000