Helm is a package manager for Kubernetes that allows easier packaging, configuring and deployment of applications and services on your Kubernetes cluster, however it is not “only” a tool for application deployment. In this post I will discuss using it as a configuration and compliance management tool for AKS.
It is good practice to manage items such as resource quotas and limits and role bindings at the AKS Namespace level, with the namespace being the virtual cluster within the physical cluster used to group application or environment specific resources.
Today, we will look at how to create a new namespace using Helm that configures the below for us:
- CPU and memory limits
- CPU and memory quotas
- RBAC rolebindings for namespace access
More information on Kubernetes limits and quotas can be found here.
Prerequisites
The requirements for this exercise are:
- Create an AKS Administrators AAD Group and add your user ID to this group
- The above groups Object ID
- Azure CLI installed on your workstation
- kubectl installed on your workstation
- Helm v3 installed on your workstation
- An AKS Cluster with RBAC and AKS Managed Azure Active Directory enabled
Process
Now that the requirements are ready, we can create our Helm charts for our namespace creation and configuration. Browse to a local folder on your workstation where you want to store the Helm charts, I am using a folder called ‘Charts’.
Usually we would use the Helm create command to create the Helm chart, however in this instance we will manually create the folders/files as we do not need the standard structure that is created.
Folder Structure
Create the below folder/file structure on your local workstation, for this demo I am using C:\Charts\namespace-create\.
Note: In a real-world scenario you would have these files and folders checked in to source control.
Helm Files
Add the following contents to each file. As these are YAML files be careful with your indentation.
.helmignore
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
Chart.yaml
apiVersion: v2
name: namespace-create
description: A Helm chart used to scaffold the creation and configuration of a namespace with foundational elements
type: application
version: 0.1.0
appVersion: 0.1.0
values.yaml
enableBuiltinClusterRoleBindings: true
adminGroupId: [AKS Admins AAD Group Object ID] # Replace this value with your AKS Admin AAD Group Object ID – xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
# Resource Quotas for Namespace
enableResourceQuota: true
requestCpuQuota: 2
requestMemoryQuota: 2Gi
limitCpuQuota: 5
limitMemoryQuota: 5Gi
# Default Limit Ranges for Pods in Namespace
enableDefaultLimitRange: true
defaultMemRequest: 128Mi
defaultMemLimit: 512Mi
defaultCpuRequest: 0.5
defaultCpuLimit: 1.0
limitRange.yaml
apiVersion: v1
kind: LimitRange
metadata:
name: namespace-default-limit-range
spec:
limits:
- default:
memory: {{ .Values.defaultMemLimit }}
cpu: {{ .Values.defaultCpuLimit }}
defaultRequest:
memory: {{ .Values.defaultMemRequest }}
cpu: {{ .Values.defaultCpuRequest }}
type: Container
resourceQuota.yaml
{{- if .Values.enableResourceQuota -}}
kind: ResourceQuota
apiVersion: v1
metadata:
name: namespace-resourcequota
namespace: {{ .Release.Namespace }}
spec:
hard:
requests.cpu: {{ .Values.requestCpuQuota }}
requests.memory: {{ .Values.requestMemoryQuota }}
limits.cpu: {{ .Values.limitCpuQuota }}
limits.memory: {{ .Values.limitMemoryQuota }}
{{- end }}
roleBinding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: namespace-admin
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- kind: Group
namespace: {{ .Release.Namespace }}
name: {{ .Values.adminGroupId }}
Let’s take a quick look at what each of these files is used for.
- .helmignore
Is used to specify files you don’t want to include in your helm chart. - Chart.yaml
This file is required and includes the API version, version, application version and name of the chart. - values.yaml
This file is used to pass values into the chart. The values are a key value pair like below.favouriteDrink: coffeenameIn our chart files we can then access these values using the below.data:
myvalue: "Hello World"
drink: {{ .Values.favoriteDrink }} - limitRange.yaml
This file sets our limit values for CPU and memory as well as request limits for CPU and memory for each pod that is deployed to the namespace. - resourceQuota.yaml
This file sets our resource quota limits for CPU and memory as well as request limits for CPU and memory for each pod that is deployed to the namespace. - roleBinding.yaml
This file configures the role bindings for namespace RBAC and assigns the AAD Group accordingly. In this example we are only setting the Admin role.
Connect to your Cluster
Now we need to set our context to our cluster in the .kube/config file. We can do this by entering the below command, replacing the values with ones relevant for your environment.
az aks get-credentials --name --resource-group
You will see a message like below.
Merged "aks-blog-cluster" as current context in C:\Users\gary_\.kube\config
Now we can run commands on our AKS Cluster, lets run the below command to see the existing namespaces. As this is the first time connecting you may be prompted to login.
kubectl get namespace
If your cluster is new and default you should see the below for the namespace list.
NAME STATUS AGE
default Active 3h24m
kube-node-lease Active 3h24m
kube-public Active 3h24m
kube-system Active 3h24m
Make sure that your command line session is set to the directory that is storing your Helm chart file, for me this is C:\Charts\.
Apply the config with Helm Install
Now we can create our new namespace with our resource limits, quotas and role bindings configured automatically using the below command.
helm install {releasename} {chart folder} --namespace {namespace} --create-namespace
For my example this is.
helm install aks-blog namespace-create --namespace demo-namespace --create-namespace
If successful you will see the below.
NAME: aks-blog
LAST DEPLOYED: Wed Oct 21 13:20:33 2020
NAMESPACE: demo-namespace
STATUS: deployed
REVISION: 1
TEST SUITE: None
Now let run our kubectl get namespace command again.
NAME STATUS AGE
default Active 3h24m
demo-namespace Active 3h24m
kube-public Active 3h24m
kube-system Active 3h24m
We now have our new demo-namespace, we can check that our configuration has applied by running the below command.
kubectl get limitrange,ResourceQuota,roleBindings --namespace demo-namespace
We should see our namespace limits and quotas along with our rolebinding.
NAME AGE REQUEST resourcequota/namespace-resourcequota 15m requests.cpu: 0/2, requests.memory: 0/2Gi Summary
This post has shown how to effectively use helm as a Kubernetes manifest engine and provides a structured, repeatable method to create namespaces with pre-determined configurations.
If you would like to know more feel free to reach out to discuss our AKS Foundations Accelerator offering.
