Azure
Setup kubeadm K8s cluster on Azure provider.
You should have defined (or created) the following resources:
- Azure subscription;
- Resource Group (for all resources like VMs, disks, etc);
- VNET and a VNET Resource Group (if different from the VMs RG);
- Network Security Group.
All steps bellow describe the process to create a single-master and single-node cluster.
Keep in mind you must replace all resources' names with your own definitions.
We will need to run some AZ CLI command, to do so, make sure you are using your correct subscription during the entire process.
az account set --subscription "MY-SUBSCRIPTION"
az account list
You will see your subscription as "Default: true".
It will be used to configure the Load Balancer later.
az vm availability-set create `
--name MY-RESOURCE-GROUP `
--resource-group MY-RESOURCE-GROUP
You can also create a Scalability Set if you are going to use auto scaling or Application Gateway endpoints.
This IP will be used in the Load Balancer to connect to the master node.
Using Azure Portal, create a Public IP with the following options.
SKU: Basic
IP address assignment: Static
First of all, you have to get the ID of the subnet where the VMs will be created.
az network vnet list -g MY-VNET-RESOURCE-GROUP
Identify and get subnet ID. It will be something like:
/subscriptions/d66a4ccf-4568-41fc-8b2f-3d930a88486e/resourceGroups/MY-VNET-RESOURCE-GROUP/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/MY-SUBNET
Then, create your VMs.
az vm create `
--name vm-master-001 `
--resource-group MY-RESOURCE-GROUP `
--image UbuntuLTS `
--size Standard_F4s `
--authentication-type password `
--admin-username ambev `
--admin-password MySecretPWD! `
--nsg MY-NSG `
--nsg-rule SSH `
--os-disk-size-gb 100 `
--availability-set MY-AV `
--subnet /subscriptions/d66a4ccf-4568-41fc-8b2f-3d930a88486e/resourceGroups/MY-VNET-RESOURCE-GROUP/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/MY-SUBNET
Create one "master" and one "worker".
SSH to both your servers, master and worker.
Install docker dependencies.
sudo apt-get -y install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
Setup the repository.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
Install Docker.
sudo apt-get update
sudo apt-get -y install docker-ce=18.06.2~ce~3-0~ubuntu
Check with docker version you should install to support you K8s version:
https://kubernetes.io/docs/setup/production-environment/container-runtimes/#docker
Edit Docker service file.
nano /lib/systemd/system/docker.service
Edit Docker service file.
nano /lib/systemd/system/docker.service
Add the
--exec-opt native.cgroupdriver=systemd
config at the end of the line "ExecStart":ExecStart=/usr/bin/dockerd -H fd:// --exec-opt native.cgroupdriver=systemd
Restart services:
systemctl daemon-reload
systemctl restart docker
SSH to your master node.
Follow the instructions here: https://app.gitbook.com/@devops-buzz/s/public/kubeadm/cheat-sheet#install-kubeadm-kubectl-and-kubelet
Then get the packages versions and copy them. They will be used to install the worker node with exactly the same versions: https://app.gitbook.com/@devops-buzz/s/public/kubeadm/cheat-sheet#get-kubeadm-dependencies-versions
SSH to your worker node, and install the same version: https://app.gitbook.com/@devops-buzz/s/public/~/drafts/-LhMcXSw47Ajn24zlx5O/primary/kubeadm/cheat-sheet#specific-version
On Azure Portal, go to "Azure Active Directory", "App registrations".
Select "New registration".
Fill up the form.

Go to "All services", "Subscriptions" and select your subscription.

Go to "Access control (IAM), "Add", "Add role assignment".
Go to "Role" and "Assign access to". "Select" the application you created.
Go to "Azure Active Directory", "App registrations" and find your application.

Get the "Application (client) ID".

Go to "Certificates & secrets", "New client secret".

Fill up the form.

Copy and save the secret value.
The secret value will be displayed just once. Copy and save it now!
SSH to your master server.
Create kubeadm config file.
nano /etc/kubeadm.conf
With the following content.
apiServer:
certSANs:
- MY-SERVER-PUBLIC-IP
- MY-PUBLIC-IP-URL.com
- MY-CREATED-PUBLIC-IP
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: MY-SERVER-INTERNAL-IP:6443
controllerManager:
extraArgs:
cloud-config: /etc/kubernetes/cloud.conf
cloud-provider: azure
extraVolumes:
- hostPath: /etc/kubernetes/cloud.conf
mountPath: /etc/kubernetes/cloud.conf
name: cloud
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.240.0.0/16
scheduler: {}
Create cloud config file.
nano /etc/kubernetes/cloud.conf
With the following content.
{
"cloud":"AzurePublicCloud",
"tenantId": "Def04b19-7776-4a94-b45b-375c77a8f966",
"subscriptionId": "d44a4ccf-4578-87fc-8F2d-3d340a88486e",
"aadClientId": "CCef9bf7-b34e-69be-a1c9-d5be47b846d2",
"aadClientSecret": "s]ksASDFffaseWuT2MJsafdfdsf18OsdfJzc1",
"resourceGroup": "MY-RESOURCE-GROUP",
"location": "brazilsouth",
"vmType": "Standard_F4s",
"subnetName": "MY-SUBNET-NAME",
"securityGroupName": "MY-RESOURCE-GROUP",
"vnetName": "MY-VNET",
"vnetResourceGroup": "MY-VNET-RESOURCE-GROUP",
"routeTableName": "MY-RESOURCE-GROUP",
"primaryAvailabilitySetName": "MY-RESOURCE-GROUP",
"primaryScaleSetName": "",
"cloudProviderBackoff": true,
"cloudProviderBackoffRetries": 6,
"cloudProviderBackoffExponent": 1.5,
"cloudProviderBackoffDuration": 5,
"cloudProviderBackoffJitter": 1,
"cloudProviderRatelimit": true,
"cloudProviderRateLimitQPS": 3,
"cloudProviderRateLimitBucket": 10,
"useManagedIdentityExtension": false,
"userAssignedIdentityID": "",
"useInstanceMetadata": true,
"loadBalancerSku": "Basic",
"excludeMasterFromStandardLB": false,
"providerVaultName": "",
"maximumLoadBalancerRuleCount": 250,
"providerKeyName": "k8s",
"providerKeyVersion": ""
}
- To get the "tenantId", run
az account show
and get "tenantId" field. - To get the "subscriptionId", run
az account show
and get "id" field. - "aadClientId" is the "Application (client) ID" from Azure AD Application created above.
- "aadClientSecret" is the Application secret created above.
kubeadm init --config kubeadm.conf
The command above will return the "join" command, like:
kubeadm join YOUR-SERVER-INTERNAL-IP:6443 --token 2m8oey.aeejkbpfgpg0izi0 \
--discovery-token-ca-cert-hash sha256:bb4527127bbe01f049532c55f681dcf25ce31a1b4ff81797b615b2a247862e75
Save this command in safe place. It will be used to add nodes to your cluster.
SSH to your master node and run as root.
mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
SSH to your master node and run as root.
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
Azure does not support Calico.
Wait until the master node is "READY".
watch kubectl get pods --all-namespaces
SSH to your node and run the "join" command you got above.
kubectl label node \
MY-node-001 node-role.kubernetes.io/worker=worker
Create any service type "LoadBalancer". This will create create a Load Balancer on Azure called "kubernetes".
Create the manifest file.
nano service.yml
With the following content.
kind: Service
apiVersion: v1
metadata:
name: first-service
spec:
selector:
app: first-deployment
ports:
- protocol: TCP
port: 80
type: LoadBalancer
Deploy the service.
kubectl create -f service.yml
Go to Azure portal, find the Load Balancer "kuberntes" in your Resource Group.
Wait a few minutes. You should have a "Backend pool" called "kubernets" with your node in it.
Add a "Frontend IP Configuration" and select the public IP you created above.
Create a "Backend pool" add your master server.
Create a "health probe" for you master server on port 6443.
Create a "Load Balancing Rule" using your Public IP on port 6443 to your master pool.
At this point you should be able to use your kubeconfig your public DNS.
Go to Azure portal, find your VMs, go to "Networking", lick on the NIC, go to "IP configurations", click on the IP, on "Assignment", set it to "Static".
Find the VMs Public IPs and double check if they are "Static".
Last modified 3yr ago