Azure

Setup kubeadm K8s cluster on Azure provider.
Requirements
You should have defined (or created) the following resources:
Azure subscription;
Resource Group (for all resources like VMs, disks, etc);
VNET and a VNET Resource Group (if different from the VMs RG);
Network Security Group.
All steps bellow describe the process to create a single-master and single-node cluster.
Keep in mind you must replace all resources' names with your own definitions.
Select your Azure subscription
We will need to run some AZ CLI command, to do so, make sure you are using your correct subscription during the entire process.
az account set --subscription "MY-SUBSCRIPTION"
az account list
You will see your subscription as "Default: true".
Create an Availability Set
It will be used to configure the Load Balancer later.
az vm availability-set create `
  --name MY-RESOURCE-GROUP `
  --resource-group MY-RESOURCE-GROUP
You can also create a Scalability Set if you are going to use auto scaling or Application Gateway endpoints.
Create a Public IP
This IP will be used in the Load Balancer to connect to the master node.
Using Azure Portal, create a Public IP with the following options.
SKU: Basic
IP address assignment: Static
Create Virtual Machines
First of all, you have to get the ID of the subnet where the VMs will be created.
az network vnet list -g MY-VNET-RESOURCE-GROUP
Identify and get subnet ID. It will be something like:
/subscriptions/d66a4ccf-4568-41fc-8b2f-3d930a88486e/resourceGroups/MY-VNET-RESOURCE-GROUP/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/MY-SUBNET
Then, create your VMs.
az vm create `  
  --name vm-master-001 `
  --resource-group MY-RESOURCE-GROUP `
  --image UbuntuLTS `
  --size Standard_F4s `
  --authentication-type password `
  --admin-username ambev `
  --admin-password MySecretPWD! `
  --nsg MY-NSG `
  --nsg-rule SSH `
  --os-disk-size-gb 100 `
  --availability-set MY-AV `
  --subnet /subscriptions/d66a4ccf-4568-41fc-8b2f-3d930a88486e/resourceGroups/MY-VNET-RESOURCE-GROUP/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/MY-SUBNET
Create one "master" and one "worker".
Install Docker
SSH to both your servers, master and worker.
Install docker dependencies.
sudo apt-get -y install \    
  apt-transport-https \
  ca-certificates \
  curl \
  gnupg-agent \
  software-properties-common
Setup the repository.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
Install Docker.
sudo apt-get update
sudo apt-get -y install docker-ce=18.06.2~ce~3-0~ubuntu
Check with docker version you should install to support you K8s version:
https://kubernetes.io/docs/setup/production-environment/container-runtimes/#docker​
Edit Docker service file.
nano /lib/systemd/system/docker.service
Edit Docker service file.
nano /lib/systemd/system/docker.service
Add the --exec-opt native.cgroupdriver=systemdconfig at the end of the line "ExecStart":
ExecStart=/usr/bin/dockerd -H fd:// --exec-opt native.cgroupdriver=systemd
Restart services:
systemctl daemon-reload
systemctl restart docker
Install kubeadm, kubectl and kubelet
SSH to your master node.
Follow the instructions here: https://app.gitbook.com/@devops-buzz/s/public/kubeadm/cheat-sheet#install-kubeadm-kubectl-and-kubelet​
Then get the packages versions and copy them. They will be used to install the worker node with exactly the same versions: https://app.gitbook.com/@devops-buzz/s/public/kubeadm/cheat-sheet#get-kubeadm-dependencies-versions​
SSH to your worker node, and install the same version: https://app.gitbook.com/@devops-buzz/s/public/~/drafts/-LhMcXSw47Ajn24zlx5O/primary/kubeadm/cheat-sheet#specific-version​
Create an AD App registrations
Register an application
On Azure Portal, go to "Azure Active Directory", "App registrations".
Select "New registration".
 Fill up the form.
Add role assignment
Go to "All services", "Subscriptions" and select your subscription.
Go to "Access control (IAM), "Add", "Add role assignment".
Go to "Role" and "Assign access to". "Select" the application you created.
Go to "Azure Active Directory", "App registrations" and find your application.
Get the "Application (client) ID".
New client secret
Go to "Certificates & secrets", "New client secret".
Fill up the form.
Copy and save the secret value.
The secret value will be displayed just once. Copy and save it now!
Setup cluster
SSH to your master server.
Create config files
Create kubeadm config file.
nano /etc/kubeadm.conf
With the following content.
apiServer:
  certSANs:
  - MY-SERVER-PUBLIC-IP
  - MY-PUBLIC-IP-URL.com
  - MY-CREATED-PUBLIC-IP
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: MY-SERVER-INTERNAL-IP:6443
controllerManager:
  extraArgs:
    cloud-config: /etc/kubernetes/cloud.conf
    cloud-provider: azure
  extraVolumes:
  - hostPath: /etc/kubernetes/cloud.conf
    mountPath: /etc/kubernetes/cloud.conf
    name: cloud
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.14.2
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.240.0.0/16
scheduler: {}
Create cloud config file.
nano /etc/kubernetes/cloud.conf
With the following content.
{
    "cloud":"AzurePublicCloud",
    "tenantId": "Def04b19-7776-4a94-b45b-375c77a8f966",
    "subscriptionId": "d44a4ccf-4578-87fc-8F2d-3d340a88486e",
    "aadClientId": "CCef9bf7-b34e-69be-a1c9-d5be47b846d2",
    "aadClientSecret": "s]ksASDFffaseWuT2MJsafdfdsf18OsdfJzc1",
    "resourceGroup": "MY-RESOURCE-GROUP",
    "location": "brazilsouth",
    "vmType": "Standard_F4s",
    "subnetName": "MY-SUBNET-NAME",
    "securityGroupName": "MY-RESOURCE-GROUP",
    "vnetName": "MY-VNET",
    "vnetResourceGroup": "MY-VNET-RESOURCE-GROUP",
    "routeTableName": "MY-RESOURCE-GROUP",
    "primaryAvailabilitySetName": "MY-RESOURCE-GROUP",
    "primaryScaleSetName": "",
    "cloudProviderBackoff": true,
    "cloudProviderBackoffRetries": 6,
    "cloudProviderBackoffExponent": 1.5,
    "cloudProviderBackoffDuration": 5,
    "cloudProviderBackoffJitter": 1,
    "cloudProviderRatelimit": true,
    "cloudProviderRateLimitQPS": 3,
    "cloudProviderRateLimitBucket": 10,
    "useManagedIdentityExtension": false,
    "userAssignedIdentityID": "",
    "useInstanceMetadata": true,
    "loadBalancerSku": "Basic",
    "excludeMasterFromStandardLB": false,
    "providerVaultName": "",
    "maximumLoadBalancerRuleCount": 250,
    "providerKeyName": "k8s",
    "providerKeyVersion": ""
}
To get the "tenantId", run az account show and get "tenantId" field.
To get the "subscriptionId", run az account show and get "id" field.
"aadClientId" is the "Application (client) ID" from Azure AD Application created above.
"aadClientSecret" is the Application secret created above.
Init cluster
kubeadm init --config kubeadm.conf
The command above will return the "join" command, like:
kubeadm join YOUR-SERVER-INTERNAL-IP:6443 --token 2m8oey.aeejkbpfgpg0izi0 \
    --discovery-token-ca-cert-hash sha256:bb4527127bbe01f049532c55f681dcf25ce31a1b4ff81797b615b2a247862e75
Save this command in safe place. It will be used to add nodes to your cluster.
Get kubeconfig
SSH to your master node and run as root.
mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Deploy Weave CNI
SSH to your master node and run as root.
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
Azure does not support Calico.
Wait until the master node is "READY".
watch kubectl get pods --all-namespaces
Add nodes
SSH to your node and run the "join" command you got above.
Label nodes (optional)
kubectl label node \
  MY-node-001 node-role.kubernetes.io/worker=worker
Setup Load Balancer
Create any service type "LoadBalancer". This will create create a Load Balancer on Azure called "kubernetes".
Create the manifest file.
nano service.yml
With the following content.
kind: Service
apiVersion: v1
metadata:
  name: first-service
spec:
  selector:
    app: first-deployment
  ports:
  - protocol: TCP
    port: 80
  type: LoadBalancer
Deploy the service.
kubectl create -f service.yml
Go to Azure portal, find the Load Balancer "kuberntes" in your Resource Group.
Wait a few minutes. You should have a "Backend pool" called "kubernets" with your node in it.
Add a "Frontend IP Configuration" and select the public IP you created above.
Create a "Backend pool" add your master server.
Create a "health probe" for you master server on port 6443.
Create a "Load Balancing Rule" using your Public IP on port 6443 to your master pool.
At this point you should be able to use your kubeconfig your public DNS.
Setup static IPs
Go to Azure portal, find your VMs, go to "Networking", lick on the NIC, go to "IP configurations", click on the IP, on "Assignment", set it to "Static".
Find the VMs Public IPs and double check if they are "Static".
​
​
Kubeadm - Previous
Setup cluster
Bare-metal
Last modified 3yr ago