DevOps Buzz
Search…
⌃K

Node management

Change kernel parameters

Pod’s securityContext will most likely result in SysctlForbidden erros.
To workaround, create DaemonSet.
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: more-fs-watchers
namespace: kube-system
labels:
app: more-fs-watchers
spec:
template:
metadata:
labels:
name: more-fs-watchers
spec:
hostNetwork: true
hostPID: true
hostIPC: true
initContainers:
- command:
- sh
- -c
- sysctl -w fs.inotify.max_user_watches=524288;
image: alpine:3.6
imagePullPolicy: IfNotPresent
name: sysctl
resources: {}
securityContext:
privileged: true
volumeMounts:
- name: sys
mountPath: /sys
containers:
- resources:
requests:
cpu: 0.01
image: alpine:3.6
name: sleepforever
command: ["tail"]
args: ["-f", "/dev/null"]
volumes:
- name: sys
hostPath:
path: /sys

Reboot node

  • Manually, through the Azure portal or the Azure CLI.
  • By upgrading your AKS cluster. The cluster upgrades cordon and drain nodes automatically and then bring a new node online with the latest Ubuntu image and a new patch version or a minor Kubernetes version. For more information, see Upgrade an AKS cluster.
  • By using Kured, an open-source reboot daemon for Kubernetes. Kured runs as a DaemonSet and monitors each node for the presence of a file that indicates that a reboot is required. Across the cluster, OS reboots are managed by the same cordon and drain process as a cluster upgrade.
References

SSH to nodes

Set your subscription.
az account set --subscription "MY-SUBSCRIPTION"
Set an env var with your cluster resources RG.
CLUSTER_RESOURCE_GROUP=MC_my-aks-name
Add your RSA key to the node.
az vm user update \
--resource-group $CLUSTER_RESOURCE_GROUP \
--name PUT-YOUR-NODE-NAME-HERE \
--username azureuser \
--ssh-key-value ~/.ssh/id_rsa.pub
Get your node IP.
az vm list-ip-addresses --resource-group $CLUSTER_RESOURCE_GROUP -o table
Run a pod.
kubectl run -it --rm aks-ssh --image=debian
Install SSH client.
apt-get update && apt-get install openssh-client vim -y
Setup the id_rsa file.
mkdir ~/.ssh
vi ~/.ssh/id_rsa
# Paste your id_rsa
chmod 600 ~/.ssh/id_rsa
SSH to your node.