# Bare-metal ## Install docker Check which Docker version Kubernetes supports: By the time of this writing, the latest supported docker version is 18.06.2. Install dependencies: ```bash sudo apt-get -y install \ apt-transport-https \ ca-certificates \ curl \ gnupg-agent \ software-properties-common ``` Add the repo: ```bash curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" ``` Install Docker: ```bash sudo apt-get update sudo apt-get -y install docker-ce=18.06.2~ce~3-0~ubuntu ``` {% hint style="info" %} Replace 18.06.2 with the latest supported docker version. {% endhint %} ## Setup firewall Master ![](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LfyPEENom-5baIjZIOV%2F-LfyPQ8pLNqxlGrj8uVq%2Fimage.png?alt=media\&token=17b91be6-3180-4f8c-94b1-6f770fd6c8a9) Nodes ![](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LfyPEENom-5baIjZIOV%2F-LfyPUuYwJSawCKZ36bC%2Fimage.png?alt=media\&token=2df8a669-a3d8-4fa0-9c48-13383647f1d6) ### References ## Install kubelet, kubeadm and kubectl ```bash apt-get update && apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat </etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update apt-get install -y kubelet kubeadm kubectl apt-mark hold kubelet kubeadm kubectl ``` ## Init cluster {% hint style="warning" %} Make sure you server's host name is configured (avoid changing the host name after the cluster is created). {% endhint %} ```bash kubeadm init \ --pod-network-cidr=172.31.0.0/20 \ --apiserver-advertise-address=0.0.0.0 \ --apiserver-cert-extra-sans=13.211.85.85 ``` {% hint style="warning" %} Copy and save the `kubeadm join` command. {% endhint %} Configure your user to run kubectl: ```bash mkdir -p $HOME/.kube sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` Deploy Weave Net: ``` kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')" ``` {% hint style="warning" %} By the time of this writing, Kubeadm has an issue. Coredns will remaing "pending" until you deploy Weave Net: {% endhint %} Confirm that all of the pods are running with the following command. ```bash watch kubectl get pods --all-namespaces ``` Wait until each pod has the STATUS of Running. Confirm that master node is ready: ```bash kubectl get nodes ``` {% hint style="warning" %} If you want to deploy Calico instead, check the latest documentation: {% endhint %} ### References ## Dashboard Deploy the dashboard: ``` kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml ``` In your workstation... ``` kubectl proxy ``` Create dashboard user: ## Join node Install kubelet, kubeadm and kubectl. ### Setup Docker cgroupdriver systemd Check docker systemd service config file: ```bash systemctl cat docker ``` Sample output: ```bash /lib/systemd/system/docker.service ``` Edit the file: ```bash nano /lib/systemd/system/docker.service ``` Add `--exec-opt native.cgroupdriver=systemd` to the `ExecStart` option: ```bash ExecStart=/usr/bin/dockerd -H fd:// --exec-opt native.cgroupdriver=systemd ``` Restart daemon: ```bash systemctl daemon-reload systemctl restart docker ``` ### Add node to the cluster {% hint style="warning" %} The worker node must have exactly the same master versions of the following packages: kubelet, kubeadm, kubectl, kubernetes-cni. Reference [here](https://devops-buzz.gitbook.io/public/kubeadm/known-errors-and-solutions#error-execution-phase-kubelet-start-configmaps-kubelet-config-1-14-is-forbidden-user-system-bootstrap-g-651-e8-cannot-get-resource-configmaps-in-api-group-in-the-namespace-kube-system). {% endhint %} Install packages, for example: ```bash apt-get install \ kubelet=1.13.4-00 \ kubeadm=1.13.4-00 \ kubectl=1.13.4-00 \ kubernetes-cni=0.6.0-00 ``` Hold packages versions: ```bash apt-mark hold kubelet kubeadm kubectl kubernetes-cni ``` {% hint style="warning" %} Make sure the master node firewall allows the new node to access it on port 6443. {% endhint %} {% hint style="warning" %} Make sure you server's host name is configured (avoid changing the host name after). {% endhint %} Run the joint command: ``` kubeadm join 172.31.7.247:6443 --token g651e8.p09een664u224v76 --discovery-token-ca-cert-hash sha256:b589d4690ac427c5b7046233963d058e2e2cb99f168f90d58ec15a351aa4243b ``` {% hint style="info" %} Use your cluster join command. {% endhint %} {% hint style="info" %} If by any chance you need to run the join command more than one time and you the certificate `FileAvailable--etc-kubernetes-pki-ca.crt`, run this command before: ```bash rm -fr /etc/kubernetes/* ``` {% endhint %} You should see this output: ``` This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the master to see this node join the cluster. ``` **Go to your master server** and label the new node: ```bash kubectl label node PUT-YOURNODE-HOSTNAME-HERE node-role.kubernetes.io/worker=worker ``` ## Remove node List your current nodes: ```bash kubectl get nodes ``` Output example: ```bash NAME STATUS ROLES AGE VERSION ip-172-31-7-247 Ready master 46h v1.13.4 ip-172-31-9-145 Ready worker 45h v1.13.4 ``` Let's suppose you want to remove the node `ip-172-31-9-145`: ```bash kubectl drain ip-172-31-9-145 --delete-local-data --force --ignore-daemonsets ``` Check node status: ```bash kubectl get nodes ``` Output example: ```bash NAME STATUS ROLES AGE VERSION ip-172-31-7-247 Ready master 46h v1.13.4 ip-172-31-9-145 Ready,SchedulingDisabled worker 45h v1.13.4 ``` Then, **on the node being removed**, reset all kubeadm installed state:: ```bash kubeadm reset ``` The reset process does not reset or clean up iptables rules or IPVS tables. If you wish to reset iptables, you must do so manually: ```bash iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X ``` If you want to reset the IPVS tables, you must run the following command: ```bash ipvsadm -C ``` ## Kubeadm over VPN ``` kubeadm init \ --apiserver-advertise-address=10.187.216.232 \ --pod-network-cidr=10.32.0.0/12 \ --service-cidr=10.96.0.0/12 ```