# Bare-metal ## Install docker Check which Docker version Kubernetes supports: By the time of this writing, the latest supported docker version is 18.06.2. Install dependencies: ```bash sudo apt-get -y install \ apt-transport-https \ ca-certificates \ curl \ gnupg-agent \ software-properties-common ``` Add the repo: ```bash curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" ``` Install Docker: ```bash sudo apt-get update sudo apt-get -y install docker-ce=18.06.2~ce~3-0~ubuntu ``` {% hint style="info" %} Replace 18.06.2 with the latest supported docker version. {% endhint %} ## Setup firewall Master ![](/files/-LfyPQ8pLNqxlGrj8uVq) Nodes ![](/files/-LfyPUuYwJSawCKZ36bC) ### References ## Install kubelet, kubeadm and kubectl ```bash apt-get update && apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat </etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update apt-get install -y kubelet kubeadm kubectl apt-mark hold kubelet kubeadm kubectl ``` ## Init cluster {% hint style="warning" %} Make sure you server's host name is configured (avoid changing the host name after the cluster is created). {% endhint %} ```bash kubeadm init \ --pod-network-cidr=172.31.0.0/20 \ --apiserver-advertise-address=0.0.0.0 \ --apiserver-cert-extra-sans=13.211.85.85 ``` {% hint style="warning" %} Copy and save the `kubeadm join` command. {% endhint %} Configure your user to run kubectl: ```bash mkdir -p $HOME/.kube sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` Deploy Weave Net: ``` kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')" ``` {% hint style="warning" %} By the time of this writing, Kubeadm has an issue. Coredns will remaing "pending" until you deploy Weave Net: {% endhint %} Confirm that all of the pods are running with the following command. ```bash watch kubectl get pods --all-namespaces ``` Wait until each pod has the STATUS of Running. Confirm that master node is ready: ```bash kubectl get nodes ``` {% hint style="warning" %} If you want to deploy Calico instead, check the latest documentation: {% endhint %} ### References ## Dashboard Deploy the dashboard: ``` kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml ``` In your workstation... ``` kubectl proxy ``` Create dashboard user: ## Join node Install kubelet, kubeadm and kubectl. ### Setup Docker cgroupdriver systemd Check docker systemd service config file: ```bash systemctl cat docker ``` Sample output: ```bash /lib/systemd/system/docker.service ``` Edit the file: ```bash nano /lib/systemd/system/docker.service ``` Add `--exec-opt native.cgroupdriver=systemd` to the `ExecStart` option: ```bash ExecStart=/usr/bin/dockerd -H fd:// --exec-opt native.cgroupdriver=systemd ``` Restart daemon: ```bash systemctl daemon-reload systemctl restart docker ``` ### Add node to the cluster {% hint style="warning" %} The worker node must have exactly the same master versions of the following packages: kubelet, kubeadm, kubectl, kubernetes-cni. Reference [here](https://devops-buzz.gitbook.io/public/kubeadm/known-errors-and-solutions#error-execution-phase-kubelet-start-configmaps-kubelet-config-1-14-is-forbidden-user-system-bootstrap-g-651-e8-cannot-get-resource-configmaps-in-api-group-in-the-namespace-kube-system). {% endhint %} Install packages, for example: ```bash apt-get install \ kubelet=1.13.4-00 \ kubeadm=1.13.4-00 \ kubectl=1.13.4-00 \ kubernetes-cni=0.6.0-00 ``` Hold packages versions: ```bash apt-mark hold kubelet kubeadm kubectl kubernetes-cni ``` {% hint style="warning" %} Make sure the master node firewall allows the new node to access it on port 6443. {% endhint %} {% hint style="warning" %} Make sure you server's host name is configured (avoid changing the host name after). {% endhint %} Run the joint command: ``` kubeadm join 172.31.7.247:6443 --token g651e8.p09een664u224v76 --discovery-token-ca-cert-hash sha256:b589d4690ac427c5b7046233963d058e2e2cb99f168f90d58ec15a351aa4243b ``` {% hint style="info" %} Use your cluster join command. {% endhint %} {% hint style="info" %} If by any chance you need to run the join command more than one time and you the certificate `FileAvailable--etc-kubernetes-pki-ca.crt`, run this command before: ```bash rm -fr /etc/kubernetes/* ``` {% endhint %} You should see this output: ``` This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the master to see this node join the cluster. ``` **Go to your master server** and label the new node: ```bash kubectl label node PUT-YOURNODE-HOSTNAME-HERE node-role.kubernetes.io/worker=worker ``` ## Remove node List your current nodes: ```bash kubectl get nodes ``` Output example: ```bash NAME STATUS ROLES AGE VERSION ip-172-31-7-247 Ready master 46h v1.13.4 ip-172-31-9-145 Ready worker 45h v1.13.4 ``` Let's suppose you want to remove the node `ip-172-31-9-145`: ```bash kubectl drain ip-172-31-9-145 --delete-local-data --force --ignore-daemonsets ``` Check node status: ```bash kubectl get nodes ``` Output example: ```bash NAME STATUS ROLES AGE VERSION ip-172-31-7-247 Ready master 46h v1.13.4 ip-172-31-9-145 Ready,SchedulingDisabled worker 45h v1.13.4 ``` Then, **on the node being removed**, reset all kubeadm installed state:: ```bash kubeadm reset ``` The reset process does not reset or clean up iptables rules or IPVS tables. If you wish to reset iptables, you must do so manually: ```bash iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X ``` If you want to reset the IPVS tables, you must run the following command: ```bash ipvsadm -C ``` ## Kubeadm over VPN ``` kubeadm init \ --apiserver-advertise-address=10.187.216.232 \ --pod-network-cidr=10.32.0.0/12 \ --service-cidr=10.96.0.0/12 ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://www.devops.buzz/public/kubeadm/setup-cluster-1/setup-cluster.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.