# Using EKS and Gitlab CI to deploy applications ## Introduction The main goal is to have a production-ready environment, showcasing AWS architecture, Terraform, Ansible, Kubernetes (EKS), Gitlab CI, DockerHub and Helm. Key aspects: * All resources are managed in code. Even the bootstrap of the project . * HA where applicable (EKS Load Balancer) * The application is deployed from code. ## General overview The process described on this tutorial show how to: 1. Terraform: leverage an S3 bucket to store states ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/001-backend-s3/main.tf)); 2. Terraform: leverage a DynamoDB table to store locks ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/002-backend-lock/main.tf)); 3. Terraform: leverage a Virtual Machine ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/003-vm-mgmt/main.tf)); 4. Ansible: setup VM swap, hostname and packages. Also, create a Docker container which runs the EKS setup process, setup kubeconfig and Helm ([here](https://gitlab.com/tadeugr/eintopf/blob/master/ansible/playbooks/mgmt.yaml)). 5. Gitlab CI: run a pipeline to build ([here](https://gitlab.com/tadeugr/rouladen/blob/master/.gitlab-ci.yml#L10)) a Docker image and push to DockerHub ([here](https://cloud.docker.com/repository/docker/tadeugr/strudel)); 6. Gitlab CI: deploy the Docker image on K8s using Helm ([here](https://gitlab.com/tadeugr/rouladen/blob/master/.gitlab-ci.yml#L27) and [here](https://hub.docker.com/r/tadeugr/devops)). ## Stack overview * AWS * Terraform * Ansible * Gitlab CI * Dockerhub * EKS * K8s * Helm ## Before you begin ### AWS account Make sure you have an AWS account. AWS will host the Kubernetes cluster (EKS) and a small Virtual Machine. {% hint style="info" %} This tutorial does not fit the AWS free tier. Make sure you delete all resources to avoid charges. {% endhint %} ### Gitlab account Make sure you have a Gitlab account to commit your code and use pipelines. ### Docker The stack bootstrap is done using Docker, so make sure you have Docker installed on your workstation. ## Provisioning the infrastructure ### Setup AWS IAM You need to create an IAM user which will be used with Terraform. Login to you AWS console, go to Services, IAM. ![IAM Service](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmEJICx2hQguuLsnzp6%2F-LmELlsQ-mbhWTFCz51s%2Fimage.png?alt=media\&token=3d869df0-7964-425b-b8f3-ba3b24ec340c) Go to Users, Add user. ![Add IAM user](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmELv3Dzu_W4vA3_BoJ%2F-LmEMbFIXTf53oKkt9KM%2Fimage.png?alt=media\&token=b0026b77-62a9-4bc0-9c8a-94861e318c3c) Add a user called `iac` (stands for Infra as Code) with Programmatic Access. ![iac user](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmENXGpyRrNkwoCx92H%2F-LmEN_6VlT9kp-hdnksl%2Fimage.png?alt=media\&token=32caf20f-fa61-4538-8818-0d8dadc08398) Attach AdministratorAccess and click on Next: Tags button. ![policy](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmENXGpyRrNkwoCx92H%2F-LmEOJ0wYJTFwuwxGw6v%2Fimage.png?alt=media\&token=24a8d37a-518e-43f6-b578-eec0ac34064b) {% hint style="info" %} You can use restricted policies if you want. In this example we are using AdministratorAccess to keep it simple. {% endhint %} Optionally add tags then click on Next: Review. ![IAM tags](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmENXGpyRrNkwoCx92H%2F-LmEP6pefhl7cruYpvmw%2Fimage.png?alt=media\&token=ee5e1cc1-c930-492e-858a-7f49e3543347) Review and create user. ![Create user](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmEPJq3CzipAbi5dg5K%2F-LmEPcM0ZjK_r4r-_Gfk%2Fimage.png?alt=media\&token=7576aa06-3358-4631-b451-6c2a28d3e8ea) Click on `Show` in `Secret access key` section. Copy and save in a safe place your `Access key ID` and `Secret access key`. ![Credentials](https://1923299483-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LVfLS9KLUfusOjV0FX_%2F-LmEPJq3CzipAbi5dg5K%2F-LmEQ4zXON8_j3IsnKs2%2Fimage.png?alt=media\&token=6783d67b-4d5b-4ed0-bd07-d7341827b446) {% hint style="info" %} The credentials are displayed just once. Save them now or you will have to create a new user. {% endhint %} From now on, the `Access key ID` and `Secret access key` will be referenced in this tutorial as `iac IAM user credentials`. ### Terraform apply Clone or fork `eintopf` repository *(Terraform and Ansible scripts)*: #### Setup AWS credentials Follow the instructions here: Use your **iac IAM user credentials**. #### Run Terraform scripts Once your have your AWS credentials setup, run the Terraform scripts. To do so, follow the instructions here: At this point, if you go to you AWS console, you should have: * An S3 bucket; * A DynamoDB table; * A Virtual Machine. ## Setup the infrastructure `eintopf` also has Ansible playbooks and roles do setup the infrastructure. The playbook `playbooks/mgmt.yaml` setup a Virtual Machine with a Docker container. This container is responsible for setting up the EKS cluster. First, make sure you have ansible-vault configured: Then, setup the infrastructure: At this point, if you go to you AWS console, you should have: * An EKS cluster. Also, the Virtual Machine called `mgmt` hosts a Docker container called `devops`. Inside this container you will find you Kubernetes cluster `kubeconfig` file, which is used with `kubectl` and also must be configured on your Gitlab CI variables to run the deployment test. More info here: You can copy the `kubeconfig` file content and paste and save it in a safe place, specially if you want o use `kubectl` from your workstation and not form the VM. To convert the `kubeconfig` content in the format to be used on Gitlab CI variable, run: ```bash cat /root/.kube/config | base64 ``` ## Test a deployment Clone or fork the following repo: Setup the required variables to run the pipeline: Push a change and watch the deployment logs.