# Using EKS and Gitlab CI to deploy applications ## Introduction The main goal is to have a production-ready environment, showcasing AWS architecture, Terraform, Ansible, Kubernetes (EKS), Gitlab CI, DockerHub and Helm. Key aspects: * All resources are managed in code. Even the bootstrap of the project . * HA where applicable (EKS Load Balancer) * The application is deployed from code. ## General overview The process described on this tutorial show how to: 1. Terraform: leverage an S3 bucket to store states ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/001-backend-s3/main.tf)); 2. Terraform: leverage a DynamoDB table to store locks ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/002-backend-lock/main.tf)); 3. Terraform: leverage a Virtual Machine ([here](https://gitlab.com/tadeugr/eintopf/blob/master/terraform/003-vm-mgmt/main.tf)); 4. Ansible: setup VM swap, hostname and packages. Also, create a Docker container which runs the EKS setup process, setup kubeconfig and Helm ([here](https://gitlab.com/tadeugr/eintopf/blob/master/ansible/playbooks/mgmt.yaml)). 5. Gitlab CI: run a pipeline to build ([here](https://gitlab.com/tadeugr/rouladen/blob/master/.gitlab-ci.yml#L10)) a Docker image and push to DockerHub ([here](https://cloud.docker.com/repository/docker/tadeugr/strudel)); 6. Gitlab CI: deploy the Docker image on K8s using Helm ([here](https://gitlab.com/tadeugr/rouladen/blob/master/.gitlab-ci.yml#L27) and [here](https://hub.docker.com/r/tadeugr/devops)). ## Stack overview * AWS * Terraform * Ansible * Gitlab CI * Dockerhub * EKS * K8s * Helm ## Before you begin ### AWS account Make sure you have an AWS account. AWS will host the Kubernetes cluster (EKS) and a small Virtual Machine. {% hint style="info" %} This tutorial does not fit the AWS free tier. Make sure you delete all resources to avoid charges. {% endhint %} ### Gitlab account Make sure you have a Gitlab account to commit your code and use pipelines. ### Docker The stack bootstrap is done using Docker, so make sure you have Docker installed on your workstation. ## Provisioning the infrastructure ### Setup AWS IAM You need to create an IAM user which will be used with Terraform. Login to you AWS console, go to Services, IAM. ![IAM Service](/files/-LmELlsQ-mbhWTFCz51s) Go to Users, Add user. ![Add IAM user](/files/-LmEMbFIXTf53oKkt9KM) Add a user called `iac` (stands for Infra as Code) with Programmatic Access. ![iac user](/files/-LmEN_6VlT9kp-hdnksl) Attach AdministratorAccess and click on Next: Tags button. ![policy](/files/-LmEOJ0wYJTFwuwxGw6v) {% hint style="info" %} You can use restricted policies if you want. In this example we are using AdministratorAccess to keep it simple. {% endhint %} Optionally add tags then click on Next: Review. ![IAM tags](/files/-LmEP6pefhl7cruYpvmw) Review and create user. ![Create user](/files/-LmEPcM0ZjK_r4r-_Gfk) Click on `Show` in `Secret access key` section. Copy and save in a safe place your `Access key ID` and `Secret access key`. ![Credentials](/files/-LmEQ4zXON8_j3IsnKs2) {% hint style="info" %} The credentials are displayed just once. Save them now or you will have to create a new user. {% endhint %} From now on, the `Access key ID` and `Secret access key` will be referenced in this tutorial as `iac IAM user credentials`. ### Terraform apply Clone or fork `eintopf` repository *(Terraform and Ansible scripts)*: #### Setup AWS credentials Follow the instructions here: Use your **iac IAM user credentials**. #### Run Terraform scripts Once your have your AWS credentials setup, run the Terraform scripts. To do so, follow the instructions here: At this point, if you go to you AWS console, you should have: * An S3 bucket; * A DynamoDB table; * A Virtual Machine. ## Setup the infrastructure `eintopf` also has Ansible playbooks and roles do setup the infrastructure. The playbook `playbooks/mgmt.yaml` setup a Virtual Machine with a Docker container. This container is responsible for setting up the EKS cluster. First, make sure you have ansible-vault configured: Then, setup the infrastructure: At this point, if you go to you AWS console, you should have: * An EKS cluster. Also, the Virtual Machine called `mgmt` hosts a Docker container called `devops`. Inside this container you will find you Kubernetes cluster `kubeconfig` file, which is used with `kubectl` and also must be configured on your Gitlab CI variables to run the deployment test. More info here: You can copy the `kubeconfig` file content and paste and save it in a safe place, specially if you want o use `kubectl` from your workstation and not form the VM. To convert the `kubeconfig` content in the format to be used on Gitlab CI variable, run: ```bash cat /root/.kube/config | base64 ``` ## Test a deployment Clone or fork the following repo: Setup the required variables to run the pipeline: Push a change and watch the deployment logs. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://www.devops.buzz/public/stack-setup/eks-+-gitlab-ci.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.