DevOps Buzz
  • Initial page
  • About me
  • Ansible
    • Cheat Sheet
    • Dynamic inventory
  • AKS
    • Cheat Sheet
    • Autoscale
    • Backup
    • Dashboard
    • Ingress
    • Node management
    • News
    • Persistent Volumes
  • Arch Linux
    • Docker
    • Install
    • Network
    • VirtualBox guest
  • Azure
    • API
    • CLI/Powershell
    • DevOps
      • Cheat Sheet
    • Application Gateway
    • Tools
    • Sops
  • AWS
    • Cheat Sheet
    • Tools
    • Cognito with Amplify and React
  • Bash / Shell
    • Cheat Sheet
    • Tools
  • Bitbucket
    • Cheat Sheet
  • CSS
    • Cheat Sheet
  • Distros
    • Manjaro
  • Docker
    • Cheat Sheet
    • ELK
    • Ubuntu NoVNC
    • Tools
    • Known errors and solutions
  • Elasticsearch
    • Cheat Sheet
    • Tools
  • ELK
    • Cheat Sheet
  • emacs
    • Cheat sheet
  • Gatekeeper (OPA)
    • Cheat Sheet
    • Developer Quick Start
  • GCP
    • Cheat Sheet
  • General
    • Tools
    • News
  • Git
    • Cheat Sheet
    • Tools
  • Golang
    • Cheat Sheet
  • Guidelines / Standards
    • Cheat Sheet
  • i3wm
    • Cheat Sheet
  • Ipsec
    • Cheat sheet
  • Istio
    • Cheat Sheet
  • Kind
    • Cheat Sheet
  • Kops
    • Cheat Sheet
  • Kubeadm
    • Cheat Sheet
    • Change serviceSubnet CIDR
    • Setup cluster
      • Azure
      • Bare-metal
      • Vagrant + VirtualBox
    • Multi master
    • Known errors and solutions
  • Kubernetes
    • Kubectl Cheat Sheet
    • etcd Cheat Sheet
    • Tools
    • News
    • Deployments
      • Deployment examples
      • Blue/Green Deployment
      • Canary Deployment
    • Dashboard
    • ELK
    • Helm
    • Ingress
    • logz.io
    • Minikube
    • Monitoring
    • Node Management
    • Operators
    • Security
    • Volumes
    • Networking
    • kube-controller-manager
      • Node crash recovery
    • Known errors and solutions
  • Lumen
    • Cheat Sheet
  • MACOS
    • Cheat Sheet
    • Qemu
    • Zsh
  • microk8s
    • Cheat Sheet
  • MongoDB
    • Cheat Sheet
    • Tools
  • MySQL
    • Cheat Sheet
  • Network
    • Tools
    • WDS - Wireless Distribution System
    • Expose server under NAT
  • nvim
    • Cheat Sheet
  • Openvpn
    • OpenVPN server on Ubuntu 18.06
    • Stunnel
  • PHP
    • Composer
  • Prometheus
    • Tools
  • Python
    • Cheat Sheet
    • Pydantic
    • Tools / Modules
    • Virtualenv
  • RabbitMQ
    • Cheat Sheet
  • ReactJS
    • Fixes
    • For beginners
  • Ruby
    • Cheat Sheet
    • Rails
    • rvm
  • Rundeck
    • Cheat Sheet
  • Rust
    • Cheat Sheet
  • Squid
    • Setup server
    • Expose NAT server
  • SRE
    • Cheat Sheet
  • SSH
    • Passwordless auth with RSA key
    • Reverse tunnel
    • Cheat Sheet
  • SSL
    • Cheat Sheet
    • certbot
  • STACK SETUP
    • Using EKS and Gitlab CI to deploy applications
  • Terraform
    • Cheat Sheet
    • Tools
  • Tmux
    • Cheat Sheet
  • Tor
    • Cheat Sheet
  • Ubuntu
    • Cheat Sheet
  • Vagrant
    • Cheat Sheet
  • VirtualBox
    • Cheat Sheet
  • Windows
    • Windows Docker
    • Fingerprint
    • SSH Client
    • Tools
    • VirtualBox
    • WSL
Powered by GitBook
On this page
  • Introduction
  • General overview
  • Stack overview
  • Before you begin
  • AWS account
  • Gitlab account
  • Docker
  • Provisioning the infrastructure
  • Setup AWS IAM
  • Terraform apply
  • Setup the infrastructure
  • Test a deployment
  1. STACK SETUP

Using EKS and Gitlab CI to deploy applications

Setup EKS using Terraform and Ansible. Deploy applications on EKS using Gitlab CI and Helm.

PreviouscertbotNextCheat Sheet

Last updated 5 years ago

Introduction

The main goal is to have a production-ready environment, showcasing AWS architecture, Terraform, Ansible, Kubernetes (EKS), Gitlab CI, DockerHub and Helm.

Key aspects:

  • All resources are managed in code. Even the bootstrap of the project .

  • HA where applicable (EKS Load Balancer)

  • The application is deployed from code.

General overview

The process described on this tutorial show how to:

  1. Terraform: leverage an S3 bucket to store states ();

  2. Terraform: leverage a DynamoDB table to store locks ();

  3. Terraform: leverage a Virtual Machine ();

  4. Ansible: setup VM swap, hostname and packages. Also, create a Docker container which runs the EKS setup process, setup kubeconfig and Helm ().

  5. Gitlab CI: run a pipeline to build () a Docker image and push to DockerHub ();

  6. Gitlab CI: deploy the Docker image on K8s using Helm ( and ).

Stack overview

  • AWS

  • Terraform

  • Ansible

  • Gitlab CI

  • Dockerhub

  • EKS

  • K8s

  • Helm

Before you begin

AWS account

Make sure you have an AWS account. AWS will host the Kubernetes cluster (EKS) and a small Virtual Machine.

This tutorial does not fit the AWS free tier. Make sure you delete all resources to avoid charges.

Gitlab account

Make sure you have a Gitlab account to commit your code and use pipelines.

Docker

The stack bootstrap is done using Docker, so make sure you have Docker installed on your workstation.

Provisioning the infrastructure

Setup AWS IAM

You need to create an IAM user which will be used with Terraform.

Login to you AWS console, go to Services, IAM.

Go to Users, Add user.

Add a user called iac (stands for Infra as Code) with Programmatic Access.

Attach AdministratorAccess and click on Next: Tags button.

You can use restricted policies if you want. In this example we are using AdministratorAccess to keep it simple.

Optionally add tags then click on Next: Review.

Review and create user.

Click on Show in Secret access key section. Copy and save in a safe place your Access key ID and Secret access key.

The credentials are displayed just once. Save them now or you will have to create a new user.

From now on, the Access key ID and Secret access key will be referenced in this tutorial as iac IAM user credentials.

Terraform apply

Setup AWS credentials

Use your iac IAM user credentials.

Run Terraform scripts

Once your have your AWS credentials setup, run the Terraform scripts.

At this point, if you go to you AWS console, you should have:

  • An S3 bucket;

  • A DynamoDB table;

  • A Virtual Machine.

Setup the infrastructure

eintopf also has Ansible playbooks and roles do setup the infrastructure. The playbook playbooks/mgmt.yaml setup a Virtual Machine with a Docker container. This container is responsible for setting up the EKS cluster.

At this point, if you go to you AWS console, you should have:

  • An EKS cluster.

You can copy the kubeconfig file content and paste and save it in a safe place, specially if you want o use kubectl from your workstation and not form the VM.

To convert the kubeconfig content in the format to be used on Gitlab CI variable, run:

cat /root/.kube/config | base64

Test a deployment

Push a change and watch the deployment logs.

Clone or fork eintopf repository (Terraform and Ansible scripts):

Follow the instructions here:

To do so, follow the instructions here:

First, make sure you have ansible-vault configured:

Then, setup the infrastructure:

Also, the Virtual Machine called mgmt hosts a Docker container called devops. Inside this container you will find you Kubernetes cluster kubeconfig file, which is used with kubectl and also must be configured on your Gitlab CI variables to run the deployment test. More info here:

Clone or fork the following repo:

Setup the required variables to run the pipeline:

here
here
here
here
here
here
here
here
https://gitlab.com/tadeugr/eintopf
https://gitlab.com/tadeugr/eintopf#setup-aws-credentials
https://gitlab.com/tadeugr/eintopf#provision-the-infrastructure-with-terraform
https://gitlab.com/tadeugr/eintopf?nav_source=navbar#setup-ansible-vault-key
https://gitlab.com/tadeugr/eintopf?nav_source=navbar#setup-the-infrastructure-with-ansible
https://gitlab.com/tadeugr/eintopf?nav_source=navbar#how-to-use-kubectl
https://gitlab.com/tadeugr/rouladen
https://gitlab.com/tadeugr/rouladen#requirements
IAM Service
Add IAM user
iac user
policy
IAM tags
Create user
Credentials