DevOps Buzz
  • Initial page
  • About me
  • Ansible
    • Cheat Sheet
    • Dynamic inventory
  • AKS
    • Cheat Sheet
    • Autoscale
    • Backup
    • Dashboard
    • Ingress
    • Node management
    • News
    • Persistent Volumes
  • Arch Linux
    • Docker
    • Install
    • Network
    • VirtualBox guest
  • Azure
    • API
    • CLI/Powershell
    • DevOps
      • Cheat Sheet
    • Application Gateway
    • Tools
    • Sops
  • AWS
    • Cheat Sheet
    • Tools
    • Cognito with Amplify and React
  • Bash / Shell
    • Cheat Sheet
    • Tools
  • Bitbucket
    • Cheat Sheet
  • CSS
    • Cheat Sheet
  • Distros
    • Manjaro
  • Docker
    • Cheat Sheet
    • ELK
    • Ubuntu NoVNC
    • Tools
    • Known errors and solutions
  • Elasticsearch
    • Cheat Sheet
    • Tools
  • ELK
    • Cheat Sheet
  • emacs
    • Cheat sheet
  • Gatekeeper (OPA)
    • Cheat Sheet
    • Developer Quick Start
  • GCP
    • Cheat Sheet
  • General
    • Tools
    • News
  • Git
    • Cheat Sheet
    • Tools
  • Golang
    • Cheat Sheet
  • Guidelines / Standards
    • Cheat Sheet
  • i3wm
    • Cheat Sheet
  • Ipsec
    • Cheat sheet
  • Istio
    • Cheat Sheet
  • Kind
    • Cheat Sheet
  • Kops
    • Cheat Sheet
  • Kubeadm
    • Cheat Sheet
    • Change serviceSubnet CIDR
    • Setup cluster
      • Azure
      • Bare-metal
      • Vagrant + VirtualBox
    • Multi master
    • Known errors and solutions
  • Kubernetes
    • Kubectl Cheat Sheet
    • etcd Cheat Sheet
    • Tools
    • News
    • Deployments
      • Deployment examples
      • Blue/Green Deployment
      • Canary Deployment
    • Dashboard
    • ELK
    • Helm
    • Ingress
    • logz.io
    • Minikube
    • Monitoring
    • Node Management
    • Operators
    • Security
    • Volumes
    • Networking
    • kube-controller-manager
      • Node crash recovery
    • Known errors and solutions
  • Lumen
    • Cheat Sheet
  • MACOS
    • Cheat Sheet
    • Qemu
    • Zsh
  • microk8s
    • Cheat Sheet
  • MongoDB
    • Cheat Sheet
    • Tools
  • MySQL
    • Cheat Sheet
  • Network
    • Tools
    • WDS - Wireless Distribution System
    • Expose server under NAT
  • nvim
    • Cheat Sheet
  • Openvpn
    • OpenVPN server on Ubuntu 18.06
    • Stunnel
  • PHP
    • Composer
  • Prometheus
    • Tools
  • Python
    • Cheat Sheet
    • Pydantic
    • Tools / Modules
    • Virtualenv
  • RabbitMQ
    • Cheat Sheet
  • ReactJS
    • Fixes
    • For beginners
  • Ruby
    • Cheat Sheet
    • Rails
    • rvm
  • Rundeck
    • Cheat Sheet
  • Rust
    • Cheat Sheet
  • Squid
    • Setup server
    • Expose NAT server
  • SRE
    • Cheat Sheet
  • SSH
    • Passwordless auth with RSA key
    • Reverse tunnel
    • Cheat Sheet
  • SSL
    • Cheat Sheet
    • certbot
  • STACK SETUP
    • Using EKS and Gitlab CI to deploy applications
  • Terraform
    • Cheat Sheet
    • Tools
  • Tmux
    • Cheat Sheet
  • Tor
    • Cheat Sheet
  • Ubuntu
    • Cheat Sheet
  • Vagrant
    • Cheat Sheet
  • VirtualBox
    • Cheat Sheet
  • Windows
    • Windows Docker
    • Fingerprint
    • SSH Client
    • Tools
    • VirtualBox
    • WSL
Powered by GitBook
On this page
  • Deploy using Docker
  • Elasticsearch
  • Kibana
  • Deploy using docker-compose
  • Deploy Filebeat
  1. Docker

ELK

Elastichsearch, Logstash and Kibana

Deploy using Docker

Elasticsearch

The vm.max_map_count kernel setting needs to be set to at least 262144 for production use

sysctl -w vm.max_map_count=262144

Deploy elasticsearch:

docker run \
  -tid \
  --name elasticsearch \
  -p 9200:9200 \
  -p 9300:9300 \
  docker.elastic.co/elasticsearch/elasticsearch:6.7.0

Kibana

docker run \
  -tid \
  --name kibana \
  --net=host \
  -p 5601:5601 \
  docker.elastic.co/kibana/kibana:6.7.0

If you want to mount the config file, use:

--volume="/your-path/kibana.yml:/usr/share/kibana/config/kibana.yml:ro" \

Deploy using docker-compose

Run:

git clone https://github.com/deviantony/docker-elk.git
cd docker-elk
docker-compose up

Deploy Filebeat

Create the config file:

sudo nano filebeat.docker.yml

Paste

filebeat:
  config:
    modules:
      path: ${path.config}/modules.d/*.yml
      reload.enabled: false
    
  autodiscover:
    providers:
      - type: docker
        hints.enabled: true

  prospectors:  
  - input_type: log
    paths:
      - /var/log/*.log

output.logstash:
  hosts: ["localhost:5044"]
  
#output.elasticsearch:
#  hosts: ["http://<HOST>:<IP>"]

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB

Run the container:

docker run -tid \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  docker.elastic.co/beats/filebeat:6.7.0 filebeat

Go to Management -> Index Patterns and create your index.

PreviousCheat SheetNextUbuntu NoVNC

Last updated 6 years ago

Access Kibana:

http://localhost:5601