[ELK Stack] 로그 파일을 활용하여 모니터링 환경 구축하기

안녕하세요. yeTi입니다.

오늘은 로그 파일을 활용하여 모니터링 환경을 ELK (ElasticSearch + Logstash + Kibana) 를 이용하여 구축해보도록 하겠습니다.

설치 환경

- OS : CentOS release 6.9 (Final)

- Filebeat : 6.3.2-1 64bit

- Logstash : 6.3.2

- Elasticsearch : 6.3.0

1. logstash 설치

- 참고 URL : https://www.elastic.co/guide/en/logstash/current/installing-logstash.html#package-repositories

- ] rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

- ] vi /etc/yum.repos.d/logstash.repo

- [logstash-6.x]

name=Elastic repository for 6.x packages

baseurl=https://artifacts.elastic.co/packages/6.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

- ] yum install logstash

- Input Plugin 설치

- cd /usr/share/logstash

- bin/logstash-plugin install logstash-input-beats

2. logstash 설정

- vi /etc/logstash/conf.d/filebeat.conf

input {

  beats {

port => 19606

client_inactivity_timeout=>120

  }

}

filter {

  json {

source => "message"

remove_field => ["@version","@timestamp","path","host","type"

 ,"message","tags","source","offset","beat"

 ,"input","prospector"

]

  }

}

output {

  elasticsearch {

hosts => ["localhost:19202"]

index => "api"

document_type => "log"

  }

}

3. logstash 실행

- initctl start logstash

4. logstash 로깅

- tail -f /var/log/logstash/logstash-plain.log

5. filebeat 설치

- sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

- vi /etc/yum.repos.d/elastic.repo

[elastic-6.x]

name=Elastic repository for 6.x packages

baseurl=https://artifacts.elastic.co/packages/6.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

- sudo yum install filebeat

- sudo chkconfig --add filebeat

6. filebeat 설정

- vi /etc/filebeat/filebeat.yml

filebeat.inputs:

- type: log

  enabled: true

  paths:

    - /var/log/*.log

output.logstash:

  hosts: ["localhost:19606"]

7. filebeat 실행

- service filebeat start

8. filebeat 로깅

- tail -f /var/log/filebeat/filebeat

관련 글

• 2018/11/28 - [IT/Monitoring] - [Kibana] 가능한 모니터링 종류 나열

참고 문헌

• findstar님의 블로그 : https://findstar.pe.kr/2018/05/28/install-and_configuration-filebeat-logstash/
• Elastic 문서 : https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html