1 前言

一个问题，一篇文章，一出故事。
今天遇到需要设置Filebeat上传Java类型的日志，于是整理当前章节。

2 最佳实践

2.1 环境配置

2.1.1 准备Logstash环境

如何部署带安全认证的Logstash？

2.1.2 安装Filebeat

如何基于Oracle Linux 9.x部署Filebeat？

以上是安装Filebeat的步骤，本章的重点是如何设置并成功上传Java类型的日志。

2.2 设置Logstash

2.2.1 创建配置文件

vim /etc/logstash/conf.d/websso.cmdschool.org_5042.conf 

加入如下配置，

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5042
    type => "5042"
    ssl => false
    ssl_certificate_authorities => ["/etc/pki/tls/certs/ca.crt"]
    ssl_certificate => "/etc/pki/tls/certs/logstash.cmdschool.org.crt"
    ssl_key => "/etc/pki/tls/private/logstash.cmdschool.org.key"
    #ssl_verify_mode => "force_peer"
    ssl_verify_mode => "none"
  }
}

output {
  if [type] == "5042" {
    elasticsearch {
      hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
      index => "websso-%{+YYYY.MM.dd}"
      user => "elastic"
      password => "elasticpwd"
    }
  }
}

2.2.2 测试配置文件

/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/websso.cmdschool.org_5042.conf

2.2.3 重启服务是配置生效

systemctl restart logstash.service
systemctl status logstash.service

2.2.4 开放服务端口

firewall-cmd --permanent --add-port=5042/tcp
firewall-cmd --reload
firewall-cmd --list-all

2.3 设置Filebeat

2.3.1 创建配置文件

cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.default
vim /etc/filebeat/filebeat.yml

加入如下配置，

filebeat.inputs:
- type: log
  id: rhbk01.cmdschool.org
  enabled: true
  paths:
   - /var/log/keycloak/keycloak.log*
  #ignore_older: 7d
  multiline.type: pattern
  multiline.pattern: '^\d{4}-\d{2}-\d{2}'  # 假设日志条目以日期开始
  multiline.negate: true
  multiline.match: after
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
output.logstash:
  hosts: ["logstash.cmdschool.org:5042"]
  ssl.enabled: false
  ssl.verification_mode: none
  ssl.certificate_authorities: ["/etc/pki/tls/certs/ca.crt"]
  ssl.certificate: "/etc/pki/tls/certs/logstash.cmdschool.org.crt"
  ssl.key: "/etc/pki/tls/private/logstash.cmdschool.org.key"
processors:
  - add_host_metadata: ~

请注意加粗部分是本章针对Java日志的重点设置，然后，建议你使用如下命令测试配置，

filebeat test config

顺其然地，使用如下命令测试上传到ES的输出配置，

filebeat test output

2.3.2 禁用影响服务的模块

filebeat modules disable logstash

2.3.3 启动服务并设置服务自启动

systemctl start filebeat.service
systemctl enable filebeat.service
systemctl status filebeat.service

2.4 完整的设置流程

界面部分如果你不知道如下配置，请参阅如下完整的章节，

如何部署带安全认证的Filebeat与logstash集成？

参阅文档
==============
https://www.elastic.co/docs/reference/beats/filebeat/multiline-examples