如何部署带安全认证的Filebeat与logstash集成?
- By : Will
- Category : Elastic Stack

1 前言
一个问题,一篇文章,一出故事。
本章将整理Filebeat与Logstash集成,然后通过Logstash将日志保存到Elasticsearch中。
2 最佳实践
2.1 环境信息
2.1.1 主机信息
Host Name = azelasticsearch[01-05].cmdschool.org
IP Address = 10.168.0.[100 – 104] OS = Oracle Linux 9.x x86_64
Elasticsearch Version = 8.14.1
另外,如果你需要部署Elasticsearch集群环境,请参阅以下文档,
Host Name = kibana.cmdschool.org
OS = Oracle Linux 9.x x86_64
IP Address = 10.168.0.99
另外,如果你需要部署Kibana集群环境,请参阅以下文档,
Host Name = logstash.cmdschool.org
IP Address = 10.168.0.103 OS = Oracle Linux 9.x x86_64
Elasticsearch Version = 8.14.3
另外,如果你需要部署Logstash环境,请参阅以下文档,
Host Name = azfilebeat.cmdschool.org
OS = Oracle Linux 9.x x86_64
IP Address = any
另外,如果你需要部署Filebeat,请参阅以下文档,
2.2.2 配置名称解析
In All,
vim /etc/hosts
加入如下配置,
10.168.0.100 azelasticsearch01 azelasticsearch01.cmdschool.org 10.168.0.101 azelasticsearch02 azelasticsearch02.cmdschool.org 10.168.0.102 azelasticsearch03 azelasticsearch03.cmdschool.org 10.168.0.102 azelasticsearch04 azelasticsearch04.cmdschool.org 10.168.0.102 azelasticsearch05 azelasticsearch05.cmdschool.org 10.168.0.103 logstash ogstash.cmdschool.org 10.168.0.99 azkibana azkibana.cmdschool.org
注:以上配置仅用于测试,正式环境请使用DNS代替
2.3 配置Filebeat
2.3.1 确认logstash颁发的证书存在
ls /etc/pki/tls/certs/ca.crt ls /etc/pki/tls/certs/logstash.cmdschool.org.crt ls /etc/pki/tls/private/logstash.cmdschool.org.key
2.3.2 修改配置文件
cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.default vim /etc/filebeat/filebeat.yml
配置修改如下,
filebeat.inputs: - type: log id: filebeat.cmdschool.org enabled: true paths: - /var/log/maillog filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false setup.template.settings: index.number_of_shards: 3 output.logstash: hosts: ["logstash.cmdschool.org:5044"] ssl.certificate_authorities: ["/etc/pki/tls/certs/ca.crt"] ssl.certificate: "/etc/pki/tls/certs/logstash.cmdschool.org.crt" ssl.key: "/etc/pki/tls/private/logstash.cmdschool.org.key" processors: - add_host_metadata: ~
然后,你可以运行以下命令测试配置,
filebeat test config
然后,你可以运行以下命令测试Filebeat与Elasticsearch之间的配置,
filebeat test output
2.3.3 禁用logstash模块
filebeat modules disable logstash
按照我们的经验,logstash模块开启会影响日志的上传,所以你需要禁用该模块,然后使用如下命令确认,
filebeat modules list
可见如下输出,
Enabled: Disabled: activemq apache auditd #... logstash #...
2.3.4 重启服务使配置生效
systemctl restart filebeat.service
2.3.5 配置日志视图
echo "test" | s-nail -s "test" -r admin@cmdschool.org will@cmdschool.org
由于我们监控的是邮件日志,因此我们使用以上命令触发日志,
然后,我们使用Kibana确认自动创建的日志索引,
http://kibana.cmdschool.org:5601/app/management/data/index_management/indices
如上图所示,
单击【Management】->【Data】->【Index Managerment】可见看到因为触发日志而产生的索引
如上图所示,
单击【Management】->【Kibana】->【Data Views】->【Create data view】
在“Name”中填写数据视图名称,本范例为“filebeat”
在“Index pattern”中填写索引模式,本范例为“filebeat-*”
如上图所示,
单击【Analytics】->【Discover】即可查看服务产生的日志。
参阅文档
================
软件的介绍
——————
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html
Filebeat工作原理
——————
https://www.elastic.co/guide/en/beats/filebeat/current/how-filebeat-works.html
Filebeat安装
—————–
https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html
Filebeat与Elasticsearch集群直连配置方法
—————————————-
http://kibana.cmdschool.org:5601/app/home#/tutorial/logstashLogs
Filebeat的配置
——————
https://www.elastic.co/guide/en/beats/filebeat/current/howto-guides.html
Elastic Kibana postfix
———————–
https://github.com/ActionScripted/elastic-kibana-postfix
filebeat module postfix
————————-
https://github.com/maurom/filebeat-module-postfix
与logstash的安全验证
———————–
https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ssl-logstash.html
https://www.elastic.co/guide/en/beats/filebeat/current/securing-communication-elasticsearch.html
https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/17
没有评论