如何实现多个隔离的logstash管道?

Elastic Stack

1 前言

一个问题,一篇文章,一出故事。
笔者想实现一台logstash服务器上同时定义多个不同倾听端口的logstash管道,但是发现只要从任意一个输入(input)模块的端口输入的数据都会从多个输出(output)模块上出来,于是想要实现多个管道的隔离。

2 最佳实践

2.1 环境信息

如果你没有logstash环境,请参阅如下章节搭建,

如何部署带安全认证的Logstash?

2.1 定义多个隔离的管道

2.1.1 定义管道1

vim /etc/logstash/conf.d/dg-server.cmdschool.org_5045.conf 

加入如下配置,

input {
  beats {
    port => 5045
    type => "5045"
    ssl => true
    ssl_certificate_authorities => ["/etc/pki/tls/certs/ca.crt"]
    ssl_certificate => "/etc/pki/tls/certs/dg-server.cmdschool.org.crt"
    ssl_key => "/etc/pki/tls/private/dg-server.cmdschool.org.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  if [type] == "5045" {
    elasticsearch {
      hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
      index => "dg-server-%{+YYYY.MM.dd}"
      user => "elastic"
      password => "elasticpwd"
    }
  }
}

2.1.2 定义管道2

vim /etc/logstash/conf.d/ca-server.cmdschool.org_5046.conf 

加入如下配置,

input {
  beats {
    port => 5046
    type => "5046"
    ssl => true
    ssl_certificate_authorities => ["/etc/pki/tls/certs/ca.crt"]
    ssl_certificate => "/etc/pki/tls/certs/ca-server.cmdschool.org.crt"
    ssl_key => "/etc/pki/tls/private/ca-server.cmdschool.org.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  if [type] == "5046" {
    elasticsearch {
      hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
      index => "ca-server-%{+YYYY.MM.dd}"
      user => "elastic"
      password => "elasticpwd"
    }
  }
}

2.1.3 测试配置

/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/dg-server.cmdschool.org_5045.conf
/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/ca-server.cmdschool.org_5046.conf

2.1.4 重启服务使配置生效

systemctl restart logstash.service
没有评论

发表回复

Elastic Stack
如何部署与交换机集成的Logstash?

1 基础知识 一个问题,一篇文章,一出故事。 笔者最近需要配置接收交换机日志的Logstash管道, …

Bash
如何用Tigase监控Elasticsearch集群?

1 前言 一个问题,一篇文章,一出故事。 笔者生产中有一套Elasticsearch集群,笔者为了能 …

Bash
如何用Base Shell获取ES集群状态?

1 前言 一个问题,一篇文章,一出故事。 笔者想要通过Base Shell获取Elasticsear …