1 前言

一个问题，一篇文章，一出故事。
之前研究过logstash与交换机的继承，相见如下文档，

如何部署与交换机集成的Logstash？

但是发现生产中有很多交换机不支持修改上传日志的端口，于是需要根据上传的交换机所在的IP网段分流日志，于是整理本章节。
如果有需要，请熟悉包含Elastersearch和基本Logstash环境配置，

如何部署带安全认证的Logstash？

2 最佳实践

2.1 创建配置

vim /etc/logstash/conf.d/network.cmdschool.org_514.conf

加入如下配置，

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  syslog {
        port => 514
        type => "514"
  }
}

filter {
    if [type] == "514" {
        cidr {
            add_tag => [ "dg-network" ]
            address => [ "%{[host][ip]}" ] 
            network => [
                "10.168.0.0/24",
                "10.168.1.0/24"
            ]
        }

        cidr {
            add_tag => [ "ca-network" ]
            address => [ "%{[host][ip]}" ] 
            network => [
                "10.168.2.0/24",
                "10.168.3.0/24"
            ]
        }

        cidr {
            add_tag => [ "hk-network" ]
            address => [ "%{[host][ip]}" ] 
            network => [
                "10.168.4.0/24"
            ]
        }

        if "dg-network" in [tags] {
            mutate { add_tag => ["dg-network"] }
        } else if "ca-network" in [tags] {
            mutate { add_tag => ["ca-network"] }
        } else if "hk-network" in [tags] {
            mutate { add_tag => ["hk-network"] }
        } else {
            mutate { add_tag => ["network"] }
        }
    }
}

output {
    if [type] == "514" {
        if "dg-network" in [tags] {
            elasticsearch {
                hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
                index => "dg-network-%{+YYYY.MM.dd}"
                user => "elastic"
                password => "elasticpwd"
            }
        } else if "ca-network" in [tags] {
            elasticsearch {
                hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
                index => "ca-network-%{+YYYY.MM.dd}"
                user => "elastic"
                password => "elasticpwd"
            }
        } else if "hk-network" in [tags] {
            elasticsearch {
                hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
                index => "hk-network-%{+YYYY.MM.dd}"
                user => "elastic"
                password => "elasticpwd"
            }
        } else if "network" in [tags] {
            elasticsearch {
                hosts => ["http://azelasticsearch01:9200", "http://azelasticsearch02:9200", "http://azelasticsearch03:9200", "http://azelasticsearch04:9200", "http://azelasticsearch05:9200"]
                index => "network-%{+YYYY.MM.dd}"
                user => "elastic"
                password => "elasticpwd"
            }
        }
    }
}

配置的处理逻辑如下：
如果logstash接收到的日志type字段值为“514”,则调用filter cidr插件判断host.ip所属的网段，
– 如果属于dg网段的,用filter mutate插件“{ add_tag => [“dg-network”] }”打上标记。
– 如果属于ca网段的,用filter mutate插件“{ add_tag => [“ca-network”] }”打上标记。
– 如果属于hk网段的,用filter mutate插件“{ add_tag => [“hk-network”] }”打上标记。
– 如果匹配不到网段的,用filter mutate插件“{ add_tag => [“network”] }”打上标记。
然后，根据[tags]值把日志分流到不同的索引，
– 如果[tags]值为”dg-network”就存储到elk的“dg-network-%{+YYYY.MM.dd}”索引。
– 如果[tags]值为”ca-network”就存储到elk的“ca-network-%{+YYYY.MM.dd}”索引。
– 如果[tags]值为”hk-network”就存储到elk的“hk-network-%{+YYYY.MM.dd}”索引。
– 如果[tags]值为”network”就存储到elk的“network-%{+YYYY.MM.dd}”索引。
配置创建后，建议你使用如下命令测试配置语法，

/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/network.cmdschool.org_514.conf

2.2 重启服务使配置生效

systemctl restart logstash

参阅文档
==================
https://poe.com/s/myopb3dYAlFBKKBGqXut

Cidr filter plugin
————————-
https://www.elastic.co/guide/en/logstash/current/plugins-filters-cidr.html

Mutate filter plugin
————————–
https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html

Filter plugins
——————-
https://www.elastic.co/guide/en/logstash/current/filter-plugins.html