如何部署带安全认证的Elasticsearch 8.x集群?

Big Data Framework

1 基础知识

1.1 集群的介绍

– Elasticsearch集群允许节点单点故障
– Elasticsearch集群出现单点故障时,正常的节点会替代非正常的节点继续提供服务

1.2 集群角色

– master role,即主角色,拥有该角色的节点为其他节点的主控节点
– data role,即数据角色,拥有该角色的节点负责保存集群的数据和相关的操作,可细分为如下角色,
— data_content role,即内容数据角色,拥有该角色的节点保存用户创建的内容
— data_hot role,即热数据角色,拥有该角色的节点会存储时间序列数据并具备快熟的数据读写能力(需硬件支持)
— data_warm role,即暖数据角色,拥有该角色的节点索引无需更新,但可查询
— data_cold role,即冷数据角色,拥有该角色的节点索引只读且访问频率较低(可用性能较低硬件)
– ingest role,即提取角色,拥有该角色的节点提供一个以上的管道处理能力
– ml role,即机器学习( Machine learning)角色,拥有该角色的节点拥有机器学习能力
– remote_cluster_client role,即远程节点角色,拥有该角色的节点拥有远程连接其他集群的能力
– transform,即转换角色,拥有该角色的节点具有转换能力(转换索引为汇总索引)。

1.3 集群的节点

1.3.1 集群的节点类型

– Master-eligible node,即及格主节点(拥有主角色),由其他节点选举产生的集群主控及格节点
– Data node,即数据节点(拥有数据角色),保存集群数据和相关操作(例如CRUD、search、aggregations)的节点
– Ingest node,即提取节点(拥有提取角色),提供一个以上的管道处理节点
– Remote-eligible node,即及格远程节点(拥有远程节点角色),承担远程连接其他集群的及格节点
– Machine learning node,即机器学习节点(拥有机器学习角色),承担机器学习功能的节点
– Transform node,即转换节点(拥有机器学习角色),承担转换(转换索引为汇总索引)功能的节点

1.3.2 弹性集群的节点要求

– 至少有一个被选举出及格主节点
– 每个角色至少分配到一个节点中
– 每个复制都必须有一个分片
基于以上,集群必须具备以下条件,
– 集群必须至少有三个节点(节点数量要求奇数)
– 每个角色必须分配到两个节点
– 每个分片至少有两个副本(一个主副本和一个以上的从副本)

2 最佳实践

2.1 集群环境配置

2.1.1 集群信息

Host Name = azelasticsearch[01-05].cmdschool.org
IP Address = 10.168.0.[100 – 104] OS = Oracle Linux 9.x x86_64
Elasticsearch Version = 8.14.1

2.1.2 节点的部署

如果你尚未部署节点,烦请参阅如下章节部署每个集群节点,

如何基于Oracle Linux 9.x部署Elasticsearch?

2.1.3 名称解析配置

In es[01-05],

echo '10.168.0.100 azelasticsearch01 azelasticsearch01.cmdschool.org' >> /etc/hosts
echo '10.168.0.101 azelasticsearch02 azelasticsearch02.cmdschool.org' >> /etc/hosts
echo '10.168.0.102 azelasticsearch03 azelasticsearch03.cmdschool.org' >> /etc/hosts
echo '10.168.0.103 azelasticsearch04 azelasticsearch04.cmdschool.org' >> /etc/hosts
echo '10.168.0.104 azelasticsearch05 azelasticsearch05.cmdschool.org' >> /etc/hosts

2.1.4 配置公钥认证(可选)

In azelasticsearch01,
请参阅如下章节配置公钥认证,

如何部署公钥认证?


然后,通过如下命令测试登录,

ssh azelasticsearch01
ssh azelasticsearch02
#...

2.1.5 清空节点数据

In azelasticsearch01,

ssh azelasticsearch01 systemctl stop elasticsearch
ssh azelasticsearch02 systemctl stop elasticsearch
ssh azelasticsearch03 systemctl stop elasticsearch
ssh azelasticsearch04 systemctl stop elasticsearch
ssh azelasticsearch05 systemctl stop elasticsearch
ssh azelasticsearch01 rm -rf /data/elasticsearch/*
ssh azelasticsearch02 rm -rf /data/elasticsearch/*
ssh azelasticsearch03 rm -rf /data/elasticsearch/*
ssh azelasticsearch04 rm -rf /data/elasticsearch/*
ssh azelasticsearch05 rm -rf /data/elasticsearch/*
ssh azelasticsearch01 rm -rf /etc/elasticsearch/certs
ssh azelasticsearch02 rm -rf /etc/elasticsearch/certs
ssh azelasticsearch03 rm -rf /etc/elasticsearch/certs
ssh azelasticsearch04 rm -rf /etc/elasticsearch/certs
ssh azelasticsearch05 rm -rf /etc/elasticsearch/certs

需要注意的是,该方法适用于没有任何数据的集群

2.2 集群配置前的准备

In azelasticsearch01,

2.2.1 创建CA证书

/usr/share/elasticsearch/bin/elasticsearch-certutil ca

具体向导如下,

#...
By default the 'ca' mode produces a single PKCS#12 output file which holds:
#...
Please enter the desired output file [elastic-stack-ca.p12]: 
Enter password for elastic-stack-ca.p12 : 

2.2.2 创建SSL证书

/usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

具体向导如下,

#...
By default the 'cert' mode produces a single PKCS#12 output file which holds:
#...
Enter password for CA (elastic-stack-ca.p12) : 
Please enter the desired output file [elastic-certificates.p12]: 
Enter password for elastic-certificates.p12 : 

Certificates written to /usr/share/elasticsearch/elastic-certificates.p12
#...

2.2.3 部署证书

ssh azelasticsearch01 mkdir -p /etc/elasticsearch/certs
ssh azelasticsearch02 mkdir -p /etc/elasticsearch/certs
ssh azelasticsearch03 mkdir -p /etc/elasticsearch/certs
ssh azelasticsearch04 mkdir -p /etc/elasticsearch/certs
ssh azelasticsearch05 mkdir -p /etc/elasticsearch/certs
scp /usr/share/elasticsearch/elastic-certificates.p12 azelasticsearch01:/etc/elasticsearch/certs/
scp /usr/share/elasticsearch/elastic-certificates.p12 azelasticsearch02:/etc/elasticsearch/certs/
scp /usr/share/elasticsearch/elastic-certificates.p12 azelasticsearch03:/etc/elasticsearch/certs/
scp /usr/share/elasticsearch/elastic-certificates.p12 azelasticsearch04:/etc/elasticsearch/certs/
scp /usr/share/elasticsearch/elastic-certificates.p12 azelasticsearch05:/etc/elasticsearch/certs/
ssh azelasticsearch01 chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12
ssh azelasticsearch02 chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12
ssh azelasticsearch03 chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12
ssh azelasticsearch04 chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12
ssh azelasticsearch05 chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12

2.2.4 创建KeyStore

/usr/share/elasticsearch/bin/elasticsearch-keystore create

需要注意的是,“KeyStore”创建的密码应该与证书“elastic-certificates”创建的密码一致,具体向导如下,

An elasticsearch keystore already exists. Overwrite? [y/N]y
Created elasticsearch keystore in /etc/elasticsearch/elasticsearch.keystore

部署KeyStore,

scp /etc/elasticsearch/elasticsearch.keystore azelasticsearch02:/etc/elasticsearch/elasticsearch.keystore
scp /etc/elasticsearch/elasticsearch.keystore azelasticsearch03:/etc/elasticsearch/elasticsearch.keystore
scp /etc/elasticsearch/elasticsearch.keystore azelasticsearch04:/etc/elasticsearch/elasticsearch.keystore
scp /etc/elasticsearch/elasticsearch.keystore azelasticsearch05:/etc/elasticsearch/elasticsearch.keystore

2.3 配置集群

2.3.1 修改配置

In azelasticsearch01,

vim /etc/elasticsearch/elasticsearch.yml

配置修改如下,

cluster.name: esCluster01
node.name: azelasticsearch01
node.attr.rack: r1
path.data: /data/elasticsearch
path.logs: /var/log/elasticsearch
#bootstrap.memory_lock: true
network.host: _ens192:ipv4_
http.port: 9200
discovery.seed_hosts: ["azelasticsearch01", "azelasticsearch02", "azelasticsearch03"]
cluster.initial_master_nodes: ["azelasticsearch01", "azelasticsearch02", "azelasticsearch03"]
action.destructive_requires_name: true
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: false
  #keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12
http.host: 0.0.0.0

– 参数“cluster.name”声明集群的名称
– 参数“node.name”声明节点名称
– 参数“node.attr.rack”声明节点所在的机架
– 参数“path.data”声明节点数据存储的路径
– 参数“path.logs”声明节点日志的输出路径
– 参数“network.host”声明节点的主机地址(支持特殊设置,例如“_ens192_”或_ens192:ipv4_”等)
– 参数“http.port”声明http服务的端口
– 参数“discovery.seed_hosts”声明可见的集群成员主机名称,用户形成集群(推荐三个)
– 参数“cluster.initial_master_nodes”声明集群主节点的列表
– 参数“action.destructive_requires_name”声明破坏性的请求只能通过集群初始化或滚动重启来设置
然后,你可以使用如下命令部署配置,

scp /etc/elasticsearch/elasticsearch.yml azelasticsearch02:/etc/elasticsearch/
scp /etc/elasticsearch/elasticsearch.yml azelasticsearch03:/etc/elasticsearch/
scp /etc/elasticsearch/elasticsearch.yml azelasticsearch04:/etc/elasticsearch/
scp /etc/elasticsearch/elasticsearch.yml azelasticsearch05:/etc/elasticsearch/
ssh azelasticsearch02 "sed -i 's/node.name: azelasticsearch01/node.name: azelasticsearch02/g' /etc/elasticsearch/elasticsearch.yml"
ssh azelasticsearch03 "sed -i 's/node.name: azelasticsearch01/node.name: azelasticsearch03/g' /etc/elasticsearch/elasticsearch.yml"
ssh azelasticsearch04 "sed -i 's/node.name: azelasticsearch01/node.name: azelasticsearch04/g' /etc/elasticsearch/elasticsearch.yml"
ssh azelasticsearch05 "sed -i 's/node.name: azelasticsearch01/node.name: azelasticsearch05/g' /etc/elasticsearch/elasticsearch.yml"

2.3.2 启动服务

ssh azelasticsearch01 systemctl start elasticsearch
ssh azelasticsearch02 systemctl start elasticsearch
ssh azelasticsearch03 systemctl start elasticsearch
ssh azelasticsearch04 systemctl start elasticsearch
ssh azelasticsearch05 systemctl start elasticsearch

2.3.3 检查端口倾听

In azelasticsearch[01-05],

for i in `pgrep -u elasticsearch java`; do netstat -anp | grep $i; done

其中一个节点显示如下,

unix  3      [ ]         STREAM     CONNECTED     69134    8127/java            
tcp6       0      0 10.10.200.100:9300      :::*                    LISTEN      8185/java           
tcp6       0      0 :::9200                 :::*                    LISTEN      8185/java           
unix  3      [ ]         STREAM     CONNECTED     71765    8185/java            
unix  3      [ ]         STREAM     CONNECTED     71764    8185/java            
unix  2      [ ]         STREAM     CONNECTED     71762    8185/java 

注:以上可见“9200”与“9300”端口已经倾听于主机的接口地址,允许与非本机通讯。

2.3.4 开启服务端口

In azelasticsearch[01-05],

firewall-cmd --permanent --add-port 9200/tcp --add-port 9300/tcp
firewall-cmd  --reload
firewall-cmd  --list-all

2.3.5 初始化集群密码

In azelasticsearch01,

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

具体向导如下,

#...
Please confirm that you would like to continue [y/N]y
#...
Enter password for [elastic]: elasticpwd
Reenter password for [elastic]: elasticpwd
Enter password for [apm_system]: apmpwd
Reenter password for [apm_system]: apmpwd
Enter password for [kibana_system]: kibanapwd
Reenter password for [kibana_system]: kibanapwd
Enter password for [logstash_system]: logstashpwd
Reenter password for [logstash_system]: logstashpwd
Enter password for [beats_system]: beatspwd
Reenter password for [beats_system]: beatspwd
Enter password for [remote_monitoring_user]: remote_monitoringpwd
Reenter password for [remote_monitoring_user]: remote_monitoringpwd
#...

2.4 检查集群

2.4.1 测试集群连接

In azelasticsearch01,

curl -u elastic:elasticpwd http://azelasticsearch01:9200 http://azelasticsearch02:9200 http://azelasticsearch03:9200

可见如下显示,

{
  "name" : "azelasticsearch01",
  "cluster_name" : "esCluster01",
  "cluster_uuid" : "SgTuCp2zRyS1bEaDzidorw",
  "version" : {
    "number" : "8.14.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "93a57a1a76f556d8aee6a90d1a95b06187501310",
    "build_date" : "2024-06-10T23:35:17.114581191Z",
    "build_snapshot" : false,
    "lucene_version" : "9.10.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
{
  "name" : "azelasticsearch02",
  "cluster_name" : "esCluster01",
  "cluster_uuid" : "SgTuCp2zRyS1bEaDzidorw",
  #...
}
{
  "name" : "azelasticsearch03",
  "cluster_name" : "esCluster01",
  "cluster_uuid" : "SgTuCp2zRyS1bEaDzidorw",
  #...
}

2.4.2 检查集群健康状态

In azelasticsearch01,

curl -u elastic:elasticpwd -X GET "azelasticsearch01:9200/_cluster/health?pretty" "azelasticsearch02:9200/_cluster/health?pretty" "azelasticsearch03:9200/_cluster/health?pretty"

可见如下显示,


  "cluster_name" : "esCluster01",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 5,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
{
  "cluster_name" : "esCluster01",
  "status" : "green",
    #...
}
{
  "cluster_name" : "esCluster01",
  "status" : "green",
  #...
}

2.5 集成elastic其他服务

2.5.1 集成Kibana

如何部署带安全认证的Kibana?

2.5.2 集成Filebeat

如何配置Filebeat集成Elasticsearch集群?

2.5.3 集成Logstash

如何部署带安全认证的Logstash?

如何部署Filebeat与logstash集成?

参阅文档
===========================

集群支持矩阵
—————–
https://www.elastic.co/cn/support/matrix#matrix_compatibility

集群的安装配置
——————
https://www.elastic.co/guide/en/elasticsearch/reference/8.11/rpm.html

集群健康API
—————
https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html

配置文件解析
—————
https://www.elastic.co/guide/en/elasticsearch/reference/7.x/important-settings.html

elasticearch集群索引
———————-
https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability.html

elasticsearch文档索引
——————
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html

数据角色
———-
https://www.elastic.co/guide/en/elasticsearch/reference/8.11/data-tier-shard-filtering.html

节点角色的分配设置
———–
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html

elastic github
————–
https://github.com/elastic

没有评论

发表回复

Elastic Stack
如何部署与交换机集成的Logstash?

1 基础知识 一个问题,一篇文章,一出故事。 笔者最近需要配置接收交换机日志的Logstash管道, …

Bash
如何用Tigase监控Elasticsearch集群?

1 前言 一个问题,一篇文章,一出故事。 笔者生产中有一套Elasticsearch集群,笔者为了能 …

Bash
如何用Base Shell获取ES集群状态?

1 前言 一个问题,一篇文章,一出故事。 笔者想要通过Base Shell获取Elasticsear …