如何备份Elasticsearch数据?
- By : Will
- Category : Elastic Stack
- Tags: Backup, Elasticsearch, 备份
1 基础知识
1.1 快照的介绍
– 快照可从Elasticsearch集群中备份数据
– 快照可拍摄单个索引或整个集群的快照
– 快照存储可选本地存储或远程存储(S3、HDFS、Azure、Google云存储和 存储库插件)
– 快照以增量数据的方式存储,允许用户执行频繁的快照备份
– 快照支持以API接口的方式触发快照还原
– 快照还原默认会还原所有索引,也支持还原特定索引或还原集群
– 快照还原时支持修改索引名称和选定的索引设置
– 快照还原到不同版本的集群需要确认兼容性
1.2 设置快照的步骤
– 注册存储库
– 手动首次快照
– 设置自动备份
– 还原快照
2 最佳实践
2.1 环境准备
2.1.1 准备集群环境
如果你没有Elasticsearch集群环境,建议你参阅以下文档建立集群环境,
2.1.2 准备存储
本章将使用samba服务作为备份的存储,如果你不熟悉此配置,请参阅以下章节,
需要注意的是,
– 以上假设你将存储挂载到“/backup/elasticsearch”(不建议使用本地存储)
– 集群的每个节点都需要挂载共同的samba存储
另外,节点的简略配置步骤如下,
In es[001-003],
yum -y install cifs-utils autofs id elasticsearch echo '/backup /etc/auto.backup' >> /etc/auto.master echo 'elasticsearch -fstype=cifs,rw,noperm,username=backup,password=backuppwd,uid=998,gid=996 ://10.168.0.49/backup' > /etc/auto.backup systemctl restart autofs systemctl enable autofs.service cd /backup/elasticsearch
2.2 注册存储库
2.2.1 配置存储信任
In es[001-003],
echo 'path.repo: /backup/elasticsearch' >> /etc/elasticsearch/elasticsearch.yml systemctl restart elasticsearch
2.2.2 注册存储库
In es001,
curl -X PUT "es001:9200/_snapshot/backup?pretty" -H 'Content-Type: application/json' -d' { "type": "fs", "settings": { "location": "/backup/elasticsearch", "max_snapshot_bytes_per_sec": "50mb", "max_restore_bytes_per_sec": "50mb", "compress": "true" } } '
注册成功后,你可以使用如下命令查看和删除存储库(谨慎执行删除,因为后面的步骤需要注册存储库),
curl -X GET "es001:9200/_snapshot/_all?pretty" curl -X DELETE "es001:9200/_snapshot/my_backup?pretty"
2.3 手动备份快照
2.3.1 执行手动快照备份
In es001,
curl -X PUT "es001:9200/_snapshot/backup/snapshot_1?wait_for_completion=true&pretty"
2.3.2 查询快照状态
In es001,
curl -X GET "es001:9200/_snapshot/backup/snapshot_1?pretty"
2.4 设置自动备份
curl -X PUT "es001:9200/_slm/policy/everyday-snapshots?pretty" -H 'Content-Type: application/json' -d' { "schedule": "0 30 1 * * ?", "name": "%3ceveryday-snap-%7bnow%2fd%7d%3e", "repository": "backup", "config": { "indices": ["*"] }, "retention": { "expire_after": "30d", "min_count": 5, "max_count": 50 } } '
– 参数“schedule”声明触发时间(语法与CronTab的一致)
– 参数“name”声明快照名称,例由“<everyday-snap-{now/d}>”URL转码,显示为 “<everyday-snap-2020.01.06>”样式
– 参数“repository”声明快照存储库名称
– 参数“config”声明快照请求配置参数
– 参数“indices”声明包括哪些数据流或索引,本例声明所有数据流和索引
– 参数“retention”声明快照过期策略
– 参数“expire_after”声明快照的过期时间过期
– 参数“min_count”声明快照最少保留的份数(忽略过期时间)
– 参数“max_count”声明快照最大保留的份数
然后,我们通过手动触发的方式测试,
curl -X POST "es001:9200/_slm/policy/everyday-snapshots/_execute?pretty"
设置完成后,可使用如下命令查看执行结果,
curl -X GET "es001:9200/_slm/policy/everyday-snapshots?human&pretty"
参阅文档
==============
快照和还原
————–
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/snapshot-restore.html
注册快照存储库
—————
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/snapshots-register-repository.html
手动备份快照
—————-
https://www.elastic.co/guide/en/elasticsearch/reference/7.6/snapshots-take-snapshot.html
自动备份快照
—————
https://www.elastic.co/guide/en/elasticsearch/reference/7.6/getting-started-snapshot-lifecycle-management.html
还原操作
————-
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/snapshots-restore-snapshot.html
没有评论