如何在RHEL8 编译安装Gluster?

GlusterFS

1 前言

笔者之前使用yum部署过Gluster的服务,但最近笔者发现官方已经不再提供旧版本的rpm包下载,而是将各个版本作为CentOS分发软件包的一部分。
如果你使用CentOS部署Gluster,这将不受任何影响,但如果你想使用RHEL系统,那么旧版本的Gluster只能从红帽官方获取(购买订阅),否则无法获取致同一版本的Gluster,基于跨平台的需求,笔者决定尝试自己编译软件包。
另外,由于本章重点是编译安装,故而本章不介绍Gluster的基础知识和集群的配置,如果你需要,请参考以下章节,
https://www.cmdschool.org/archives/6056
如果你需要使用yum安装,请参阅如下章节,
https://www.cmdschool.org/archives/6080

2 编译安装Gluster

2.1 系统环境

2.1.1 环境信息

OS = RHEL 8.0 x86_64
ip addresses = any
host name = any

2.1.2 安装编译环境

yum -y install gcc gcc-c++ make expat-devel
yum -y install autoconf automake libtool

2.1.3 关闭SELinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

2.2 软件环境

2.2.1 下载编译的软件包

cd ~
wget https://download.gluster.org/pub/gluster/glusterfs/6/6.3/glusterfs-6.3.tar.gz

2.2.2 解压编译的软件包

cd ~
tar -xf glusterfs-6.3.tar.gz

2.2 最佳实践

2.2.1 创建编译环境

cd ~/glusterfs-6.3
./autogen.sh

2.2.2 预编译软件包

cd ~/glusterfs-6.3
./configure --bindir=/usr/bin/ \
            --sbindir=/usr/sbin/ \
            --libexecdir=/usr/libexec/ \
            --sysconfdir=/etc/ \
            --libdir=/usr/lib64/ \
            --includedir=/usr/include/ \
            --datarootdir=/usr/share/ \
            --infodir=/usr/share/info/ \
            --localedir=/usr/share/locale/ \
            --mandir=/usr/share/man/ \
            --docdir=/usr/share/doc/glusterfs/ \
            --with-systemddir=/usr/lib/systemd/system/

如果遇到以下错误提示,

configure: error: `rpcgen` not found, glusterfs needs `rpcgen` exiting..

你可能需要参照以下链接解决依赖关系,
https://www.cmdschool.org/archives/5968
如果遇到以下错误提示,

configure: error: Flex or lex required to build glusterfs.

如果遇到以下错误提示,

yum install -y flex

如果遇到以下错误提示,

configure: error: GNU Bison required to build glusterfs.

你可能需要使用以下命令解决依赖关系,

yum install -y bison

如果遇到以下错误提示,

configure: error: OpenSSL crypto library is required to build glusterfs

你可能需要使用以下命令解决依赖关系,

yum install -y openssl-devel

如果遇到以下错误提示,

configure: error: a Linux compatible libuuid is required to build glusterfs

你可能需要使用以下命令解决依赖关系,

yum install -y libuuid-devel

如果遇到以下错误提示,

configure: error: Support for POSIX ACLs is required

你可能需要使用以下命令解决依赖关系,

yum install -y libacl-devel

如果遇到以下错误提示,

configure: error: libxml2 devel libraries not found

你可能需要使用以下命令解决依赖关系,

yum install -y libxml2-devel

如果遇到以下错误提示,

configure: error:
            ---------------------------------------------------------------------------------
            libtirpc (and/or ipv6-default) were enabled but libtirpc-devel is not installed
            and there were no legacy glibc rpc headers and library to fall back to.
            ---------------------------------------------------------------------------------

你可能需要使用以下命令解决依赖关系,

yum install -y libtirpc-devel

如果遇到以下错误提示,

configure: error: liburcu-bp not found

如果遇到以下错误提示,

../../../../contrib/userspace-rcu/rculist-extra.h:33:6: error: redefinition of ‘cds_list_add_tail_rcu’
 void cds_list_add_tail_rcu(struct cds_list_head *newp,
      ^~~~~~~~~~~~~~~~~~~~~
In file included from glusterd-rcu.h:15,
                 from glusterd-sm.h:26,
                 from glusterd.h:28,
                 from glusterd.c:19:
/usr/local/include/urcu/rculist.h:44:6: note: previous definition of ‘cds_list_add_tail_rcu’ was here
 void cds_list_add_tail_rcu(struct cds_list_head *newp,
      ^~~~~~~~~~~~~~~~~~~~~
make[5]: *** [Makefile:736: glusterd_la-glusterd.lo] Error 1
make[4]: *** [Makefile:456: all-recursive] Error 1
make[3]: *** [Makefile:456: all-recursive] Error 1
make[2]: *** [Makefile:462: all-recursive] Error 1
make[1]: *** [Makefile:586: all-recursive] Error 1
make: *** [Makefile:481: all] Error 2

你可能需要参照以下链接解决依赖关系,
https://www.cmdschool.org/archives/6022
如果看到如下提示信息,

[...]
GlusterFS configure summary
===========================
FUSE client          : yes
Infiniband verbs     : no
epoll IO multiplex   : yes
fusermount           : yes
readline             : no
georeplication       : yes
Linux-AIO            : no
Enable Debug         : no
Enable ASAN          : no
Enable TSAN          : no
Use syslog           : yes
XML output           : yes
Unit Tests           : no
Track priv ports     : yes
POSIX ACLs           : yes
SELinux features     : yes
firewalld-config     : no
Events               : yes
EC dynamic support   : x64 sse avx
Use memory pools     : yes
Nanosecond m/atimes  : yes
Server components    : yes
Legacy gNFS server   : no
IPV6 default         : no
Use TIRPC            : yes
With Python          : 3.6
Cloudsync            : no

你可能需要使用如下命令解决依赖关系后再次运行“configure”命令(no表示尚未瞒住依赖关系),

yum install -y rdma-core-devel readline-devel libaio-devel

安装后再次运行“configure”命令显示如下,

[...]
GlusterFS configure summary
===========================
FUSE client          : yes
Infiniband verbs     : yes
epoll IO multiplex   : yes
fusermount           : yes
readline             : yes
georeplication       : yes
Linux-AIO            : yes
Enable Debug         : no
Enable ASAN          : no
Enable TSAN          : no
Use syslog           : yes
XML output           : yes
Unit Tests           : no
Track priv ports     : yes
POSIX ACLs           : yes
SELinux features     : yes
firewalld-config     : no
Events               : yes
EC dynamic support   : x64 sse avx
Use memory pools     : yes
Nanosecond m/atimes  : yes
Server components    : yes
Legacy gNFS server   : no
IPV6 default         : no
Use TIRPC            : yes
With Python          : 3.6
Cloudsync            : no

2.2.3 编译并安装软件包

make
make install

2.3 配置Gluster

2.3.1 服务进程启动测试

/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level WARNING -l /var/log/gluster/gluster.log

如果你需要手动结束服务,请使用如下命令,

kill 2 `pgrep glusterd`

启动完毕后,我们建议你通过如下命令检查是否有错误,

cat /var/log/gluster/gluster.log

如果你遇到如下错误提示,

[2019-07-03 07:54:20.571854] W [MSGID: 101095] [xlator.c:374:xlator_dynload] 0-xlator: liburcu-bp.so.1: cannot open shared object file: No such file or directory
[2019-07-03 07:54:20.571904] E [MSGID: 101002] [graph.y:213:volume_type] 0-parser: Volume 'management', line 2: type 'mgmt/glusterd' is not valid or not found on this machine
[2019-07-03 07:54:20.571930] E [MSGID: 101019] [graph.y:321:volume_end] 0-parser: "type" not specified for volume management
[2019-07-03 07:54:20.571964] E [MSGID: 100026] [glusterfsd.c:2636:glusterfs_process_volfp] 0-: failed to construct the graph
[2019-07-03 07:54:20.572191] W [glusterfsd.c:1570:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xac) [0x40a1bc] -->/usr/sbin/glusterd(glusterfs_process_volfp+0x243) [0x40a0f3] -->/usr/sbin/glusterd(cleanup_and_exit+0x4b) [0x4062fb] ) 0-: received signum (-1), shutting down

你需要通过如下步骤注册库文件,

find /usr/ -name \*liburcu-bp.so\*

可见如下提示,

/usr/lib64/liburcu-bp.so.6
/usr/lib64/liburcu-bp.so.6.0.0
/usr/local/lib/liburcu-bp.so.1.0.0
/usr/local/lib/liburcu-bp.so.1
/usr/local/lib/liburcu-bp.so

确认库文件是否已经注册

ldconfig -v | grep liburcu-bp.so

命令显示如下,

        liburcu-bp.so.6 -> liburcu-bp.so.6.0.0

由以上命令可知“/usr/local/lib/”目录下的lib文件尚未注册,通过以下命令注册该库文件,

echo '/usr/local/lib' > /etc/ld.so.conf.d/userspace-rcu.conf

再次运行命令确定当前已经注册的lib文件版本,

ldconfig -v | grep liburcu-bp.so

命令显示如下,

        liburcu-bp.so.1 -> liburcu-bp.so.1.0.0
        liburcu-bp.so.6 -> liburcu-bp.so.6.0.0

如果你遇到如下错误提示,

[2019-07-04 03:53:55.500015] W [MSGID: 103071] [rdma.c:4472:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel creation failed [No such device]
[2019-07-04 03:53:55.500047] W [MSGID: 103055] [rdma.c:4782:init] 0-rdma.management: Failed to initialize IB Device
[2019-07-04 03:53:55.500053] W [rpc-transport.c:363:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed
[2019-07-04 03:53:55.500152] W [rpcsvc.c:1985:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2019-07-04 03:53:55.500159] E [MSGID: 106244] [glusterd.c:1785:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2019-07-04 03:53:57.177642] E [MSGID: 101032] [store.c:447:gf_store_handle_retrieve] 0-: Path corresponding to /var/lib/glusterd/glusterd.info. [No such file or directory]
[...]

你可能需要执行以下命令修正该错误提示,

rm -rf /var/lib/glusterd/

2.3.2 修改服务控制脚本

vim /usr/lib/systemd/system/glusterd.service

脚本修改如下,

[Unit]
Description=GlusterFS, a clustered file-system server
Documentation=man:glusterd(8)
Requires=rpcbind.service
After=network.target rpcbind.service
Before=network-online.target

[Service]
Type=forking
PIDFile=/var/run/glusterd.pid
LimitNOFILE=65536
EnvironmentFile=-/etc/sysconfig/glusterd
ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid -L $GLUSTERD_LOGLEVEL -l $GLUSTERD_LOGFILE $GLUSTERD_OPTIONS
KillMode=process
SuccessExitStatus=15

[Install]
WantedBy=multi-user.target

增加以上脚本所需的环境变量,

vim /etc/sysconfig/glusterd

加入如下代码,

# Change the glusterd service defaults here.
# See "glusterd --help" outpout for defaults and possible values.

GLUSTERD_LOGFILE="/var/log/gluster/gluster.log"
GLUSTERD_LOGLEVEL="INFO"

修改完成后,你需要使用以下命令重载服务脚本是其生效,

systemctl daemon-reload

2.3.3 启动并配置服务自启动

systemctl start glusterd.service
systemctl enable glusterd.service

如果遇到以下提示信息,

Failed to start glusterd.service: Unit rpcbind.service not found.

请安装如下依赖包解决依赖关系,

yum install -y rpcbind

启动完毕后,我们建议你使用如下命令确认服务启动,

netstat -antp | grep glusterd

如果见到如下输出则服务正常,

tcp        0      0 0.0.0.0:24007           0.0.0.0:*               LISTEN      95740/glusterd

2.4 补充官方建议的包

2.4.1 补充rpm包

yum install autoconf automake bison dos2unix flex fuse-devel glib2-devel libacl-devel libaio-devel libattr-devel libcurl-devel libibverbs librdmacm libtirpc-devel libtool libxml2-devel lvm2-libs make openssl-devel pkgconf python36-devel readline-devel rpm-build sqlite-devel systemtap-sdt-devel tar

2.4.1 补充Python扩展包

pip3 install pylzma
pip3 install eventlet
pip3 install netifaces
pip3 install simplejson
pip3 install WebOb
pip3 install pyxattr

2.4.1 被替代或没有明确的包

cmocka或cmockery2-devel <-- 使用编译安装解决
libibverbs-devel <-- 使用libibverbs代替安装
librdmacm-devel <-- 使用librdmacm代替安装
lvm2-devel <-- 使用lvm2-libs代替安装
pkgconfig <-- 使用pkgconf代替安装
python-paste-deploy <-- 没有找到,暂时不安装
python-sphinx <-- 没有找到,暂时不安装
userspace-rcu-devel <-- 使用编译安装解决

参阅文档
====================

Gluster GitHub
——————-
https://github.com/gluster/glusterfs

Gluster的文档
—————–
https://docs.gluster.org/en/latest/

当前的软件包发布主页
——————–
https://www.gluster.org/install/

软件的下载
—————–
https://download.gluster.org/pub/gluster/glusterfs/

发行说明
———-
https://docs.gluster.org/en/latest/release-notes/

版本生命周期
————
https://www.gluster.org/release-schedule/

编译安装
—————–
https://docs.gluster.org/en/latest/Developer-guide/Building-GlusterFS/
https://github.com/gluster/glusterfs/blob/master/INSTALL

liburcu的编译错误处理
—————-
https://lists.gluster.org/pipermail/gluster-devel/2016-August/050521.html

启动脚本的错误处理
—————-
https://github.com/gluster/glusterfs/issues/417
https://serverfault.com/questions/716147/how-to-reset-glusterd-configuration
https://github.com/gluster/glusterfs/issues/637

没有评论

发表评论

GlusterFS
如何在RHEL8 部署Gluster客户端?

1 基础知识 1.1 Gluster卷的客户端类型 1.1.1 Gluster Native Cli …

GlusterFS
如何在RHEL8 部署Gluster服务端?

1 基础知识 1.1 Gluster的简介 – Gluster是一个可扩展的分布式文件系 …

GlusterFS
如何在RHEL8 YUM安装Gluster?

1 前言 由于Gluster官方没有提供很详细的编译安装文档,本章将通过在RHEL8上安装Glust …