如何编译部署systemd启动的haproxy？

Linux基础

1 基础知识

1.1 软件的介绍

– HAProxy是一款开源免费、快速、可靠的解决方案
– HAProxy是基于TCP和HTTP协议提供高可用、负载均衡和代理服务

1.2 软件的功能

– 负载均衡
– 高可用
– 代理服务

2 最佳实践

2.1 环境信息

OS = CentOS 7.3 x86_64
IP Address = 10.168.0.70
host name = haproxy.cmdschool.org

2.2 安装前的准备

2.2.1 下载软件包

cd ~
https://www.haproxy.org/download/2.1/src/haproxy-2.1.4.tar.gz

2.2.2 解压软件包

cd ~
tar -xf haproxy-2.1.4.tar.gz

2.2.3 准备编译环境

yum -y install gcc make

2.2.4 关闭SELinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

2.2.5 开放所需的端口

firewall-cmd --permanent --add-port 8080/tcp 
firewall-cmd --reload
firewall-cmd --list-all

2.3 编译安装

2.3.1 定义编译安装选项

cd ~/haproxy-2.1.4
vim Makefile

根据实际情况定义如下编译选项，

#### Installation options.
PREFIX = /usr
SBINDIR = $(PREFIX)/sbin
MANDIR = $(PREFIX)/share/man
DOCDIR = $(PREFIX)/doc/haproxy
USE_SYSTEMD = 1
TARGET = linux3100
CPU = generic
ARCH = x86_64

其中“USE_SYSTEMD = 1”参数用于启用systemd的启动方式，另外“TARGET”与“ARCH”的定义可参考如下命令的输出，

uname -a

可见如下提示，

Linux localhost.localdomain 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

2.3.2 编译软件包

cd ~/haproxy-2.1.4
make

如果遇到以下提示，

src/haproxy.c:77:31: fatal error: systemd/sd-daemon.h: No such file or directory
 #include 
                               ^
compilation terminated.
make: *** [src/haproxy.o] Error 1

请使用如下指令解决依赖关系后再从新编译，

yum install -y systemd-devel

2.3.3 安装软件包

cd ~/haproxy-2.1.4
make install

可见如下输出，

‘haproxy’ -> ‘/usr/sbin/haproxy’
‘doc/haproxy.1’ -> ‘/usr/share/man/man1/haproxy.1’
install: creating directory ‘/usr/doc’
install: creating directory ‘/usr/doc/haproxy’
‘doc/configuration.txt’ -> ‘/usr/doc/haproxy/configuration.txt’
‘doc/management.txt’ -> ‘/usr/doc/haproxy/management.txt’
‘doc/seamless_reload.txt’ -> ‘/usr/doc/haproxy/seamless_reload.txt’
‘doc/architecture.txt’ -> ‘/usr/doc/haproxy/architecture.txt’
‘doc/peers-v2.0.txt’ -> ‘/usr/doc/haproxy/peers-v2.0.txt’
‘doc/regression-testing.txt’ -> ‘/usr/doc/haproxy/regression-testing.txt’
‘doc/cookie-options.txt’ -> ‘/usr/doc/haproxy/cookie-options.txt’
‘doc/lua.txt’ -> ‘/usr/doc/haproxy/lua.txt’
‘doc/WURFL-device-detection.txt’ -> ‘/usr/doc/haproxy/WURFL-device-detection.txt’
‘doc/proxy-protocol.txt’ -> ‘/usr/doc/haproxy/proxy-protocol.txt’
‘doc/linux-syn-cookies.txt’ -> ‘/usr/doc/haproxy/linux-syn-cookies.txt’
‘doc/SOCKS4.protocol.txt’ -> ‘/usr/doc/haproxy/SOCKS4.protocol.txt’
‘doc/network-namespaces.txt’ -> ‘/usr/doc/haproxy/network-namespaces.txt’
‘doc/DeviceAtlas-device-detection.txt’ -> ‘/usr/doc/haproxy/DeviceAtlas-device-detection.txt’
‘doc/51Degrees-device-detection.txt’ -> ‘/usr/doc/haproxy/51Degrees-device-detection.txt’
‘doc/netscaler-client-ip-insertion-protocol.txt’ -> ‘/usr/doc/haproxy/netscaler-client-ip-insertion-protocol.txt’
‘doc/peers.txt’ -> ‘/usr/doc/haproxy/peers.txt’
‘doc/close-options.txt’ -> ‘/usr/doc/haproxy/close-options.txt’
‘doc/SPOE.txt’ -> ‘/usr/doc/haproxy/SPOE.txt’
‘doc/intro.txt’ -> ‘/usr/doc/haproxy/intro.txt’

2.3.4 确认安装

haproxy -v

可见如下显示，

HA-Proxy version 2.1.4 2020/04/02 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.4.html

2.3.5 获取使用帮助

haproxy -h

可见如下显示，

HA-Proxy version 2.1.4 2020/04/02 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.4.html
Usage : haproxy [-f ]* [ -vdVD ] [ -n  ] [ -N  ]
        [ -p  ] [ -m  ] [ -C  ] [-- *]
        -v displays version ; -vv shows known build options.
        -d enters debug mode ; -db only disables background mode.
        -dM[] poisons memory with  (defaults to 0x50)
        -V enters verbose mode (disables quiet mode)
        -D goes daemon ; -C changes to  before loading files.
        -W master-worker mode.
        -q quiet mode : don't display messages
        -c check mode : only check config files and exit
        -n sets the maximum total # of connections (uses ulimit -n)
        -m limits the usable amount of memory (in MB)
        -N sets the default, per-proxy maximum # of connections (0)
        -L set local peer name (default to hostname)
        -p writes pids of all children to this file
        -dp disables poll() usage even when available
        -dR disables SO_REUSEPORT usage
        -dr ignores server address resolution failures
        -dV disables SSL verify on servers side
        -sf/-st [pid ]* finishes/terminates old pids.
        -x  get listening sockets from a unix socket
        -S [,...] new master CLI

2.4 部署编译程序

2.4.1 部署服务控制脚本

cp ~/haproxy-2.1.4/contrib/systemd/haproxy.service.in /usr/lib/systemd/system/haproxy.service
vim /usr/lib/systemd/system/haproxy.service

配置修改如下，

[Unit]
Description=HAProxy Load Balancer
After=network.target

[Service]
EnvironmentFile=-/etc/default/haproxy
#EnvironmentFile=-/etc/sysconfig/haproxy
Environment="CONFIG=/etc/haproxy/haproxy.cfg" "PIDFILE=/run/haproxy.pid"
ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS
ExecStart=/usr/sbin/haproxy -W -f $CONFIG -p $PIDFILE $EXTRAOPTS
ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS
#ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
Restart=always
SuccessExitStatus=143
Type=forking

# The following lines leverage SystemD's sandboxing options to provide
# defense in depth protection at the expense of restricting some flexibility
# in your setup (e.g. placement of your configuration files) or possibly
# reduced performance. See systemd.service(5) and systemd.exec(5) for further
# information.

# NoNewPrivileges=true
# ProtectHome=true
# If you want to use 'ProtectSystem=strict' you should whitelist the PIDFILE,
# any state files and any other files written using 'ReadWritePaths' or
# 'RuntimeDirectory'.
# ProtectSystem=true
# ProtectKernelTunables=true
# ProtectKernelModules=true
# ProtectControlGroups=true
# If your SystemD version supports them, you can add: @reboot, @swap, @sync
# SystemCallFilter=~@cpu-emulation @keyring @module @obsolete @raw-io

[Install]
WantedBy=multi-user.target

重点是修改变量“ExecStart”、“@SBINDIR@”和“Type”，修改完成后我们需要运行以下命令使配置生效，

systemctl daemon-reload

根据“EnvironmentFile”定义的参数，我们需要增加额外选项配置文件 

vim /etc/default/haproxy

加入如下内容，

# Add extra options to the haproxy daemon here. This can be useful for
# specifying multiple configuration files with multiple -f options.
# See haproxy(1) for a complete list of options.
EXTRAOPTS="-S /run/haproxy-master.sock"

根据变量“CONFIG”的定义，我们需要部署启动脚本所需的配置文件

mkdir -p /etc/haproxy
cp ~/haproxy-2.1.4/examples/option-http_proxy.cfg /etc/haproxy/haproxy.cfg
cat /etc/haproxy/haproxy.cfg

配置显示如下，

#
# demo config for Proxy mode
#

global
        maxconn         20000
        ulimit-n        16384
        log             127.0.0.1 local0
        uid             200
        gid             200
        chroot          /var/empty
        nbproc          4
        daemon

frontend test-proxy
        bind            192.168.200.10:8080
        mode            http
        log             global
        option          httplog
        option          dontlognull
        option          nolinger
        option          http_proxy
        maxconn         8000
        timeout client  30s

        # layer3: Valid users
        acl allow_host src 192.168.200.150/32
        http-request deny if !allow_host

        # layer7: prevent private network relaying
        acl forbidden_dst url_ip 192.168.0.0/24
        acl forbidden_dst url_ip 172.16.0.0/12
        acl forbidden_dst url_ip 10.0.0.0/8
        http-request deny if forbidden_dst

        default_backend test-proxy-srv


backend test-proxy-srv
        mode            http
        timeout connect 5s
        timeout server  5s
        retries         2
        option          nolinger
        option          http_proxy

        # layer7: Only GET method is valid
        acl valid_method        method GET
        http-request deny if !valid_method

        # layer7: protect bad reply
        http-response deny if { res.hdr(content-type) audio/mp3 }

根据配置文件“GID”和“UID”的定义，我们需要使用如下命令创建运行用户 

groupadd  -g 200 haproxy
useradd -u 200 -g 200 -d /var/spool/haproxy -s /sbin/nologin haproxy

2.4.2 尝试启动服务

systemctl start haproxy.service

如果遇到如下提示，

Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.

我们建议通过以下命令查看日志查找问题的根源，

tail -n 50 /var/log/messages

可见如下提示，

Apr 15 22:33:14 localhost haproxy: [ALERT] 105/223314 (14715) : Starting frontend test-proxy: cannot bind socket [192.168.200.10:8080]
Apr 15 22:33:14 localhost systemd: haproxy.service: control process exited, code=exited status=1
Apr 15 22:33:14 localhost systemd: Failed to start HAProxy Load Balancer.
Apr 15 22:33:14 localhost systemd: Unit haproxy.service entered failed state.
Apr 15 22:33:14 localhost systemd: haproxy.service failed.

根据提示使用如下命令修改配置，

vim /etc/haproxy/haproxy.cfg

我们修改如下配置，

[...]
frontend test-proxy
        bind            0.0.0.0:8080
        [...]

然后我们再次使用以下命令尝试启动，

systemctl start haproxy.service

服务启动成功，我们抱着严谨的精神再次查看日志文件，可见如下提示，

Apr 15 22:39:12 localhost haproxy: [ALERT] 105/223912 (14754) : [/usr/sbin/haproxy.main()] FD limit (16384) too low for maxconn=20000/maxsock=40028. Please raise 'ulimit-n' to 40028 or more to avoid any trouble.This will fail in >= v2.3

我们再次根据提示修改配置文件，

[...]
global
        [...]
        ulimit-n        40039
        [...]

修改完成后，我们需要重启服务使配置生效，

systemctl restart haproxy.service

最后我们使用如下命令查看配置，

cat /etc/haproxy/haproxy.cfg

整体配置如下，

#
# demo config for Proxy mode
#

global
        maxconn         20000
        ulimit-n        40039
        log             127.0.0.1 local0
        uid             200
        gid             200
        chroot          /var/empty
        nbproc          4
        daemon

frontend test-proxy
        bind            0.0.0.0:8080
        mode            http
        log             global
        option          httplog
        option          dontlognull
        option          nolinger
        option          http_proxy
        maxconn         8000
        timeout client  30s

        # layer3: Valid users
        acl allow_host src 192.168.200.150/32
        http-request deny if !allow_host

        # layer7: prevent private network relaying
        acl forbidden_dst url_ip 192.168.0.0/24
        acl forbidden_dst url_ip 172.16.0.0/12
        acl forbidden_dst url_ip 10.0.0.0/8
        http-request deny if forbidden_dst

        default_backend test-proxy-srv


backend test-proxy-srv
        mode            http
        timeout connect 5s
        timeout server  5s
        retries         2
        option          nolinger
        option          http_proxy

        # layer7: Only GET method is valid
        acl valid_method        method GET
        http-request deny if !valid_method

        # layer7: protect bad reply
        http-response deny if { res.hdr(content-type) audio/mp3 }

2.4.3 设置服务开机自动启动

systemctl enable haproxy.service

另外，常用服务控制命令如下，

systemctl start haproxy.service
systemctl stop haproxy.service
systemctl reload haproxy.service
systemctl restart haproxy.service
systemctl status haproxy.service

参阅文档
=======================

官方文档
—————-
https://www.haproxy.org/download/2.1/doc/management.txt
https://www.haproxy.org/#docs

软件下载
——————
https://www.haproxy.org/download/
https://www.haproxy.org/#down

官方首页
—————–
https://www.haproxy.org/

官方github
—————-
http://git.haproxy.org/

安装参阅文档
—————
https://linuxscriptshub.com/install-haproxy-centos-7/

如何编译出systemd控制脚本
———————–
https://discourse.haproxy.org/t/failing-to-start-with-systemd-and-unexplained-issues/849/6

limit的配置方法
——————–
https://docs.oracle.com/cd/E19476-01/821-0505/file-descriptor-requirements.html

没有评论

发表回复

Linux基础
如何切换Oracle Linux到红帽兼容内核？

1 基础知识 – RHCK是Red Hat Compatible Kernel的英文缩写 …

Linux基础
如何解决tabby-terminal秘钥交换错误提示II？

1 前言 一个问题，一篇文章，一出故事。 笔者使用Tabby Terminal连接旧的Linux服务 …

Linux基础
如何解决tabby-terminal加密方法错误提示？

1 前言 一个问题，一篇文章，一出故事。 笔者使用Tabby Terminal连接旧的Linux服务 …