如何编译部署传统启动的haproxy?

Load balancing

1 基础知识

1.1 软件的介绍

– HAProxy是一款开源免费、快速、可靠的解决方案
– HAProxy是基于TCP和HTTP协议提供高可用、负载均衡和代理服务

1.2 软件的功能

– 负载均衡
– 高可用
– 代理服务

2 最佳实践

2.1 环境信息

OS = CentOS 7.3 x86_64
IP Address = 10.168.0.70
host name = haproxy.cmdschool.org

2.2 安装前的准备

2.2.1 下载软件包

cd ~
wget https://www.haproxy.org/download/2.1/src/haproxy-2.1.4.tar.gz

2.2.2 解压软件包

cd ~
tar -xf haproxy-2.1.4.tar.gz

2.2.3 准备编译环境

yum -y install gcc make

2.2.4 关闭SELinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

2.2.5 开放所需的端口

firewall-cmd --permanent --add-port 8080/tcp 
firewall-cmd --reload
firewall-cmd --list-all

2.3 编译安装

2.3.1 定义编译安装选项

cd ~/haproxy-2.1.4
vim Makefile

根据实际情况定义如下编译选项,

#### Installation options.
PREFIX = /usr
SBINDIR = $(PREFIX)/sbin
MANDIR = $(PREFIX)/share/man
DOCDIR = $(PREFIX)/doc/haproxy
TARGET = linux3100
CPU = generic
ARCH = x86_64

另外,“TARGET”与“ARCH”的定义可参考如下命令的输出,

uname -a

可见如下提示,

Linux localhost.localdomain 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

2.3.2 编译软件包

cd ~/haproxy-2.1.4
make

2.3.3 安装软件包

cd ~/haproxy-2.1.4
make install

可见如下输出,

‘haproxy’ -> ‘/usr/sbin/haproxy’
‘doc/haproxy.1’ -> ‘/usr/share/man/man1/haproxy.1’
install: creating directory ‘/usr/doc’
install: creating directory ‘/usr/doc/haproxy’
‘doc/configuration.txt’ -> ‘/usr/doc/haproxy/configuration.txt’
‘doc/management.txt’ -> ‘/usr/doc/haproxy/management.txt’
‘doc/seamless_reload.txt’ -> ‘/usr/doc/haproxy/seamless_reload.txt’
‘doc/architecture.txt’ -> ‘/usr/doc/haproxy/architecture.txt’
‘doc/peers-v2.0.txt’ -> ‘/usr/doc/haproxy/peers-v2.0.txt’
‘doc/regression-testing.txt’ -> ‘/usr/doc/haproxy/regression-testing.txt’
‘doc/cookie-options.txt’ -> ‘/usr/doc/haproxy/cookie-options.txt’
‘doc/lua.txt’ -> ‘/usr/doc/haproxy/lua.txt’
‘doc/WURFL-device-detection.txt’ -> ‘/usr/doc/haproxy/WURFL-device-detection.txt’
‘doc/proxy-protocol.txt’ -> ‘/usr/doc/haproxy/proxy-protocol.txt’
‘doc/linux-syn-cookies.txt’ -> ‘/usr/doc/haproxy/linux-syn-cookies.txt’
‘doc/SOCKS4.protocol.txt’ -> ‘/usr/doc/haproxy/SOCKS4.protocol.txt’
‘doc/network-namespaces.txt’ -> ‘/usr/doc/haproxy/network-namespaces.txt’
‘doc/DeviceAtlas-device-detection.txt’ -> ‘/usr/doc/haproxy/DeviceAtlas-device-detection.txt’
‘doc/51Degrees-device-detection.txt’ -> ‘/usr/doc/haproxy/51Degrees-device-detection.txt’
‘doc/netscaler-client-ip-insertion-protocol.txt’ -> ‘/usr/doc/haproxy/netscaler-client-ip-insertion-protocol.txt’
‘doc/peers.txt’ -> ‘/usr/doc/haproxy/peers.txt’
‘doc/close-options.txt’ -> ‘/usr/doc/haproxy/close-options.txt’
‘doc/SPOE.txt’ -> ‘/usr/doc/haproxy/SPOE.txt’
‘doc/intro.txt’ -> ‘/usr/doc/haproxy/intro.txt’

2.3.4 确认安装

haproxy -v

可见如下显示,

HA-Proxy version 2.1.4 2020/04/02 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.4.html

2.3.5 获取使用帮助

haproxy -h

可见如下显示,

HA-Proxy version 2.1.4 2020/04/02 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.4.html
Usage : haproxy [-f ]* [ -vdVD ] [ -n  ] [ -N  ]
        [ -p  ] [ -m  ] [ -C  ] [-- *]
        -v displays version ; -vv shows known build options.
        -d enters debug mode ; -db only disables background mode.
        -dM[] poisons memory with  (defaults to 0x50)
        -V enters verbose mode (disables quiet mode)
        -D goes daemon ; -C changes to  before loading files.
        -W master-worker mode.
        -q quiet mode : don't display messages
        -c check mode : only check config files and exit
        -n sets the maximum total # of connections (uses ulimit -n)
        -m limits the usable amount of memory (in MB)
        -N sets the default, per-proxy maximum # of connections (0)
        -L set local peer name (default to hostname)
        -p writes pids of all children to this file
        -dp disables poll() usage even when available
        -dR disables SO_REUSEPORT usage
        -dr ignores server address resolution failures
        -dV disables SSL verify on servers side
        -sf/-st [pid ]* finishes/terminates old pids.
        -x  get listening sockets from a unix socket
        -S [,...] new master CLI

2.4 部署编译程序

2.4.1 部署服务控制脚本

cp ~/haproxy-2.1.4/examples/haproxy.init /etc/init.d/haproxy
chmod +x /etc/init.d/haproxy
vim /etc/init.d/haproxy

修改如下代码,

# [ ${NETWORKING} = "no" ] && exit 0
[ "${NETWORKING}" = "no" ] && exit 0

由于官方提供的脚本定义不够严格,没有修改的情况下执行将显示如下错误,

-bash: [: =: unary operator expected

配置修改后,整体配置显示如下,

#!/bin/sh
#
# chkconfig: - 85 15
# description: HA-Proxy is a TCP/HTTP reverse proxy which is particularly suited \
#              for high availability environments.
# processname: haproxy
# config: /etc/haproxy/haproxy.cfg
# pidfile: /var/run/haproxy.pid

# Script Author: Simon Matter 
# Version: 2004060600

# Source function library.
if [ -f /etc/init.d/functions ]; then
  . /etc/init.d/functions
elif [ -f /etc/rc.d/init.d/functions ] ; then
  . /etc/rc.d/init.d/functions
else
  exit 0
fi

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
# [ ${NETWORKING} = "no" ] && exit 0
[ "${NETWORKING}" = "no" ] && exit 0

# This is our service name
BASENAME=`basename $0`
if [ -L $0 ]; then
  BASENAME=`find $0 -name $BASENAME -printf %l`
  BASENAME=`basename $BASENAME`
fi

BIN=/usr/sbin/$BASENAME

CFG=/etc/$BASENAME/$BASENAME.cfg
[ -f $CFG ] || exit 1

PIDFILE=/var/run/$BASENAME.pid
LOCKFILE=/var/lock/subsys/$BASENAME

RETVAL=0

start() {
  quiet_check
  if [ $? -ne 0 ]; then
    echo "Errors found in configuration file, check it with '$BASENAME check'."
    return 1
  fi

  echo -n "Starting $BASENAME: "
  daemon $BIN -D -f $CFG -p $PIDFILE
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch $LOCKFILE
  return $RETVAL
}

stop() {
  echo -n "Shutting down $BASENAME: "
  killproc $BASENAME -USR1
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && rm -f $LOCKFILE
  [ $RETVAL -eq 0 ] && rm -f $PIDFILE
  return $RETVAL
}

restart() {
  quiet_check
  if [ $? -ne 0 ]; then
    echo "Errors found in configuration file, check it with '$BASENAME check'."
    return 1
  fi
  stop
  start
}

reload() {
  if ! [ -s $PIDFILE ]; then
    return 0
  fi

  quiet_check
  if [ $? -ne 0 ]; then
    echo "Errors found in configuration file, check it with '$BASENAME check'."
    return 1
  fi
  $BIN -D -f $CFG -p $PIDFILE -sf $(cat $PIDFILE)
}

check() {
  $BIN -c -q -V -f $CFG
}

quiet_check() {
  $BIN -c -q -f $CFG
}

rhstatus() {
  status $BASENAME
}

condrestart() {
  [ -e $LOCKFILE ] && restart || :
}

# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  reload)
    reload
    ;;
  condrestart)
    condrestart
    ;;
  status)
    rhstatus
    ;;
  check)
    check
    ;;
  *)
    echo $"Usage: $BASENAME {start|stop|restart|reload|condrestart|status|check}"
    exit 1
esac

exit $?

获取脚本使用帮助,

/etc/init.d/haproxy

可见如下提示,

Usage: haproxy {start|stop|restart|reload|condrestart|status|check}

根据脚本变量“CFG”的定义,我们需要使用以下命令准备配置文件

mkdir -p /etc/haproxy
cp ~/haproxy-2.1.4/examples/option-http_proxy.cfg /etc/haproxy/haproxy.cfg
cat /etc/haproxy/haproxy.cfg

可见如下配置,

#
# demo config for Proxy mode
#

global
        maxconn         20000
        ulimit-n        16384
        log             127.0.0.1 local0
        uid             200
        gid             200
        chroot          /var/empty
        nbproc          4
        daemon

frontend test-proxy
        bind            192.168.200.10:8080
        mode            http
        log             global
        option          httplog
        option          dontlognull
        option          nolinger
        option          http_proxy
        maxconn         8000
        timeout client  30s

        # layer3: Valid users
        acl allow_host src 192.168.200.150/32
        http-request deny if !allow_host

        # layer7: prevent private network relaying
        acl forbidden_dst url_ip 192.168.0.0/24
        acl forbidden_dst url_ip 172.16.0.0/12
        acl forbidden_dst url_ip 10.0.0.0/8
        http-request deny if forbidden_dst

        default_backend test-proxy-srv


backend test-proxy-srv
        mode            http
        timeout connect 5s
        timeout server  5s
        retries         2
        option          nolinger
        option          http_proxy

        # layer7: Only GET method is valid
        acl valid_method        method GET
        http-request deny if !valid_method

        # layer7: protect bad reply
        http-response deny if { res.hdr(content-type) audio/mp3 }

根据配置的GID与UID参数,我们需要创建以下运行用户

groupadd  -g 200 haproxy
useradd -u 200 -g 200 -d /var/spool/haproxy -s /sbin/nologin haproxy

2.4.2 执行配置检查

/etc/init.d/haproxy check

可见如下提示,

Configuration file is valid

2.4.3 尝试启动服务

/etc/init.d/haproxy start

你可能会遇到以下提示信息,

Starting haproxy (via systemctl):  Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.
                                                           [FAILED]

我们可通过如下命令尝试查阅日志解决,

tail -f /var/log/messages

可见如下提示,

Apr 15 21:36:09 localhost haproxy: Starting haproxy: [ALERT] 105/213609 (2369) : Starting frontend test-proxy: cannot bind socket [192.168.200.10:8080]
Apr 15 21:36:09 localhost haproxy: [FAILED]

根据提示我们使用如下命令修改配置文件,

vim /etc/haproxy/haproxy.cfg

修改如下配置参数,

[...]
frontend test-proxy
        bind            0.0.0.0:8080
[...]

再次尝试启动服务,

/etc/init.d/haproxy start

可见如下提示信息,

Starting haproxy (via systemctl):                          [  OK  ]

抱着严谨的态度再次查阅日志,

tail -f /var/log/messages

可见如下提示,

Apr 15 21:43:03 localhost haproxy: Starting haproxy: [ALERT] 105/214303 (2401) : [/usr/sbin/haproxy.main()] FD limit (16384) too low for maxconn=20000/maxsock=40014. Please raise 'ulimit-n' to 40014 or more to avoid any trouble.This will fail in >= v2.3
Apr 15 21:43:03 localhost haproxy: [  OK  ]

根据提示我们使用如下命令修改配置文件,

vim /etc/haproxy/haproxy.cfg

修改如下配置参数,

global
[...]
        ulimit-n        40014
[...]

然后重启服务,

/etc/init.d/haproxy restart

最后,我们使用以下命令查看配置,

cat /etc/haproxy/haproxy.cfg

配置显示如下,

#
# demo config for Proxy mode
#

global
        maxconn         20000
        ulimit-n        40014
        log             127.0.0.1 local0
        uid             200
        gid             200
        chroot          /var/empty
        nbproc          1
        daemon

frontend test-proxy
        bind            0.0.0.0:8080
        mode            http
        log             global
        option          httplog
        option          dontlognull
        option          nolinger
        option          http_proxy
        maxconn         8000
        timeout client  30s

        # layer3: Valid users
        acl allow_host src 192.168.200.0/32
        http-request deny if !allow_host

        # layer7: prevent private network relaying
        acl forbidden_dst url_ip 192.168.0.0/24
        acl forbidden_dst url_ip 172.16.0.0/12
        acl forbidden_dst url_ip 10.0.0.0/8
        http-request deny if forbidden_dst

        default_backend test-proxy-srv


backend test-proxy-srv
        mode            http
        timeout connect 5s
        timeout server  5s
        retries         2
        option          nolinger
        option          http_proxy

        # layer7: Only GET method is valid
        acl valid_method        method GET
        http-request deny if !valid_method

        # layer7: protect bad reply
        http-response deny if { res.hdr(content-type) audio/mp3 }

2.4.4 配置服务开机启动

chkconfig haproxy on

另外,服务控制命令如下,

/etc/init.d/haproxy start
/etc/init.d/haproxy stop
/etc/init.d/haproxy reload
/etc/init.d/haproxy restart
/etc/init.d/haproxy status

参阅文档
=======================

官方文档
—————-
https://www.haproxy.org/download/2.1/doc/management.txt
https://www.haproxy.org/#docs

软件下载
——————
https://www.haproxy.org/download/
https://www.haproxy.org/#down

官方首页
—————–
https://www.haproxy.org/

官方github
—————-
http://git.haproxy.org/

安装参阅文档
—————
https://linuxscriptshub.com/install-haproxy-centos-7/

如何编译出systemd控制脚本
———————–
https://discourse.haproxy.org/t/failing-to-start-with-systemd-and-unexplained-issues/849/6

limit的配置方法
——————–
https://docs.oracle.com/cd/E19476-01/821-0505/file-descriptor-requirements.html

没有评论

发表评论

Elastic Stack
如何部署Elasticsearch集群?

1 基础知识 1.1 集群的介绍 – Elasticsearch集群允许节点单点故障 & …

Load balancing
如何编译部署systemd启动的haproxy?

1 基础知识 1.1 软件的介绍 – HAProxy是一款开源免费、快速、可靠的解决方案 …

Load balancing
如何在RHEL8部署MariaDB Galera集群?

1 基础知识 1.1 技术简介 – MariaDB Galera集群是一个同步多主集群 …