如何部署Oracle Linux 10.x Apache Hadoop 2.6.0？

Apache-Hadoop

1 基础知识

如何二进制部署Apache Hadoop？

2 最佳实践

2.1 准备环境

2.1.1 准备操作系统

如何安装Oracle Linux 10.x？

2.1.2 准备JDK环境

如何部署最新版本的JDK运行环境？


安装完成后，请使用如下命令检查JDK的安装，

java -version

命令显示如下，

java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

2.1.3 安装rsync

dnf install -y rsync

2.1.4 下载二进制安装包

cd ~
wget https://archive.apache.org/dist/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz

其他版本请从以下链接下载，
https://archive.apache.org/dist/hadoop/common/

2.2 配置Apache Hadoop软件包

2.2.1 配置运行用户

groupadd hdfs
useradd -g hdfs -d /var/lib/hadoop-hdfs/ hdfs

2.2.2 部署Apache Hadoop软件包

cd ~
tar -xf hadoop-2.6.0.tar.gz
mv hadoop-2.6.0 /usr/

2.2.3 配置环境变量

vim /etc/profile.d/hadoop.sh

加入如下配置，

export HADOOP_HOME=/usr/hadoop-2.6.0
export HADOOP_PREFIX=${HADOOP_HOME}
export HADOOP_YARN_HOME=${HADOOP_HOME}
export PATH=${HADOOP_HOME}/bin:$PATH
export PATH=${HADOOP_HOME}/sbin:$PATH
export HADOOP_CONF_DIR=/etc/hadoop
export HADOOP_LOG_DIR=/var/log/hadoop
export HADOOP_PID_DIR=/var/run/hadoop
export HADOOP_IDENT_STRING=$USER
export HADOOP_NICENESS=0

根据声明的目录配置权限，

mkdir -p /var/log/hadoop
chown hdfs:hdfs /var/log/hadoop
mkdir -p /var/run/hadoop
chown hdfs:hdfs /var/run/hadoop

2.2.4 导入环境变量

source /etc/profile.d/hadoop.sh

2.2.5 测试安装部署

hadoop version

可见如下输出，

Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0
From source with checksum 18e43357c8f927c0695f1e9522859d6a
This command was run using /usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar

3 非分布模式下运行

cd ${HADOOP_HOME}
mkdir input
cp etc/hadoop/*.xml input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar grep input output 'dfs[a-z.]+'
cat output/*

4 伪分布模式下运行

4.1 定义配置

4.1.1 创建配置文件夹

mkdir -p /etc/hadoop

4.1.2 定义core-site.xml配置文件

cp ${HADOOP_HOME}/etc/hadoop/core-site.xml /etc/hadoop/
vim /etc/hadoop/core-site.xml

修改如下配置，

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

4.1.3 定义hdfs-site.xml配置文件

cp ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml /etc/hadoop/
vim /etc/hadoop/hdfs-site.xml

修改如下配置，

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

4.1.4 部署slaves配置文件

cp ${HADOOP_HOME}/etc/hadoop/slaves /etc/hadoop/

4.2 配置公钥认证

4.2.1 配置公钥认证

su - hdfs
mkdir ~/.ssh
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

4.2.2 测试公钥认证

su - hdfs
ssh localhost
ssh 0.0.0.0

注：以上，如果不用输入密码即可完成登录，则配置完成。

4.3 配置apache hadoop

4.3.1 格式化文件系统

rm -rf /tmp/*
su - hdfs -c 'hadoop namenode -format'

为了便于分析，建议先清空临时目录（没有定义数据默认输出到临时目录），可使用如下命令检查命令行执行的结果，

ls -ld /tmp/*

可见如下输出，

drwxr-xr-x 3 hdfs hdfs 17 Feb  9 09:12 /tmp/hadoop-hdfs
drwxr-xr-x 2 hdfs hdfs  6 Feb  9 09:12 /tmp/hsperfdata_hdfs

可使用如下命令查看目录的结构，

tree /tmp/

可见如下输出，

/tmp/
├── hadoop-hdfs
│   └── dfs
│       └── name
│           └── current
│               ├── fsimage_0000000000000000000
│               ├── fsimage_0000000000000000000.md5
│               ├── seen_txid
│               └── VERSION
└── hsperfdata_hdfs

6 directories, 4 files

4.3.2 启动守护进程

su - hdfs -c '/usr/hadoop-2.6.0/sbin/start-dfs.sh'

可见如下输出，

Starting namenodes on [localhost]
localhost: starting namenode, logging to /var/log/hadoop/hadoop-hdfs-namenode-hd01.cmdschool.org.out
localhost: 2026-02-09 11:53:06,505 INFO  [main] namenode.NameNode (StringUtils.java:startupShutdownMessage(633)) - STARTUP_MSG: 
localhost: /************************************************************
localhost: STARTUP_MSG: Starting NameNode
localhost: STARTUP_MSG:   host = hd01.cmdschool.org/10.168.0.100
localhost: STARTUP_MSG:   args = []
localhost: STARTUP_MSG:   version = 2.6.0
localhost: STARTUP_MSG:   classpath = /etc/hadoop:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar
localhost: STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
localhost: STARTUP_MSG:   java = 1.8.0_121
localhost: ************************************************************/
localhost: starting datanode, logging to /var/log/hadoop/hadoop-hdfs-datanode-hd01.cmdschool.org.out
localhost: 2026-02-09 11:53:10,929 INFO  [main] datanode.DataNode (StringUtils.java:startupShutdownMessage(633)) - STARTUP_MSG: 
localhost: /************************************************************
localhost: STARTUP_MSG: Starting DataNode
localhost: STARTUP_MSG:   host = hd01.cmdschool.org/10.168.0.100
localhost: STARTUP_MSG:   args = []
localhost: STARTUP_MSG:   version = 2.6.0
localhost: STARTUP_MSG:   classpath = /etc/hadoop:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar
localhost: STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
localhost: STARTUP_MSG:   java = 1.8.0_121
localhost: ************************************************************/
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /var/log/hadoop/hadoop-hdfs-secondarynamenode-hd01.cmdschool.org.out
0.0.0.0: 2026-02-09 11:53:16,799 INFO  [main] namenode.SecondaryNameNode (StringUtils.java:startupShutdownMessage(633)) - STARTUP_MSG: 
0.0.0.0: /************************************************************
0.0.0.0: STARTUP_MSG: Starting SecondaryNameNode
0.0.0.0: STARTUP_MSG:   host = hd01.cmdschool.org/10.168.0.100
0.0.0.0: STARTUP_MSG:   args = []
0.0.0.0: STARTUP_MSG:   version = 2.6.0
0.0.0.0: STARTUP_MSG:   classpath = /etc/hadoop:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar
0.0.0.0: STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
0.0.0.0: STARTUP_MSG:   java = 1.8.0_121
0.0.0.0: ************************************************************/

启动后，可使用如下命令再次查看并对比目录的变化，

tree /tmp/

可见如下输出，

/tmp/
├── hadoop-hdfs
│   └── dfs
│       ├── data
│       │   ├── current
│       │   │   ├── BP-1805335553-10.10.200.216-1770599561877
│       │   │   │   ├── current
│       │   │   │   │   ├── finalized
│       │   │   │   │   ├── rbw
│       │   │   │   │   └── VERSION
│       │   │   │   ├── dncp_block_verification.log.curr
│       │   │   │   └── tmp
│       │   │   └── VERSION
│       │   └── in_use.lock
│       ├── name
│       │   ├── current
│       │   │   ├── edits_inprogress_0000000000000000001
│       │   │   ├── fsimage_0000000000000000000
│       │   │   ├── fsimage_0000000000000000000.md5
│       │   │   ├── seen_txid
│       │   │   └── VERSION
│       │   └── in_use.lock
│       └── namesecondary
│           └── in_use.lock
├── hsperfdata_hdfs
│   ├── 16006
│   ├── 16173
│   └── 16355
├── Jetty_0_0_0_0_50070_hdfs____w2cu08
│   └── jsp
├── Jetty_0_0_0_0_50075_datanode____hwtdwq
│   └── jsp
└── Jetty_0_0_0_0_50090_secondary____y6aanv
    └── jsp

20 directories, 14 files

可使用如下命令查看PID目录的文件创建，

ls /var/run/hadoop/

可见如下显示，

hadoop-hdfs-datanode.pid  hadoop-hdfs-namenode.pid  hadoop-hdfs-secondarynamenode.pid

注：从上面PID文件可以分析出，当前启用“datanode”，“namenode”，“secondarynamenode”三种角色。
如果想要查看启动的进程，可执行如下命令，

pgrep -u hdfs java -a

可见如下输出，

36109 /usr/java/jdk1.8.0_121/bin/java -Dproc_namenode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop -Dhadoop.log.file=hadoop-hdfs-namenode-hd01.cmdschool.org.log -Dhadoop.home.dir=/usr/hadoop-2.6.0 -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/usr/hadoop-2.6.0/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.NameNode
36281 /usr/java/jdk1.8.0_121/bin/java -Dproc_datanode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop -Dhadoop.log.file=hadoop-hdfs-datanode-hd01.cmdschool.org.log -Dhadoop.home.dir=/usr/hadoop-2.6.0 -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/usr/hadoop-2.6.0/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode
36470 /usr/java/jdk1.8.0_121/bin/java -Dproc_secondarynamenode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop -Dhadoop.log.file=hadoop-hdfs-secondarynamenode-hd01.cmdschool.org.log -Dhadoop.home.dir=/usr/hadoop-2.6.0 -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/usr/hadoop-2.6.0/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode

如果你想确认服务是否启用，请使用如下命令，

ss -antp | grep LISTEN | grep -f <(pgrep -u hdfs java)

可见如下输出，

LISTEN    0      50           0.0.0.0:50010      0.0.0.0:*     users:(("java",pid=36281,fd=186))                                      
LISTEN    0      128          0.0.0.0:50020      0.0.0.0:*     users:(("java",pid=36281,fd=204))                                      
LISTEN    0      128          0.0.0.0:50070      0.0.0.0:*     users:(("java",pid=36109,fd=185))                                      
LISTEN    0      128          0.0.0.0:50075      0.0.0.0:*     users:(("java",pid=36281,fd=190))                                      
LISTEN    0      128          0.0.0.0:50090      0.0.0.0:*     users:(("java",pid=36470,fd=198))                                      
LISTEN    0      128        127.0.0.1:9000       0.0.0.0:*     users:(("java",pid=36109,fd=204))

测试到这里，请使用以下命令停止服务，

su - hdfs -c '/usr/hadoop-2.6.0/sbin/stop-dfs.sh'

然后，请使用以下命令测试服务的逐个启动，

su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start namenode'
su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start datanode'
su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start secondarynamenode'

以上，每执行一条命令，我们建议你使用如下命令观察目录PID文件的创建，

ls /var/run/hadoop/

然后，请使用以下命令测试服务的逐个停止，

su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh stop namenode'
su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh stop datanode'
su - hdfs -c 'source /etc/profile.d/hadoop.sh;/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh stop secondarynamenode'

4.3.3 配置名称节点服务控制脚本

vim /usr/lib/systemd/system/hdfs-nn.service

可加入如下配置，

[Unit]
Description=Apache HDFS namenode manager
Wants=network.target
Before=network.target
After=network-pre.target
Documentation=https://hadoop.apache.org/docs/

[Service]
Type=forking
User=hdfs
Group=hdfs

Environment="JAVA_HOME=/usr/java/jdk1.8.0_121"
Environment="HADOOP_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_PREFIX=/usr/hadoop-2.6.0"
Environment="HADOOP_YARN_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_CONF_DIR=/etc/hadoop"
Environment="HADOOP_LOG_DIR=/var/log/hadoop"
Environment="HADOOP_PID_DIR=/var/run/hadoop"
Environment="HADOOP_IDENT_STRING=hdfs"
Environment="HADOOP_NICENESS=0"

ExecStartPre=/bin/sh -c 'mkdir -p /var/run/hadoop;chown hdfs:hdfs /var/run/hadoop;chmod 775 /var/run/hadoop'
ExecStartPre=/bin/sh -c 'mkdir -p /var/log/hadoop;chown hdfs:hdfs /var/log/hadoop;chmod 775 /var/log/hadoop'
ExecStart=/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start namenode
ExecStop=/usr/bin/kill -SIGINT $MAINPID
PIDFile=/var/run/hadoop/hadoop-hdfs-namenode.pid

LimitNOFILE=65535
LimitCORE=infinity

Restart=on-failure
RestartSec=5
TimeoutSec=300
KillMode=process
IgnoreSIGPIPE=no
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

修改完脚本后，你需要使用如下命令重载服务，

systemctl daemon-reload

你可使用如下命令控制服务和查询状态，

systemctl start hdfs-nn.service
systemctl status hdfs-nn.service
systemctl stop hdfs-nn.service
systemctl restart hdfs-nn.service

测试完毕，建议你使用如下命令设置服务自动启动，

systemctl enable hdfs-nn.service

4.3.4 配置数据节点服务控制脚本

vim /usr/lib/systemd/system/hdfs-dn.service

可加入如下配置，

[Unit]
Description=Apache HDFS datanode manager
Wants=network.target
Before=network.target
After=network-pre.target
Documentation=https://hadoop.apache.org/docs/

[Service]
Type=forking
User=hdfs
Group=hdfs

Environment="JAVA_HOME=/usr/java/jdk1.8.0_121"
Environment="HADOOP_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_PREFIX=/usr/hadoop-2.6.0"
Environment="HADOOP_YARN_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_CONF_DIR=/etc/hadoop"
Environment="HADOOP_LOG_DIR=/var/log/hadoop"
Environment="HADOOP_PID_DIR=/var/run/hadoop"
Environment="HADOOP_IDENT_STRING=hdfs"
Environment="HADOOP_NICENESS=0"

ExecStartPre=/bin/sh -c 'mkdir -p /var/run/hadoop;chown hdfs:hdfs /var/run/hadoop;chmod 775 /var/run/hadoop'
ExecStartPre=/bin/sh -c 'mkdir -p /var/log/hadoop;chown hdfs:hdfs /var/log/hadoop;chmod 775 /var/log/hadoop'
ExecStart=/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start datanode
ExecStop=/usr/bin/kill -SIGINT $MAINPID
PIDFile=/var/run/hadoop/hadoop-hdfs-datanode.pid

LimitNOFILE=65535
LimitCORE=infinity

Restart=on-failure
RestartSec=5
TimeoutSec=300
KillMode=process
IgnoreSIGPIPE=no
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

修改完脚本后，你需要使用如下命令重载服务，

systemctl daemon-reload

你可使用如下命令控制服务和查询状态，

systemctl start hdfs-dn.service
systemctl status hdfs-dn.service
systemctl stop hdfs-dn.service
systemctl restart hdfs-dn.service

测试完毕，建议你使用如下命令设置服务自动启动，

systemctl enable hdfs-dn.service

4.3.5 配置第二名称节点服务控制脚本

vim /usr/lib/systemd/system/hdfs-snn.service

可加入如下配置，

[Unit]
Description=Apache HDFS secondarynamenode manager
Wants=network.target
Before=network.target
After=network-pre.target
Documentation=https://hadoop.apache.org/docs/

[Service]
Type=forking
User=hdfs
Group=hdfs

Environment="JAVA_HOME=/usr/java/jdk1.8.0_121"
Environment="HADOOP_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_PREFIX=/usr/hadoop-2.6.0"
Environment="HADOOP_YARN_HOME=/usr/hadoop-2.6.0"
Environment="HADOOP_CONF_DIR=/etc/hadoop"
Environment="HADOOP_LOG_DIR=/var/log/hadoop"
Environment="HADOOP_PID_DIR=/var/run/hadoop"
Environment="HADOOP_IDENT_STRING=hdfs"
Environment="HADOOP_NICENESS=0"

ExecStartPre=/bin/sh -c 'mkdir -p /var/run/hadoop;chown hdfs:hdfs /var/run/hadoop;chmod 775 /var/run/hadoop'
ExecStartPre=/bin/sh -c 'mkdir -p /var/log/hadoop;chown hdfs:hdfs /var/log/hadoop;chmod 775 /var/log/hadoop'
ExecStart=/usr/hadoop-2.6.0/sbin/hadoop-daemon.sh start secondarynamenode
ExecStop=/usr/bin/kill -SIGINT $MAINPID
PIDFile=/var/run/hadoop/hadoop-hdfs-secondarynamenode.pid

LimitNOFILE=65535
LimitCORE=infinity

Restart=on-failure
RestartSec=5
TimeoutSec=300
KillMode=process
IgnoreSIGPIPE=no
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

修改完脚本后，你需要使用如下命令重载服务，

systemctl daemon-reload

你可使用如下命令控制服务和查询状态，

systemctl start hdfs-snn.service
systemctl status hdfs-snn.service
systemctl stop hdfs-snn.service
systemctl restart hdfs-snn.service

测试完毕，建议你使用如下命令设置服务自动启动，

systemctl enable hdfs-snn.service

4.3.6 开放名称节点的Web端口

firewall-cmd --permanent --add-port 50070/tcp
firewall-cmd --reload
firewall-cmd --list-all

5.3.7 访问名称节点的Web界面

http://10.168.0.100:50070/
可见如下页面，

5.4 测试apache hadoop

如何测试Apache Hadoop HDFS？

参阅文档
==================
https://www.cmdschool.org/archives/5507
https://www.cmdschool.org/archives/397

没有评论

发表回复

Apache-Hadoop
如何测试Apache Hadoop HDFS？

1 前言 一个问题，一篇文章，一出故事。 今天完成 Apache Hadoop 2.6.0 HDFS …

Apache-Hadoop
如何安装Kafka connect mqtt？

1 前言 一个问题，一篇文章，一出故事。 我们配置好Kafka connect集群后，我们来尝试安装 …

Apache-Hadoop
如何配置Kafka connect集群？

1 基础知识 1.1 Kafka Connect的介绍 – Kafak Connect是 …