如何测试Apache Hadoop HDFS?
- By : Will
- Category : Apache-Hadoop
Apache-Hadoop
1 前言
一个问题,一篇文章,一出故事。
今天完成 Apache Hadoop 2.6.0 HDFS的部署,因此测试该HDFS的需求诞生。
2 最佳实践
2.1 测试环境
2.2 测试步骤
2.2.1 在hdfs中创建文件夹
su - hdfs hdfs dfs -ls / hdfs dfs -mkdir /user hdfs dfs -ls /
注:
– 第一条命令列出hdfs的根目录(空的,什么都没有)
– 第二条命令在hdfs的根目录下创建user文件夹
2.2.2 上传文件到hdfs
su - hdfs hdfs dfs -mkdir /input hdfs dfs -put /etc/hadoop/ /input hdfs dfs -ls /input
2.2.3 获取hdfs的文件
su - hdfs hdfs dfs -get /input/hadoop ls -l
2.2.4 使用cat读取hdfs的文件
su - hdfs hdfs dfs -cat /input/hadoop/*
2.2.5 查看使用概况
hadoop dfsadmin -report
可见如下显示,
DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. Configured Capacity: 47676653568 (44.40 GB) Present Capacity: 43531128832 (40.54 GB) DFS Remaining: 43531091968 (40.54 GB) DFS Used: 36864 (36 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Live datanodes (1): Name: 127.0.0.1:50010 (localhost) Hostname: tpcmeshd01.sae.com.hk Decommission Status : Normal Configured Capacity: 47676653568 (44.40 GB) DFS Used: 36864 (36 KB) Non DFS Used: 4145524736 (3.86 GB) DFS Remaining: 43531091968 (40.54 GB) DFS Used%: 0.00% DFS Remaining%: 91.30% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1 Last contact: Mon Feb 09 15:05:04 CST 2026
没有评论