首页 诗词 字典 板报 句子 名言 友答 励志 学校 网站地图
当前位置: 首页 > 教程频道 > 开发语言 > 编程 >

Hadoop安装教程——单机方式

2013-11-08 
Hadoop安装教程——单机模式Hadoop是MapReduce的开源实现,现在介绍它的单机模式安装方法。实验平台:Ubuntu 9.

Hadoop安装教程——单机模式
Hadoop是MapReduce的开源实现,现在介绍它的单机模式安装方法。

实验平台:Ubuntu 9.04,Hadoop0.20,JDK1.6

step 1. ssh的安装设置

由于Hadoop用ssh 通信,因此先进行免密码登录设定,

<configuration>  <property>    <name>fs.default.name</name>    <value>hdfs://localhost:9000</value>  </property>  <property>    <name>hadoop.tmp.dir</name>    <value>/tmp/hadoop/hadoop-${user.name}</value>  </property></configuration>?編輯 /opt/hadoop/conf/hdfs-site.xml <configuration>  <property>    <name>dfs.replication</name>    <value>1</value>  </property></configuration>?編輯 /opt/hadoop/conf/mapred-site.xml <configuration>  <property>    <name>mapred.job.tracker</name>    <value>localhost:9001</value>  </property></configuration>


step 6. 格式化HDFS

?以上我们已经设定好 Hadoop 单机测试的环境,接着让我们来启动 Hadoop 相关服务,格式化 namenode, secondarynamenode, tasktracker
?$ cd /opt/hadoop
?$ source /opt/hadoop/conf/hadoop-env.sh
?$ hadoop namenode -format

执行画面如:
09/03/23 20:19:47 INFO dfs.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:  host = /localhost
STARTUP_MSG:  args = [-format]
STARTUP_MSG:  version = 0.20.3
STARTUP_MSG:  build = https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.20 -r 736250; compiled by 'ndaley' on Thu Jan 22 23:12:08 UTC 2009
************************************************************/
09/03/23 20:19:47 INFO fs.FSNamesystem: fsOwner=hadooper,hadooper
09/03/23 20:19:47 INFO fs.FSNamesystem: supergroup=supergroup
09/03/23 20:19:47 INFO fs.FSNamesystem: isPermissionEnabled=true
09/03/23 20:19:47 INFO dfs.Storage: Image file of size 82 saved in 0 seconds.
09/03/23 20:19:47 INFO dfs.Storage: Storage directory /tmp/hadoop-hadooper/dfs/name has been successfully formatted.
09/03/23 20:19:47 INFO dfs.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at /localhost
************************************************************/

step 7. 启动Hadoop

?接着用 start-all.sh 来启动所有服务,包含 namenode, datanode,
/opt/hadoop$ bin/start-all.sh
执行画面如:
starting namenode, logging to /opt/hadoop/logs/hadoop-hadooper-namenode-vPro.out
localhost: starting datanode, logging to /opt/hadoop/logs/hadoop-hadooper-datanode-vPro.out
localhost: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop-hadooper-secondarynamenode-vPro.out
starting jobtracker, logging to /opt/hadoop/logs/hadoop-hadooper-jobtracker-vPro.out

step 8. 安装完毕测试

?启动之后,可以检查以下网址,来观看服务是否正常。Hadoop 管理接口 Hadoop Task Tracker 状态 Hadoop DFS 状态
?http://localhost:50030/ - Hadoop 管理接口

热点排行