hadoop的超时设立

hadoop的超时设置

from?http://blog.163.com/zhengjiu_520/blog/static/3559830620130510443956/

?

最近在测试mysql往HDFS备份的性能问题，发现对mysql执行xtrabackup进行限速之后经常出现HDFS写入不正常的情况。

??? 1. 错误如下：

?11/10/12 10:50:44 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception? for block blk_8540857362443890085_4343699470java.net.SocketTimeoutException: 66000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.*.*.*:14707 remote=/*.*.*.24:80010]

??? 2. 修改超时设置

??? 后来发现HDFS写入过程中有两个超时设置：?dfs.socket.timeout和?dfs.datanode.socket.write.timeout；有些地方以为只是需要修改后面?的dfs.datanode.socket.write.timeout项就可以，其实看报错是READ_TIMEOUT。对应在hbase中的默认值如下：

??// Timeouts for communicating with DataNode for streaming writes/reads

??public static int READ_TIMEOUT =?60 * 1000;?? //其实是超过了这个值

? public static int READ_TIMEOUT_EXTENSION = 3 * 1000;

? public static int WRITE_TIMEOUT = 8 * 60 * 1000;

? public static int WRITE_TIMEOUT_EXTENSION = 5 * 1000; //for write pipeline

??? 3. 修改配置文件

??? 所以找出来是超时导致的，所以在hadoop-site.xml[hdfs-site.xml]配置文件中添加如下配置：

?? <property>
???? <name>dfs.datanode.socket.write.timeout</name>
???? <value>3000000</value>
?? </property>
?
?? <property>
???? <name>dfs.socket.timeout</name>
???? <value>3000000</value>
?? </property>
?</configuration>

?

附：备份命令

?innobackupex-1.5.1 –user=* –tmpdir=$BAKDEST –stream=tar $BAKDEST –slave-info | $COMPRESS | java hdfs write