open-uri------open()
一个带异常处理的open...
Ruby代码
def safe_open(url, retries = 5, sleeep = 0.42, headers = {})
begin
open(url, headers).read
rescue StandardError,Timeout::Error, SystemCallError, Errno::ECONNREFUSED #有些异常不是标准异常
puts $!
retries -= 1
if retries > 0
sleep sleeep and retry
else
#TODO Logging..
#TODO 多次爬取失败后记录到日志
end
end
end
要先 require 'timeout'