solrCloud 管理colleciton操作，可能出现的重大Bug
@Override public void run() { log.info("Process current queue of collection creations"); while (amILeader() && !isClosed) { try { byte[] head = workQueue.peek(true); //if (head != null) { // should not happen since we block above final ZkNodeProps message = ZkNodeProps.load(head); final String operation = message.getStr(QUEUE_OPERATION); boolean success = processMessage(message, operation); if (!success) { // TODO: what to do on failure / partial failure // if we fail, do we clean up then ? SolrException.log(log, "Collection creation of " + message.getStr("name") + " failed"); } //} workQueue.remove(); } catch (KeeperException e) { if (e.code() == KeeperException.Code.SESSIONEXPIRED || e.code() == KeeperException.Code.CONNECTIONLOSS) { log.warn("Overseer cannot talk to ZK"); return; } SolrException.log(log, "", e); throw new ZooKeeperException(SolrException.ErrorCode.SERVER_ERROR, "", e); } catch (InterruptedException e) { Thread.currentThread().interrupt(); return; } } }
代码上有一个地方是执行分布式队列里的消息，当执行成功后就会remove掉该信息，但由于没有catch 到运行时异常，就没有执行 workQueue.remove().这样这条信息一直没被处理掉，即使将服务关闭，重启，又抛运行时异常，thread死掉，进入这种死循环里。。
