问题描述

Hbase开启后hmaster闪退！Hregionserver可以开启无法关闭！

2015-04-27 10:22:22,023 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, conn
ectString=slave1:2181,master:2181,slave2:2181 sessionTimeout=180000 watcher=master:600000x0
2015-04-27 10:22:22,848 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to serv
er slave2/10.200.0.232:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-27 10:22:22,889 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
slave2/10.200.0.232:2181, initiating session
2015-04-27 10:22:22,939 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on
server slave2/10.200.0.232:2181, sessionid = 0x34cf8ad10320000, negotiated timeout = 40000
2015-04-27 10:22:23,198 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting
2015-04-27 10:22:23,223 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: s
tarting
2015-04-27 10:22:23,645 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000:
starting
2015-04-27 10:22:23,646 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000:
starting
2015-04-27 10:22:23,648 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000:
starting
2015-04-27 10:22:23,649 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000:
starting
2015-04-27 10:22:23,807 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000:
starting
2015-04-27 10:22:23,850 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000:
starting
2015-04-27 10:22:23,871 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000:
starting
2015-04-27 10:22:23,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000:
starting
2015-04-27 10:22:23,903 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000:
starting
2015-04-27 10:22:23,920 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000:
starting
2015-04-27 10:22:23,973 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60
000: starting
2015-04-27 10:22:24,016 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60
000: starting
2015-04-27 10:22:24,017 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60
000: starting
2015-04-27 10:22:24,142 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
with processName=Master, sessionId=master,60000,1430101339946
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: revision
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUser
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsDate
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUrl
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: date
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsRevision
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: user
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsVersion
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: url
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: version
2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2015-04-27 10:22:24,288 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2015-04-27 10:22:24,288 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized
2015-04-27 10:22:24,827 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.
mortbay.log) via org.mortbay.log.Slf4jLog
2015-04-27 10:22:25,014 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (clas
s=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2015-04-27 10:22:25,075 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getC
onnectors()[0].getLocalPort() before open() is -1. Opening the listener on 60010
2015-04-27 10:22:25,077 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returne
d 60010 webServer.getConnectors()[0].getLocalPort() returned 60010
2015-04-27 10:22:25,077 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 60010
2015-04-27 10:22:25,077 INFO org.mortbay.log: jetty-6.1.26
2015-04-27 10:22:25,625 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:60010
2015-04-27 10:22:25,705 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Deleting ZNode
for /hbase/backup-masters/master,60000,1430101339946 from backup master directory
2015-04-27 10:22:25,746 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbas
e/backup-masters/master,60000,1430101339946 already deleted, and this is not a retry
2015-04-27 10:22:25,746 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=master,
60000,1430101339946
2015-04-27 10:22:26,154 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Star
ting shutdown.
org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client vers
ion 4
at org.apache.hadoop.ipc.Client.call(Client.java:1070)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy10.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:238)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:203)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:8
9)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:674)
at org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:573)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:433)
at java.lang.Thread.run(Thread.java:662)
2015-04-27 10:22:26,155 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2015-04-27 10:22:26,155 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads
2015-04-27 10:22:26,155 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60000
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000:
exiting
2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000:
exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000:
exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000:
exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000:
exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60
000: exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60
000: exiting
2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60
000: exiting
2015-04-27 10:22:26,158 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on
60000
2015-04-27 10:22:26,160 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2015-04-27 10:22:26,160 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2015-04-27 10:22:26,161 INFO org.apache.hadoop.hbase.master.HMaster: Stopping infoServer
2015-04-27 10:22:26,170 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60010
2015-04-27 10:22:26,224 INFO org.apache.zookeeper.ZooKeeper: Session: 0x34cf8ad10320000 closed
2015-04-27 10:22:26,224 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exitin
g
2015-04-27 10:22:26,224 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to star
t master
java.lang.RuntimeException: HMaster Aborted
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.jav
a:160)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129)
2015-04-27 10:22:26,224 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down