在执行hive运行脚本时,出现了MetaException Metastore contains multiple versions异常错误
Exception in thread “main” java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
… 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
… 14 more
Caused by: MetaException(message:Metastore contains multiple versions (2) [ version = 1.2.0, comment = Set by MetaStore spark@10.50.70.120 ] [ version = 1.2.0, comment = Set by MetaStore spark@10.50.70.121 ])
at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6745)
at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:6703)
at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6661)
at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.
P
r
o
x
y
6.
v
e
r
i
f
y
S
c
h
e
m
a
(
U
n
k
n
o
w
n
S
o
u
r
c
e
)
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
h
i
v
e
.
m
e
t
a
s
t
o
r
e
.
H
i
v
e
M
e
t
a
S
t
o
r
e
Proxy6.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore
Proxy6.verifySchema(UnknownSource)atorg.apache.hadoop.hive.metastore.HiveMetaStoreHMSHandler.getMS(HiveMetaStore.java:572)
at org.apache.hadoop.hive.metastore.HiveMetaStore
H
M
S
H
a
n
d
l
e
r
.
c
r
e
a
t
e
D
e
f
a
u
l
t
D
B
(
H
i
v
e
M
e
t
a
S
t
o
r
e
.
j
a
v
a
:
620
)
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
h
i
v
e
.
m
e
t
a
s
t
o
r
e
.
H
i
v
e
M
e
t
a
S
t
o
r
e
HMSHandler.createDefaultDB(HiveMetaStore.java:620) at org.apache.hadoop.hive.metastore.HiveMetaStore
HMSHandler.createDefaultDB(HiveMetaStore.java:620)atorg.apache.hadoop.hive.metastore.HiveMetaStoreHMSHandler.init(HiveMetaStore.java:461)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
… 19 more
在spark中使用hivecontext时也会遇到这个异常,原因在于hive的metastore表VERSION中存在重复的记录,从而报出存在多条hive版本的异常信息
VER_ID SCHEMA_VERSION VERSION_COMMENT
ID主键 Hive版本 版本说明
1 0.13.0 Set by MetaStore
解决办法:
- 删除VERSION表中多余的版本信息即可解决
2.修改hive-site.xml中配置,取消metastore版本信息校验
hive.metastore.schema.verification false 这个异常信息多次出现,每次都是由spark的一台work插入进去,正好这台机器是虚拟机节点非物理机,猜测是在这台虚拟机网络环境不好的时候,检查不到hive metastore中version版本信息再插入一条新的记录导致重复异常的
————————————————
版权声明:本文为CSDN博主「xubc」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://blog.csdn.net/u011098327/article/details/54601229