我正在尝试提交 Spark 作业
它是这样开始的:
import javax.xml.parsers.{SAXParser, SAXParserFactory}
import org.apache.spark
import org.apache.spark.graphx.{Graph, Edge, VertexId}
import org.apache.spark.rdd.{PairRDDFunctions, RDD}
import org.apache.spark.storage.StorageLevel
import org.apache.spark.{SparkContext, SparkConf}
import scala.util.Try
import org.apache.log4j.{Level, Logger}
object MyApp {
def main(args: Array[String]) {
val sparkConf = new SparkConf().setAppName("MyApp")
val sc = new SparkContext(sparkConf)
当我启动它时,出现以下错误:
App > Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.VolatileObjectRef.zero()Lscala/runtime/VolatileObjectRef;
App > at MyApp$.main(MyApp.scala)
App > at MyApp.main(MyApp.scala)
App > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
App > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
App > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
App > at java.lang.reflect.Method.invoke(Method.java:606)
App > at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
App > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
App > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
我究竟做错了什么?
EDIT:包括完整的堆栈跟踪。
使用 Scala 2.10 和 Spark 1.2.0。
奇怪的是,在我的罐子里,我有两个课程。当我触发提交一个作业时,它可以工作(这是一个 4 行的虚拟作业),但是当我运行较长的作业(大约 40 行)时,如果失败并出现上述错误
zero()
on scala.runtime.VolatileObjectRef
已在Scala 2.11中引入
您可能有一个针对 Scala 2.11 编译并在 Scala 2.10 运行时上运行的库。
See
- v2.10: https://github.com/scala/scala/blob/2.10.x/src/library/scala/runtime/VolatileObjectRef.java https://github.com/scala/scala/blob/2.10.x/src/library/scala/runtime/VolatileObjectRef.java
- v2.11: https://github.com/scala/scala/blob/2.11.x/src/library/scala/runtime/VolatileObjectRef.java https://github.com/scala/scala/blob/2.11.x/src/library/scala/runtime/VolatileObjectRef.java
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)