0
Kann jemand die folgenden doppelten Dateifehler vermeiden?Funken App [Fehler] (*: Assembly) Deduplizierung: verschiedene Dateiinhalte im folgenden gefunden:
Ich habe eine Spark-Anwendung, die in der InteliJ Community Edition bis letzte Nacht gearbeitet hat und aus keinem ersichtlichen Grund verursachen nun zwei Gläser von esotericsoftware Fehler. Wenn ich jetzt sbt Assembly ausführe, erhalte ich folgende Fehlermeldung (en).
[warn] Merging 'META-INF\taglib.tld' with strategy 'discard'
java.lang.RuntimeException: deduplicate: different file contents found in the following:
C:\Users\osadmin\.ivy2\cache\com.esotericsoftware.kryo\kryo\bundles\kryo-2.21.jar:com/esotericsoftware/minlog/Log$Logger.class
C:\Users\osadmin\.ivy2\cache\com.esotericsoftware.minlog\minlog\jars\minlog-1.2.jar:com/esotericsoftware/minlog/Log$Logger.class
at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$$applyStrategy$1(Plugin.scala:253)
at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:270)
at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:267)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272)
at sbtassembly.Plugin$Assembly$.x$4$lzycompute$1(Plugin.scala:172)
at sbtassembly.Plugin$Assembly$.x$4$1(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute$1(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.stratMapping$1(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.inputs$lzycompute$1(Plugin.scala:214)
at sbtassembly.Plugin$Assembly$.inputs$1(Plugin.scala:204)
at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230)
at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:373)
at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:370)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] C:\Users\osadmin\.ivy2\cache\com.esotericsoftware.kryo\kryo\bundles\kryo-2.21.jar:com/esotericsoftware/minlog/Log$Logger.class
[error] C:\Users\osadmin\.ivy2\cache\com.esotericsoftware.minlog\minlog\jars\minlog-1.2.jar:com/esotericsoftware/minlog/Log$Logger.class
Ich benutze sbt und habe eine mergeStrategy wie folgt.
libraryDependencies ++= Seq("org.apache.spark" % "spark-core_2.10" % "1.5.2" % "provided" exclude("org.apache.hadoop", "hadoop-client"),
("org.mongodb" % "mongo-java-driver" % "3.2.2")
.exclude("commons-beanutils", "commons-beanutils-core")
.exclude("commons-beanutils", "commons-beanutils") ,
("org.mongodb.mongo-hadoop" % "mongo-hadoop-core" % "1.4.2")
.exclude("commons-beanutils", "commons-beanutils-core")
.exclude("commons-beanutils", "commons-beanutils"),
("com.stratio.datasource" % "spark-mongodb_2.10" % "0.11.0"),
("org.apache.spark" % "spark-sql_2.10" % "1.5.2" % "provided"),
("org.apache.spark" % "spark-hive_2.10" % "1.5.2" % "provided"),
("org.apache.spark" % "spark-streaming_2.10" % "1.5.2")
)
assemblySettings
jarName in assembly := "mongotest.jar"
val meta = """META.INF(.)*""".r
mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
{
case PathList("org", "apache", xs @ _*) => MergeStrategy.last
case PathList("plugin.properties") => MergeStrategy.last
case meta(_) => MergeStrategy.discard
case x => old(x)
}
}
assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = false)