Java.Lang.Noclassdeffounderror: Org/Apache/Spark/Logging

java.lang.NoClassDefFoundError: org/apache/spark/Logging

org.apache.spark.Logging is available in Spark version 1.5.2 or lower version. It is not in the 2.0.0. Pls change versions as follows

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_2.11</artifactId>
        <version>1.5.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.10</artifactId>
        <version>1.5.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.10</artifactId>
        <version>1.5.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
        <version>1.6.2</version>
    </dependency>

Resolving NoClassDefFoundError: org/apache/spark/Logging exception

Sean Owen suggested in http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Hit-quot-Exception-in-thread-main-java-lang-NoClassDefFoundError/td-p/44486 that "You shouldn't use org.apache.spark.Logging in your app at all. That's likely the problem and solution."

So you should be using the following or higher version

<!-- https://mvnrepository.com/artifact/commons-logging/commons-logging -->
<dependency>
    <groupId>commons-logging</groupId>
    <artifactId>commons-logging</artifactId>
    <version>1.1.1</version>
</dependency>

Updated

I specified that and now I am getting NoClassDefFoundError: org/apache/spark/streaming/dstream/DStream

for the above issue you need following dependency

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.11</artifactId>
    <version>2.0.0</version>
    <scope>provided</scope>
</dependency>

Spark: Exception in thread main java.lang.NoClassDefFoundError: org/apache/spark/Logging

The problem is that org.apache.spark.Logging is only available in Spark 1.5.2 or lower, and this class is a dependency of org.apache.spark.streaming.twitter.

Since Spark 1.5.2 is not available for Scala 2.12, we should downgrade to Scala 2.11.

So a sbt file that makes this application works is the following

name := "SparkStreaming"

version := "0.1"

scalaVersion := "2.11.12"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "1.5.2",
  "org.apache.spark" %% "spark-sql" % "1.5.2",
  "org.apache.spark" %% "spark-streaming" % "1.5.2",
  "org.apache.spark" %% "spark-streaming-twitter" % "1.6.3"
)

Zeppelin: java.lang.NoClassDefFoundError: org/apache/spark/Logging

I found the solution by adding the jar

spark-core_2.11-1.5.2.logging.jar

---

Related Topics