title: Kafka Structured Streaming java.lang.NoClassDefFoundError
body: <p>I was able to run Kafka structure streaming programming before. But suddenly all my structure streaming python programs are failing with an error. I took basic Kafka structure streaming programming from Spark website that is also failing with same error.</p>
  
  <blockquote>
    <p>py4j.protocol.Py4JJavaError: An error occurred while calling o31.load.
    : java.lang.NoClassDefFoundError:
    org/apache/kafka/common/serialization/ByteArrayDeserializer
            at org.apache.spark.sql.kafka010.KafkaSourceProvider$.(KafkaSourceProvider.scala:376)
            at org.apache.spark.sql.kafka010.KafkaSourceProvider$.(KafkaSourceProvider.scala)</p>
  </blockquote>
  
  <p>Spark submit I am using</p>
  
  <blockquote>
    <p>spark-submit --packages
    org.apache.spark:spark-sql-kafka-0-10_2.11:2.2.0
    C:\Users\ranjith.gangam\PycharmProjects\sparktest\Structured_streaming.py</p>
  </blockquote>
  
  <p>This is a code I took from Spark github</p>
  
  <pre class="lang-py prettyprint-override"><code>spark = SparkSession\
        .builder\
        .appName("StructuredKafkaWordCount")\
        .getOrCreate()
  
  # Create DataSet representing the stream of input lines from kafka
  lines = spark\
      .readStream\
      .format("kafka")\
      .option("kafka.bootstrap.servers", bootstrapServers)\
      .option(subscribeType, topics)\
      .load()\
      .selectExpr("CAST(value AS STRING)")
  
  words = lines.select(
      # explode turns each item in an array into a separate row
      explode(
          split(lines.value, ' ')
      ).alias('word')
  )
  
  # Generate running word count
  wordCounts = words.groupBy('word').count()
  # Start running the query that prints the running counts to the console
  query = wordCounts\
      .writeStream\
      .outputMode('complete')\
      .format('console')\
      .start()
  
  query.awaitTermination()
  </code></pre>