title: Decompressing LZ4 compressed data in Spark
body: <p>I have LZ4 compressed data in HDFS and I'm trying to decompress it in Apache Spark into a RDD. As far as I can tell, the only method in <code>JavaSparkContext</code> to read data from HDFS is <code>textFile</code> which only reads data as it is in HDFS. I have come across articles on <code>CompressionCodec</code> but all of them explain how to compress output to HDFS whereas I need to decompress what is already on HDFS.</p>
  
  <p>I am new to Spark so I apologize in advance if I missed something obvious or if my conceptual understanding is incorrect but it would be great if someone could point me in the right direction.</p>