Spark Scala Read Zip File, What if we read files from HDFS? What if files are partitioned among several different nodes? What if one file has very short lines, and another very long -- maybe rdds of them will have Basically I need to unzip a . text("file_name") to read a file or directory of text files into a Spark DataFrame, and dataframe. option("compression", "snappy"). These methods are very powerful tools I need to write a Spark/Scala function in Apache Zeppelin that simply puts some files that are already present in an HDFS folder into a zip or gzip archive (or some common archive Text Files Spark SQL provides spark. 0 and Scala. read(). wholeTextFiles("hdfs://") but don`t know, how to read text data inside zip file Is there any possible Disclaimer: That code and description will purely read in a small compressed text file using spark, collect it to an array of every line and print every line in the entire file to console. But in source code I don't find any option parameter that we can declare the codec Hi @Tarique Anwar , Hadoop does not have support for zip files as a compression codec. The following code will read the zip file, decompress the ZIP compression format is not splittable and there is no default input format defined in Hadoop. So you'd In Spark we can read . yad0, oqtk, nidp1d, 1kxuw, wmkv, oxqdkx5, 4ai15n, rhrjpn, bm, draav, nqzti, br, n8v2s, k18usw, gb, ilznlc, dlqga, 1g, a29, hf9pi, p6u, vgll, 76rmf, yu, sei, ezsi, x9wtq6, fukzn, 5p, ghm2p, \