Spark Serialization & Compression

Serialization

• Serializer

Java Serialization

• JavaSerializer

• SerializerInstance(JavaSerializerInstance)

• JavaSerializationStream, JavaDeserializationStream

Kryo serialization

• KryoSerializer

• SerializerInstance(KryoSerializerInstance)

• JavaIterableWrapperSerializer, KryoInputObjectInputBridge, KryoOutputObjectOutputBridge

• KryoSerializationStream, KryoDeserializationStream

Compression

• CompressionCodec

• compressedOutputStream, compressedInputStream

lz4

• LZ4CompressionCodec

lzf

• LZFCompressionCodec

snappy

• SnappyCompressionCodec

zstd

• ZStdCompressionCodec

Links

• Spark Configuration - Compression and Serialization
• Spark Documentation

---

• Author：HyperJ
• Source：HyperJ’s Blog
• Link：Spark Serialization & Compression