public class SnappyCompressorOutputStream extends CompressorOutputStream
This implementation uses an internal buffer in order to handle the back-references that are at the heart of the LZ77 algorithm. The size of the buffer must be at least as big as the biggest offset used in the compressed stream. The current version of the Snappy algorithm as defined by Google works on 32k blocks and doesn't contain offsets bigger than 32k which is the default block size used by this class.
The raw Snappy format requires the uncompressed size to be
written at the beginning of the stream using a varint
representation, i.e. the number of bytes needed to write the
information is not known before the uncompressed size is
known. We've chosen to make the uncompressedSize a parameter of the
constructor in favor of buffering the whole output until the size
is known. When using the FramedSnappyCompressorOutputStream
this limitation is taken care of by the warpping framing
format.
Modifier and Type | Field and Description |
---|---|
private LZ77Compressor |
compressor |
private ByteUtils.ByteConsumer |
consumer |
private boolean |
finished |
private static int |
FOUR_BYTE_COPY_TAG |
private static int |
FOUR_SIZE_BYTE_MARKER |
private static int |
MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE |
private static int |
MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES |
private static int |
MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES |
private static int |
MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES |
private static int |
MAX_MATCH_LENGTH |
private static int |
MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE |
private static int |
MAX_OFFSET_WITH_ONE_OFFSET_BYTE |
private static int |
MAX_OFFSET_WITH_TWO_OFFSET_BYTES |
private static int |
MIN_MATCH_LENGTH |
private static int |
MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE |
private static int |
ONE_BYTE_COPY_TAG |
private static int |
ONE_SIZE_BYTE_MARKER |
private byte[] |
oneByte |
private java.io.OutputStream |
os |
private static int |
THREE_SIZE_BYTE_MARKER |
private static int |
TWO_BYTE_COPY_TAG |
private static int |
TWO_SIZE_BYTE_MARKER |
Constructor and Description |
---|
SnappyCompressorOutputStream(java.io.OutputStream os,
long uncompressedSize)
Constructor using the default block size of 32k.
|
SnappyCompressorOutputStream(java.io.OutputStream os,
long uncompressedSize,
int blockSize)
Constructor using a configurable block size.
|
SnappyCompressorOutputStream(java.io.OutputStream os,
long uncompressedSize,
Parameters params)
Constructor providing full control over the underlying LZ77 compressor.
|
Modifier and Type | Method and Description |
---|---|
void |
close() |
static Parameters.Builder |
createParameterBuilder(int blockSize)
Returns a builder correctly configured for the Snappy algorithm using the gven block size.
|
void |
finish()
Compresses all remaining data and writes it to the stream,
doesn't close the underlying stream.
|
void |
write(byte[] data,
int off,
int len) |
void |
write(int b) |
private void |
writeBackReference(LZ77Compressor.BackReference block) |
private void |
writeBackReferenceWithFourOffsetBytes(int len,
int offset) |
private void |
writeBackReferenceWithLittleEndianOffset(int tag,
int offsetBytes,
int len,
int offset) |
private void |
writeBackReferenceWithOneOffsetByte(int len,
int offset) |
private void |
writeBackReferenceWithTwoOffsetBytes(int len,
int offset) |
private void |
writeLiteralBlock(LZ77Compressor.LiteralBlock block) |
private void |
writeLiteralBlockFourSizeBytes(LZ77Compressor.LiteralBlock block,
int len) |
private void |
writeLiteralBlockNoSizeBytes(LZ77Compressor.LiteralBlock block,
int len) |
private void |
writeLiteralBlockOneSizeByte(LZ77Compressor.LiteralBlock block,
int len) |
private void |
writeLiteralBlockThreeSizeBytes(LZ77Compressor.LiteralBlock block,
int len) |
private void |
writeLiteralBlockTwoSizeBytes(LZ77Compressor.LiteralBlock block,
int len) |
private void |
writeLiteralBlockWithSize(int tagByte,
int sizeBytes,
int len,
LZ77Compressor.LiteralBlock block) |
private void |
writeLittleEndian(int numBytes,
int num) |
private void |
writeUncompressedSize(long uncompressedSize) |
private final LZ77Compressor compressor
private final java.io.OutputStream os
private final ByteUtils.ByteConsumer consumer
private final byte[] oneByte
private boolean finished
private static final int MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES
private static final int MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE
private static final int MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES
private static final int MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES
private static final int ONE_SIZE_BYTE_MARKER
private static final int TWO_SIZE_BYTE_MARKER
private static final int THREE_SIZE_BYTE_MARKER
private static final int FOUR_SIZE_BYTE_MARKER
private static final int MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
private static final int MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
private static final int MAX_OFFSET_WITH_ONE_OFFSET_BYTE
private static final int MAX_OFFSET_WITH_TWO_OFFSET_BYTES
private static final int ONE_BYTE_COPY_TAG
private static final int TWO_BYTE_COPY_TAG
private static final int FOUR_BYTE_COPY_TAG
private static final int MIN_MATCH_LENGTH
private static final int MAX_MATCH_LENGTH
public SnappyCompressorOutputStream(java.io.OutputStream os, long uncompressedSize) throws java.io.IOException
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of datajava.io.IOException
- if writing of the size failspublic SnappyCompressorOutputStream(java.io.OutputStream os, long uncompressedSize, int blockSize) throws java.io.IOException
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of datablockSize
- the block size used - must be a power of twojava.io.IOException
- if writing of the size failspublic SnappyCompressorOutputStream(java.io.OutputStream os, long uncompressedSize, Parameters params) throws java.io.IOException
os
- the outputstream to write compressed data touncompressedSize
- the uncompressed size of dataparams
- the parameters to use by the compressor - note
that the format itself imposes some limits like a maximum match
length of 64 bytesjava.io.IOException
- if writing of the size failspublic void write(int b) throws java.io.IOException
write
in class java.io.OutputStream
java.io.IOException
public void write(byte[] data, int off, int len) throws java.io.IOException
write
in class java.io.OutputStream
java.io.IOException
public void close() throws java.io.IOException
close
in interface java.io.Closeable
close
in interface java.lang.AutoCloseable
close
in class java.io.OutputStream
java.io.IOException
public void finish() throws java.io.IOException
java.io.IOException
- if an error occursprivate void writeUncompressedSize(long uncompressedSize) throws java.io.IOException
java.io.IOException
private void writeLiteralBlock(LZ77Compressor.LiteralBlock block) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockNoSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockOneSizeByte(LZ77Compressor.LiteralBlock block, int len) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockTwoSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockThreeSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockFourSizeBytes(LZ77Compressor.LiteralBlock block, int len) throws java.io.IOException
java.io.IOException
private void writeLiteralBlockWithSize(int tagByte, int sizeBytes, int len, LZ77Compressor.LiteralBlock block) throws java.io.IOException
java.io.IOException
private void writeLittleEndian(int numBytes, int num) throws java.io.IOException
java.io.IOException
private void writeBackReference(LZ77Compressor.BackReference block) throws java.io.IOException
java.io.IOException
private void writeBackReferenceWithOneOffsetByte(int len, int offset) throws java.io.IOException
java.io.IOException
private void writeBackReferenceWithTwoOffsetBytes(int len, int offset) throws java.io.IOException
java.io.IOException
private void writeBackReferenceWithFourOffsetBytes(int len, int offset) throws java.io.IOException
java.io.IOException
private void writeBackReferenceWithLittleEndianOffset(int tag, int offsetBytes, int len, int offset) throws java.io.IOException
java.io.IOException
public static Parameters.Builder createParameterBuilder(int blockSize)
blockSize
- the block size.