minor specification clarifications, suggested by @ebiggers
This commit is contained in:
+13
-14
@@ -506,7 +506,7 @@ Compressed and regenerated size fields follow big endian convention.
|
|||||||
This section is only present when literals block type is `Compressed` (`0`).
|
This section is only present when literals block type is `Compressed` (`0`).
|
||||||
|
|
||||||
Prefix coding represents symbols from an a priori known alphabet
|
Prefix coding represents symbols from an a priori known alphabet
|
||||||
by bit sequences (codes), one code for each symbol,
|
by bit sequences (codewords), one codeword for each symbol,
|
||||||
in a manner such that different symbols may be represented
|
in a manner such that different symbols may be represented
|
||||||
by bit sequences of different lengths,
|
by bit sequences of different lengths,
|
||||||
but a parser can always parse an encoded string
|
but a parser can always parse an encoded string
|
||||||
@@ -515,14 +515,13 @@ unambiguously symbol-by-symbol.
|
|||||||
Given an alphabet with known symbol frequencies,
|
Given an alphabet with known symbol frequencies,
|
||||||
the Huffman algorithm allows the construction of an optimal prefix code
|
the Huffman algorithm allows the construction of an optimal prefix code
|
||||||
using the fewest bits of any possible prefix codes for that alphabet.
|
using the fewest bits of any possible prefix codes for that alphabet.
|
||||||
Such a code is called a Huffman code.
|
|
||||||
|
|
||||||
Prefix code must not exceed a maximum code length.
|
Prefix code must not exceed a maximum code length.
|
||||||
More bits improve accuracy but cost more header size,
|
More bits improve accuracy but cost more header size,
|
||||||
and require more memory for decoding operations.
|
and require more memory for decoding operations.
|
||||||
|
|
||||||
The current format limits the maximum depth to 15 bits.
|
The current format limits the maximum depth to 15 bits.
|
||||||
The reference decoder goes further, by limiting it to 11 bits.
|
The reference decoder goes further, by limiting it to 12 bits.
|
||||||
It is recommended to remain compatible with reference decoder.
|
It is recommended to remain compatible with reference decoder.
|
||||||
|
|
||||||
|
|
||||||
@@ -618,20 +617,19 @@ When both states have overflowed the bitstream, end is reached.
|
|||||||
##### Conversion from weights to huffman prefix codes
|
##### Conversion from weights to huffman prefix codes
|
||||||
|
|
||||||
All present symbols shall now have a `weight` value.
|
All present symbols shall now have a `weight` value.
|
||||||
A `weight` directly represents a `range` of prefix codes,
|
|
||||||
following the formulae : `range = weight ? 1 << (weight-1) : 0 ;`
|
|
||||||
Symbols are sorted by weight.
|
Symbols are sorted by weight.
|
||||||
|
Symbols with a weight of zero are removed.
|
||||||
Within same weight, symbols keep natural order.
|
Within same weight, symbols keep natural order.
|
||||||
Starting from lowest weight,
|
Starting from lowest weight,
|
||||||
symbols are being allocated to a range of prefix codes.
|
symbols are being allocated to a `range`.
|
||||||
Symbols with a weight of zero are not present.
|
A `weight` directly represents a `range`,
|
||||||
|
following the formulae : `range = weight ? 1 << (weight-1) : 0 ;`
|
||||||
It is then possible to transform weights into nbBits :
|
Similarly, it is possible to transform weights into nbBits :
|
||||||
`nbBits = nbBits ? maxBits + 1 - weight : 0;` .
|
`nbBits = nbBits ? maxBits + 1 - weight : 0;` .
|
||||||
|
|
||||||
|
|
||||||
__Example__ :
|
__Example__ :
|
||||||
Let's presume the following huffman tree has been decoded :
|
Let's presume the following list of weights has been decoded :
|
||||||
|
|
||||||
| Literal | 0 | 1 | 2 | 3 | 4 | 5 |
|
| Literal | 0 | 1 | 2 | 3 | 4 | 5 |
|
||||||
| ------- | --- | --- | --- | --- | --- | --- |
|
| ------- | --- | --- | --- | --- | --- | --- |
|
||||||
@@ -644,8 +642,9 @@ it gives the following distribution :
|
|||||||
| ------------ | --- | --- | --- | --- | --- | ---- |
|
| ------------ | --- | --- | --- | --- | --- | ---- |
|
||||||
| weight | 0 | 1 | 1 | 2 | 3 | 4 |
|
| weight | 0 | 1 | 1 | 2 | 3 | 4 |
|
||||||
| range | 0 | 1 | 1 | 2 | 4 | 8 |
|
| range | 0 | 1 | 1 | 2 | 4 | 8 |
|
||||||
| prefix codes | N/A | 0 | 1 | 2-3 | 4-7 | 8-15 |
|
| table entries| N/A | 0 | 1 | 2-3 | 4-7 | 8-15 |
|
||||||
| nb bits | 0 | 4 | 4 | 3 | 2 | 1 |
|
| nb bits | 0 | 4 | 4 | 3 | 2 | 1 |
|
||||||
|
| prefix codes | N/A | 0000| 0001| 001 | 01 | 1 |
|
||||||
|
|
||||||
|
|
||||||
#### Literals bitstreams
|
#### Literals bitstreams
|
||||||
@@ -696,12 +695,12 @@ it's possible to read the bitstream in a little-endian fashion,
|
|||||||
keeping track of already used bits.
|
keeping track of already used bits.
|
||||||
|
|
||||||
Reading the last `maxBits` bits,
|
Reading the last `maxBits` bits,
|
||||||
it's then possible to compare extracted value to the prefix codes table,
|
it's then possible to compare extracted value to decoding table,
|
||||||
determining the symbol to decode and number of bits to discard.
|
determining the symbol to decode and number of bits to discard.
|
||||||
|
|
||||||
The process continues up to reading the required number of symbols per stream.
|
The process continues up to reading the required number of symbols per stream.
|
||||||
If a bitstream is not entirely and exactly consumed,
|
If a bitstream is not entirely and exactly consumed,
|
||||||
hence reaching exactly its beginning position with all bits consumed,
|
hence reaching exactly its beginning position with _all_ bits consumed,
|
||||||
the decoding process is considered faulty.
|
the decoding process is considered faulty.
|
||||||
|
|
||||||
|
|
||||||
@@ -713,7 +712,7 @@ A literal copy command specifies a length.
|
|||||||
It is the number of bytes to be copied (or extracted) from the literal section.
|
It is the number of bytes to be copied (or extracted) from the literal section.
|
||||||
A match copy command specifies an offset and a length.
|
A match copy command specifies an offset and a length.
|
||||||
The offset gives the position to copy from,
|
The offset gives the position to copy from,
|
||||||
which can stand within a previous block.
|
which can be within a previous block.
|
||||||
|
|
||||||
There are 3 symbol types, `literalLength`, `matchLength` and `offset`,
|
There are 3 symbol types, `literalLength`, `matchLength` and `offset`,
|
||||||
which are encoded together, interleaved in a single _bitstream_.
|
which are encoded together, interleaved in a single _bitstream_.
|
||||||
|
|||||||
Reference in New Issue
Block a user