# Brotli Test Data Files without a `.br` suffix, and their `.compressed` (and `.compressed.NN`) counterparts, were copied from the `tests/testdata` directory of the [reference brotli implementation](https://github.com/google/brotli) at tag v1.2.1. That repository is licensed under the MIT license: "Copyright (c) 2009, 2010, 2013-2016 by the Brotli Authors". Some of the upstream files were not copied: - `lcet10.txt`, `mapsdatazrh`, `plrabn12.txt` and `zerosukkanooa`, because their decompressed forms are larger than 256 KiB. - `bb.binast` and `random_chunks`, because they have no `.compressed` form. The `*.qNwM.br` files were generated by the brotli v1.2.1 command line tool, from `../romeo.txt`, `../256.bytes` and `alice29.txt`, at quality level N and with window bits M: brotli -q N -w M -o romeo.txt.qNwM.br ../romeo.txt The small window (`w10` and `w16`) `alice29.txt` files exercise the history ring buffer wrapping around. The `invalid.*.br` files are malformed. Apart from `invalid.truncated.br` (the first 20 bytes of `romeo.txt.q11w22.br`), they were found by randomly mutating the other (valid) files and keeping, for each libbrotlidec v1.2.1 error code, the smallest mutated file that produced that error code. The file name mentions that error code, e.g. `invalid.cl-space.br` produces `BROTLI_DECODER_ERROR_FORMAT_CL_SPACE`. ## Large Window and Dictionary Test Data These files exercise the `QUIRK_LARGE_WINDOW` quirk and `add_dictionary`. They were generated with brotli v1.2.1: the command line tool where it suffices and otherwise a small C program calling `BrotliEncoderSetParameter` (with `BROTLI_PARAM_LARGE_WINDOW`), `BrotliEncoderPrepareDictionary` and `BrotliEncoderAttachPreparedDictionary`, linked against a libbrotlienc built with `BROTLI_EXPERIMENTAL` (needed for serialized shared dictionaries). Every file's decompressed form was checked with libbrotlidec (also built with `BROTLI_EXPERIMENTAL`). - `*.qNlwM.br` are large-window streams (quality N, window bits M), e.g. `brotli -q 11 --large_window=26 -o alice29.txt.q11lw26.br alice29.txt`. - `far17m.q9lw25.br` is a large-window (25 window bits) quality 9 stream whose decompressed form (17 MiB plus 128 KiB, generated by the test program instead of being checked in) is 64 KiB of xorshift32 output, 17 MiB of zeroes and the same 64 KiB again. The second copy refers back more than (1 << 24) bytes. A standard (24 window bits) encoding is twice as large. - `*.raw-romeo.br` use `../romeo.txt` as a raw (LZ77 prefix) dictionary, e.g. `brotli -q 11 -w 22 -D ../romeo.txt -o romeo.txt.q11w22.raw-romeo.br ../romeo.txt`. - `*.raw-chunks.br` use several raw dictionaries (compound dictionary chunks): `alice29.txt`'s bytes `[0 .. 20000)`, `[20000 .. 20001)`, `[20001 .. 20001)` (empty) and `[20001 .. 60000)`, or `../romeo.txt`'s bytes `[0 .. 300)`, `[300 .. 500)` and `[500 .. 942)`. - `shared-dictionary.*.bin` are serialized shared dictionaries (the format parsed by `BrotliSharedDictionaryAttach` in brotli's `c/common/shared_dictionary.c`), built by a script: - `full` has a 500 byte LZ77 prefix (the start of `../romeo.txt`), two custom word lists (common words from `alice29.txt` and `asyoulik.txt`), two custom transform lists (including `BROTLI_TRANSFORM_SHIFT_FIRST` and `BROTLI_TRANSFORM_SHIFT_ALL` transforms), three (word list, transform list) combinations (the third is the built-in pair) and a context map. - `words` has one custom word list, the built-in transforms and no prefix. - `prefix` has only an LZ77 prefix (the start of `asyoulik.txt`). - `utf8` has a word list of UTF-8 (and invalid UTF-8) words and shift transforms with various parameters. - `shared-dictionary.text` is made of `shared-dictionary.full.bin` words, transformed by its transforms. `*.sd-full.br` (and `*.sd-words.br`) were compressed with `shared-dictionary.full.bin` (and `.words.bin`) attached. `asyoulik.txt.q11w18.sd-prefix-raw.br` was compressed with `shared-dictionary.prefix.bin` and then `../romeo.txt`'s first 300 bytes (raw) attached. - `romeo-mutated.sd-utf8.br` is a mutated `romeo.txt.q5w10.br` that decodes (to `romeo-mutated.sd-utf8.txt`) without error when `shared-dictionary.utf8.bin` is attached, and whose dictionary references apply shift transforms to 1, 2, 3 and 4 byte UTF-8 sequences. - `invalid.lw-*.br`, `invalid.raw-*.br` and `invalid.sd-*.br` are malformed. They were found by mutating the files above, like the other `invalid.*.br` files, with the large-window quirk set (`lw`), or the raw (`raw`, the `romeo.txt` chunks) or serialized (`sd`, `shared-dictionary.full.bin`) dictionaries attached.