This change enforces the new 16MB limit on the
size of the meta-blocks and adds support for
empty meta-blocks with optional ignored metadata.

Support empty meta-blocks with optional ignored metadata.

This is a partially backward incompatible format change,
that makes previously valid brotli streams that contain
larger than 16MB meta-blocks invalid.

The impact of this should be minimal, since the 'bro'
command-line tool does not create larger than 2MB
meta-blocks, so the only streams this change could
break are those created by a custom brotli encoder.

This commit contains only the specification update,
implementation in the decoder and encoder will
follow in later commits.

Change the specification to be less strict in some cases.

In the following three cases we allow more choices
for the compressor, which can potentially lead to
less compressed bits.

  (1) Allow brotli streams where the block counts
      do not count down to exactly zero at the end
      of the meta-block. This makes it possible
      for compressors to sometimes choose a block
      count which can be represented with less bits
      than the exact block count.

  (2) Remove the restriction that prefix code
      descriptions with exactly one non-zero
      length symbol in the code length alphabet
      must have 1 bit depth. This is because
      bit depth 1 requires the most bits to encode.

  (3) Allow any copy length value in the last
      command where the copy part is ignored.
      This makes it possible for a compressor
      to choose a copy length which can be
      represented with the least amount of bits.

In addition to the changes above, this commit also
has a wording clarification in the overview section
where the use of the 'context ID' expression is
changed to be consistent with the rest of the
specification, i.e. that it is a function of the
last two literals or the copy length.

Use consistent sentence spacing in the specification.

All sentence spacing was changed to one space, except
in the boilerplate which must be preserved verbatim.

Add Mark Adler's edits to the specification.

The specification source is changed in this commit
to exactly mirror the specification edited by Mark Adler:

https://github.com/madler/brotli/blob/master/brotli-02-edit.nroff
(version 70e53d7)

[python] fix unbuffered binary mode for I/O streams in Windows pypy

Spec clarifications for Section 7.

Based on Mark Adler's review comments.

Spec clarifications for Section 8.

Based on Mark Adler's review comments.

Some wording changes to Section 2 of the spec.

Add the current version of the brotli specification.

Add the .nroff source file and a python script to generate
the .txt version (requires the nroff command).

Limit the max input meta-block size to 16MB.

Proof-of-concept encoder for parallel compression.

Add a version of the brotli encoder that compresses
each meta-block independently, only using the
original input data from previous meta-blocks
and nothing from the compressor state.
This is a proof-of-concept to show that the
current format is flexible enough to support
parallel multi-threaded compression.

Add an input block size parameter to brotli.

This will enable processing the input in smaller
chunks than the currently default 2MB for the
slow brotli, while still benefiting from the
larger sliding window.

Update the previous two bytes when decoding an uncompressed metablock.

uncompressed metablock.

Add a missing state transition to CopyUncompressedBlockToOutput().

Faster decoder for data with trivial context maps.

This makes it 20% faster when decoding data with
trivial context map (where the histogram type
depends only on the block type and not the
context).

Add quality and lgwin to the BrotliParams.

Remove the hard-coded constants for window size
and meta-block size.

Initialize internal storage for each metablock
separately and reserve only as much as needed
for the actual input.

[python] moved setup.py to the root of repository

bugfixes affecting streaming decoding

Use FastLog2() instead of log() in BitsEntropy().

Add some missing state transitions to DecodeContextMap().