| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Make sure the travis CI aarch32 bot tests NEON, and also that running
CROSS_COMPILE=arm-linux-gnueabihf make enables the use of NEON to
accelerate the back-reference copying.
|
|
|
|
* Fix typo / minor formatting / pull computable constant to the place of use.
|
|
The macro that checks for NEON support should be __ARM_NEON, not
__ARM_NEON__. [1]
AArch64 compilers define __ARM_NEON but not __ARM_NEON__.
AArch32 compilers currently seem to define both, but could be within their
rights to drop __ARM_NEON__ in future versions.
This change moves the check into the common/platform.h file, checks for
both forms, and sets BROTLI_TARGET_NEON if NEON support is available.
[1] Section 6.5.4 of the ARM C Language Extensions.
(At the time of writing, the latest version was Release 2.1.)
|
|
Window size is defined as:
`(1 << BROTLI_PARAM_LGWIN) - 16`
in `c/include/brotli/encode.h`
Therefore we should probably take these 16 bytes into account.
Done basic manual testing:
$ python3 -c 'print ("A"*2046)' > t
$ bazel run -- //:brotli -w 0 -f -o $(realpath t).br $(realpath ./t)
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000 c1 1000001 WSIZE windowsize=(1<<12)-16=4080
New version properly detects window size of `4080`, while previous one used `2032`:
$ python3 research/brotlidump.py t.br |& fgrep WSIZE
0000 b1 0110001 WSIZE windowsize=(1<<11)-16=2032
|
|
* platform: add macro for using the 'aligned' attribute
* decode: add accessor macros for HuffmanCode fields
Adds a constructor function for building HuffmanCode values
so they can be accessed quickly on different architectures.
Also adds macros for marking a HuffmanCode table pointer
that can be accessed quickly (BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD),
adjusting the index into that table (BROTLI_HC_ADJUST_TABLE_INDEX),
and getting the .bits or .value fields out of the table at the
current index (BROTLI_HC_GET_BITS/VALUE).
For example, assuming |table| contains a HuffmanCode pointer:
BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD(table);
BROTLI_HC_ADJUST_TABLE_INDEX(table, index_into_table);
*bits = BROTLI_HC_GET_BITS(table);
*value = BROTLI_HC_GET_VALUE(table);
BROTLI_HC_ADJUST_TABLE_INDEX(table, offset);
*bits2 = BROTLI_HC_GET_BITS(table);
*value2 = BROTLI_HC_GET_VALUE(table);
All uses of the HuffmanCode have been updated appropriately.
* decode: add alternative accessors for HuffmanCode on Arm AArch32
|
|
* build: add cross-compilation support to make
Set CROSS_COMPILE when running make to use the selected cross
compilation toolchain, such as arm-linux-gnueabihf, or
aarch64-linux-gnu.
Testing requires the presence of qemu - 'qemu-$(ARCH)' will be executed,
where ARCH is the first part of the toolchain triplet.
* build: add cross-compilation support to cmake
If C_COMPILER/CXX_COMPILER/CC/CXX are found to have cross-compilation
triplets in front of the compiler, then qemu will be used to execute the
tests.
* CI: add arm-linux-gnueabihf-gcc builder to Travis
The version of qemu available in Ubuntu trusty (as provided by Travis)
appears to have a bug in qemu-aarch64, which leads to the compatibility
tests failing on some inputs, erroneously rejecting the input as
corrupt.
Once Travis supports xenial, we could add an aarch64-gnu-linux-gcc
builder as well.
* CI: propagate cmake errors out of .travis.sh
Seems like even if cmake fails, the error isn't picked up by Travis.
|
|
Set bundled mode to ON when parent directory is not empty. Due to the
peculiarities of CMake if, comparing an undefined variable to the empty
string is false, so this likely never triggered.
|
|
Update
* add ASAN/MSAN unaligned read specializations
* add "brotli" prefix to u_uint64 type
* increment version to 1.0.06
* fix CoverityScan "unused assignment" warning
* fix JDK 8<->9 incompatibility
* add encoder optimization for empty input
* regenerate JS decoder
* unbreak Travis builds
|
|
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.
This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.
Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
|
|
This reverts commit 6d027d1648d957c5fee92e535a66420413fd6537.
|
|
Ensures that Aarch32 Arm builds with an Armv8 compiler do not set
BROTLI_64_BITS.
This scenario is possible with ChromeOS builds, as they may use a
toolchain with the target armv7-cros-gnueabi, but with -march=armv8.
This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will
also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this,
illegal 64-bit non-word-aligned reads (LDRD) may be emitted.
Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to
emit.
|
|
Our dist tarball is missing hash_rolling_inc.h and
hash_composite_inc.h, which causes subsequent autotools
builds to fail. Fix this by adding it to the sources list.
Signed-off-by: William A. Kennington III <william@wkennington.com>
|
|
|
|
* Update go_library to use standard importpath
Instead of using go_prefix, which is deprecated, the importpath attribute is made explicit.
* Add description to go/BUILD
|
|
* add rolling-composite-hasher for large-window mode
* make API methods explicitly public
|
|
|
|
|
|
|
|
* fix MSVC warnings
* cleanups
|
|
* Inverse bazel workspace tree.
Now each subproject directly depends on root (c) project.
This helps to mitigate Bazel bug bazelbuild/bazel#2391; short summary:
Bazel does not work if referenced subproject `WORKSPACE` uses any
repositories that embedding project does not.
Bright side: building C project is much faster;
no need to download closure, go and JDK...
|
|
|
|
Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
|
|
|
|
|
|
* Update
* fix ifdef style
* get back to fine-compiler-version-based-macros (use Hedley)
* fix q=0 histogram collection for very long copy/insert commands
|
|
Installing with --user will leave the old pip.exe script in the $PATH,
but running this will fail because pip 10 moved 'main' to internal
modules.
https://github.com/pypa/pip/issues/5240#issuecomment-382989420
|
|
|
|
* Update
* improve q=1 compression on small files
* fix "left shift before promotion"
* fix osx Travis builds
|
|
* tell bazel not to pass strict options to a fancy compiler
* fix signed-unsigned comparison warning found by MSVC
|
|
* proper fix for the "fall through" warning"
* automatic NDIRECT/NPOSTFIX tuning (better compression)
* fix unaligned access for `aarch64`-cross-`armhf` build
* fix `aarch64` detection (10% decoder speedup)
* expose `large_window` CLI option
* make default window size 16MiB
* ramp up version to 1.0.4
|
|
By default CMake checks both for C and C++ compilers, while the latter
is not needed. Setting the list of languages to just "C" in the call to
project() removes the unneeded check.
|
|
OOMs on RPi (1GB)
|
|
Add mention of Dart native bindings
|
|
* fix `bazel` build (ignore switch case fall-through)
* add `NPOSTFIX` / `NDIRECT` encoder parameters
* fix source file lists (add `params.h`)
* fix bug in `durchschlag`
* print clarifying messages wheb CLI argument parsing fails
|
|
Update
* make the zopflification aware of `NDIRECT`, `NPOSTFIX`
(better compression in `font` mode)
* add small and simple decoder tool
* fix typo
* Java: wrapper: make decoder channel more async-friendly
Ramp up version to 1.0.3 / 1.0.3
|
|
* New feature: "Large Window Brotli"
By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.
Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).
Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.
Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.
Other C changes:
- combined / moved `context.h` to `common`
- moved transforms to `common`
- unified some aspects of code formatting
- added an abstraction for encoder (static) dictionary
- moved default allocator/deallocator functions to `common`
brotli CLI:
- window size is auto-adjusted if not specified explicitly
Java:
- added "eager" decoding both to JNI wrapper and pure decoder
- huge speed-up of `DictionaryData` initialization
* Add dictionaryless compressed dictionary
* Fix `sources.lst`
* Fix `sources.lst` and add a note that `libtool` is also required.
* Update setup.py
* Fix `EagerStreamTest`
* Fix BUILD file
* Add missing `libdivsufsort` dependency
* Fix "unused parameter" warning.
|
|
* Update go and closure bazel rules
* Follow the new bazel go rules guide
* Swap go & closure rules initialization
* Update bazel to 0.10.0 in appveyor build
|
|
code length (#635)
According to the format specification regarding complex prefix codes:
> If there are at least two non-zero code lengths, any trailing zero
> code lengths are omitted, i.e., the last code length in the
> sequence must be non-zero. In this case, the sum of (32 >> code
> length) over all the non-zero code lengths must equal to 32.
> If the lengths have been read for the entire code length alphabet
> and there was only one non-zero code length, then the prefix code
> has one symbol whose code has zero length.
The script does not handle a case where there is just 1 non-zero code
length where the sum rule doesn't apply, which causes a StopIteration
exception when it attempts to read past the list boundaries.
An example of such file is tests/testdata/mapsdatazrh.compressed. I made
sure this change doesn't break anything by processing all *.compressed
files from the testdata folder with no thrown exceptions.
|
|
* merge {dec|enc}/port.h into common/platform.h
* fix one-shot q=10 1-byte input compression
* fix some unprefixed definitions
* make hashers host-endianness-independent
* extract enc/params.h from enc/quality.h
* fix API documentation / typos
* improve `BrotliEncoderMaxCompressedSize`
|
|
|
|
Missed this in my previous tests. Sorry for that.
On BSDs, both bc and sed are part of the base operating system. For sed this results in an error as the check construct (--version) is a GNU-ism and only works for GNU sed, not for bsd sed.
Similarly, BSD sed does not take parameters after the filename(s) operated on. Moving `-i` to the front fixes that. `-r` is provided for GNU compat in BSD sed as an alias of `-E`. The `-i` option in BSD sed requires an extension to work in-place.
(thank you for picking up the nginx module too!)
|
|
* Use Clang-5.0
* Disable unholy ASAN leak detector (to unbreak build)
* Reduce build matrix and use faster env, where compiler version is not important
* Add autotools build to Travis matrix
|
|
|
|
When using autotools to build the binary and libraries, the resulting libraries don't link `brotlicommon` or `m`.
This was detected when building cURL 7.57.0 which has now has brotli support. During configure it was failing
```
checking run-time libs availability... failed
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lbrotlidec -lz -L/usr/local/lib
```
inspection of config.log showed missing symbols from libbrotlicommon as the cause.
This patch results in the encryption and decryption libs to be properly linked against libbrotlicommon and libm.
See also https://bugs.freebsd.org/223966
|
|
* add autotools build
* separate semantic and ABI version
* extract sources.lst (used by CMake and Automake)
* share pkgconfig templates (used by CMake and Automake)
* decoder: always set `total_out`
* encoder: fix `BROTLI_ENSURE_CAPACITY` macro (no-op after preprocessor)
* decoder/encoder: refine `free_func` contract
|
|
|
|
|
|
|
|
* remove `const` on `BrotliDictionary` members
* extend `ZofliNode` distance range to 128MiB
* add missing `port.h` include to `quality.h`
* fix typo in encoder API-doc
* regenerate `decode.min.js`
|
