From 84ea3279f261cf28d8edcc3d2fe0e157d89381e0 Mon Sep 17 00:00:00 2001 From: Nathan Moinvaziri Date: Thu, 8 Aug 2019 01:13:28 -0700 Subject: [PATCH] Readme and index restructuring (#346) * Added cmake build instructions, build options, install instructions, and repository contents to README.md. * Moved INDEX file content to README.md files. * Added configure instructions and options. * Added CodeFactor to build integration. --- INSTALL | 64 -------------------- README.md | 143 ++++++++++++++++++++++++++++++++++++++++++--- arch/x86/INDEX | 3 - arch/x86/README.md | 8 +++ test/INDEX | 14 ----- test/README.md | 12 ++++ 6 files changed, 154 insertions(+), 90 deletions(-) delete mode 100644 INSTALL delete mode 100644 arch/x86/INDEX create mode 100644 arch/x86/README.md delete mode 100644 test/INDEX create mode 100644 test/README.md diff --git a/INSTALL b/INSTALL deleted file mode 100644 index 7aa87a50d..000000000 --- a/INSTALL +++ /dev/null @@ -1,64 +0,0 @@ -Overview -======== - -There are several methods for compiling and installing zlib-ng, depending -on your favorite operating system and development toolkits. -This document will attempt to give a general overview of some of them. - -PS: We do not recommend running 'make install' unless you know what you - are doing, as this can override the system default zlib library, and - any wrong configuration or incompatability of zlib-ng can make the - whole system unusable. - -On linux distros, an alternative way to use zlib-ng instead of zlib -for specific programs exist, use LD_PRELOAD. -If the program is dynamically linked with zlib, then zlib-ng can take -its place without risking system-wide instability. Ex: -LD_PRELOAD=/opt/zlib-ng/libz.so.1.2.11.zlib-ng /usr/bin/program - - -Configure -========= - -Using the configure script is currently the main method of setting up the -makefiles and preparing for compilation. Configure will attempt to detect -the specifics of your system, and enable some of the relevant options for you. - -Configure accepts several command-line options, some of the most important -ones are detailed below. - ---zlib-compat - This enables options that will ensure that zlib-ng is compiled with all the - functions that a standard zlib library contains, you will need to use this - if you are going to be using zlib-ng as a drop-in replacement for zlib. - ---without-optimizations - This will disable zlib-ng specific optimizations (does not disable strategies). - ---without-new-strategies - This will disable specially optimized strategies, such as deflate_quick and - deflate_medium. - -Run configure like this: -./configure --zlib-compat - -Then you can compile using make: -make -make test - - -Cmake -===== - -Cmake is an alternative to configure, basically letting you do the same thing, -but with different tools and user interfaces. - -Start by initializing cmake: -cmake . - -Then you can start the configuration tui to set the wanted options -ccmake . - -You can now compile using make: -make -make test diff --git a/README.md b/README.md index b2be6e59f..7944f2e84 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,17 @@ -zlib-ng - zlib for the next generation systems +## zlib-ng +*zlib data compression library for the next generation systems* Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org) +|CI|Status| +|:-|-| +|Travis|[![build status](https://api.travis-ci.org/zlib-ng/zlib-ng.svg)](https://travis-ci.org/zlib-ng/zlib-ng/)| +|Buildkite|[![Build status](https://badge.buildkite.com/7bb1ef84356d3baee26202706cc053ee1de871c0c712b65d26.svg?branch=develop)](https://buildkite.com/circlestorm-productions/zlib-ng)| +|CodeFactor|[![CodeFactor](https://www.codefactor.io/repository/github/zlib-ng/zlib-ng/badge)](https://www.codefactor.io/repository/github/zlib-ng/zlib-ng)| -Fork Motivation and History + +Fork Motivation --------------------------- The motivation for this fork was due to seeing several 3rd party @@ -39,10 +46,95 @@ point in having those in this fork for various reasons. A lot of improvements have gone into zlib-ng since its start, and numerous people have contributed both small and big improvements, -or valuable testing. +or valuable testing. Please read LICENSE.md, it is very simple and very liberal. +Build +----- + +There are two ways to build zlib-ng: + +### Cmake + +To build zlib-ng using the cross-platform makefile generator cmake. + +``` +cmake . +cmake --build . --config Release +ctest --verbose -C Release +``` + +Alternatively, yuou can use the cmake configuration GUI tool ccmake: + +``` +ccmake . +``` + +### Configure + +To build zlib-ng using the bash configure script: + +``` +./configure +make +make test +``` + +Build Options +------------- + +| CMake | configure | Description | Default | +|:-------------------------|:-------------------------|:---------------------------------------------------------------------------------------------|----------------------------------| +| ZLIB_COMPAT | --zlib-compat | Compile with zlib compatible API | OFF | +| ZLIB_ENABLE_TESTS | | Build test binaries | ON | +| WITH_GZFILEOP | --with-gzfileops | Compile with support for gzFile related functions | OFF | +| WITH_MSAN | --with-msan | Build with memory sanitizer | OFF | +| WITH_OPTIM | --without-optimizations | Build with optimisations | ON | +| WITH_NEW_STRATEGIES | --without-new-strategies | Use new strategies | ON | +| WITH_NATIVE_INSTRUCTIONS | | Instruct the compiler to use the full instruction set on this host (gcc/clang -march=native) | OFF | +| | --force-sse2 | Assume SSE2 instructions are always available | DISABLED (x86), ENABLED (x86_64) | +| WITH_ACLE | --without-acle | Build with ACLE CRC | ON | +| WITH_NEON | --without-neon | Build with NEON intrinsics | ON | +| WITH_DFLTCC_DEFLATE | --with-dfltcc-deflate | Use DEFLATE COMPRESSION CALL instruction for compression on IBM Z | OFF | +| WITH_DFLTCC_INFLATE | --with-dfltcc-inflate | Use DEFLATE COMPRESSION CALL instruction for decompression on IBM Z | OFF | +| WITH_SANITIZERS | --with-sanitizers | Build with address sanitizer and all supported sanitizers other than memory sanitizer | OFF | +| WITH_FUZZERS | --with-fuzzers | Build test/fuzz | OFF | + +Install +------- + +WARNING: We do not recommend manually installing unless you really +know what you are doing, because this can potentially override the system +default zlib library, and any incompatability or wrong configuration of +zlib-ng can make the whole system unusable, requiring recovery or reinstall. +If you still want a manual install, we recommend using the /opt/ path prefix. + +For Linux distros, an alternative way to use zlib-ng (if compiled in +zlib-compat mode) instead of zlib, is through the use of the +_LD_PRELOAD_ environment variable. If the program is dynamically linked +with zlib, then zlib-ng will temporarily be used instead by the program, +without risking system-wide instability. + +``` +LD_PRELOAD=/opt/zlib-ng/libz.so.1.2.11.zlib-ng /usr/bin/program +``` + +### Cmake + +To install zlib-ng system-wide using cmake: + +``` +cmake --build . --target install +``` + +### Configure + +To install zlib-ng system-wide using the configure script: + +``` +make install +``` Contributing ------------ @@ -73,9 +165,42 @@ The deflate and zlib specifications were written by L. Peter Deutsch. zlib was originally created by Jean-loup Gailly (compression) and Mark Adler (decompression). - -Build Status ------------- - -Travis CI: [![build status](https://api.travis-ci.org/zlib-ng/zlib-ng.svg)](https://travis-ci.org/zlib-ng/zlib-ng/) -Buildkite: [![Build status](https://badge.buildkite.com/7bb1ef84356d3baee26202706cc053ee1de871c0c712b65d26.svg?branch=develop)](https://buildkite.com/circlestorm-productions/zlib-ng) +Contents +-------- + +| Name | Description | +|:-----------------|:---------------------------------------------------------------| +| arch/ | Architecture-specific code | +| doc/ | Documentation for formats and algorithms | +| test/example.c | Zlib usages examples for build testing | +| test/minigzip.c | Minimal gzip-like functionality for build testing | +| test/infcover.c | Inflate code coverage for build testing | +| win32/ | Shared library version resources for Windows | +| CMakeLists.txt | Cmake build script | +| configure | Bash configure/build script | +| adler32.c | Compute the Adler-32 checksum of a data stream | +| compress.c | Compress a memory buffer | +| deflate.* | Compress data using the deflate algorithm | +| deflate_fast.c | Compress data using the deflate algorithm with fast strategy | +| deflate_medium.c | Compress data using the deflate algorithm with medium stragety | +| deflate_slow.c | Compress data using the deflate algorithm with slow strategy | +| functable.* | Struct containing function pointers to optimized functions | +| gzclose.c | Close gzip files | +| gzguts.h | Internal definitions for gzip operations | +| gzlib.c | Functions common to reading and writing gzip files | +| gzread.c | Read gzip files | +| gzwrite.c | Write gzip files | +| infback.* | Inflate using a callback interface | +| inflate.* | Decompress data | +| inffast.* | Decompress data with speed optimizations | +| inffixed.h | Table for decoding fixed codes | +| inftrees.h | Generate Huffman trees for efficient decoding | +| memcopy.h | Inline functions to copy small data chunks | +| trees.* | Output deflated data using Huffman coding | +| uncompr.c | Decompress a memory buffer | +| zconf.h.cmakein | zconf.h template for cmake | +| zendian.h | BYTE_ORDER for endian tests | +| zlib.3 | Man page for zlib | +| zlib.3.pdf | Man page in PDF format | +| zlib.map | Linux symbol information | +| zlib.pc.in | Pkg-config template | diff --git a/arch/x86/INDEX b/arch/x86/INDEX deleted file mode 100644 index 9ee3802a2..000000000 --- a/arch/x86/INDEX +++ /dev/null @@ -1,3 +0,0 @@ -fill_window_sse.c SSE2 optimized fill_window -deflate_quick.c SSE4 optimized deflate strategy for use as level 1 -crc_folding.c SSE4 + PCLMULQDQ optimized CRC folding implementation diff --git a/arch/x86/README.md b/arch/x86/README.md new file mode 100644 index 000000000..6d23945ff --- /dev/null +++ b/arch/x86/README.md @@ -0,0 +1,8 @@ +Contents +-------- + +|Name|Description| +|:-|:-| +|fill_window_sse.c|SSE2 optimized fill_window| +|deflate_quick.c|SSE4 optimized deflate strategy for use as level 1| +|crc_folding.c|SSE4 + PCLMULQDQ optimized CRC folding implementation| diff --git a/test/INDEX b/test/INDEX deleted file mode 100644 index 9b66ef6cb..000000000 --- a/test/INDEX +++ /dev/null @@ -1,14 +0,0 @@ -Makefile.in: template for Unix Makefile - -CVE-2003-0107.c: -CVE-2002-0059 : -CVE-2004-0797 : -CVE-2005-1849 : -CVE-2005-2096 : test cases for the relevant CVEs -GH-361 : test case for overlapping matches - https://github.com/zlib-ng/zlib-ng/issues/361 -GH-364 : test case for switching compression levels - https://github.com/zlib-ng/zlib-ng/issues/364 - -testCVEinputs.sh: script to run tests for CVEs where input data is supplied - diff --git a/test/README.md b/test/README.md new file mode 100644 index 000000000..2cc15ca95 --- /dev/null +++ b/test/README.md @@ -0,0 +1,12 @@ +Contents +-------- + +|Name|Description| +|-|-| +|[CVE-2003-0107.c](https://nvd.nist.gov/vuln/detail/CVE-2003-0107)|Buffer overflow in the gzprintf function, requires ZLIB_COMPAT| +|[CVE-2002-0059](https://nvd.nist.gov/vuln/detail/CVE-2002-0059)|inflateEnd to release memory more than once| +|[CVE-2004-0797](https://nvd.nist.gov/vuln/detail/CVE-2004-0797)|Error handling in inflate and inflateBack causes crash| +|[CVE-2005-1849](https://nvd.nist.gov/vuln/detail/CVE-2005-1849)|inftrees.h bug causes crash| +|[CVE-2005-2096](https://nvd.nist.gov/vuln/detail/CVE-2005-2096)|Buffer overflow when incomplete code description +|[GH-361](https://github.com/zlib-ng/zlib-ng/issues/361)|Test case for overlapping matches| +|[GH-364](https://github.com/zlib-ng/zlib-ng/issues/364)|Test case for switching compression levels|