Skip to content

Commit b5f6d7e

Browse files
authored
Auto merge of #37298 - nnethercote:faster-deflate, r=alexcrichton
Use a faster `deflate` setting In #37086 we have considered various ideas for reducing the cost of LLVM bytecode compression. This PR implements the simplest of these: use a faster `deflate` setting. It's very simple and reduces the compression time by almost half while increasing the size of the resulting rlibs by only about 2%. I looked at using zstd, which might be able to halve the compression time again. But integrating zstd is beyond my Rust FFI integration abilities at the moment -- it consists of a few dozen C files, has a non-trivial build system, etc. I decided it was worth getting a big chunk of the possible improvement with minimum effort. The following table shows the before and after percentages of instructions executed during compression while doing debug builds of some of the rustc-benchmarks with a stage1 compiler. ``` html5ever-2016-08-25 1.4% -> 0.7% hyper.0.5.0 3.8% -> 2.4% inflate-0.1.0 1.0% -> 0.5% piston-image-0.10.3 2.9% -> 1.8% regex.0.1.30 3.4% -> 2.1% rust-encoding-0.3.0 4.8% -> 2.9% syntex-0.42.2 2.9% -> 1.8% syntex-0.42.2-incr-clean 14.2% -> 8.9% ``` The omitted ones spend 0% of their time in decompression. And here are actual timings: ``` futures-rs-test 4.110s vs 4.102s --> 1.002x faster (variance: 1.017x, 1.004x) helloworld 0.223s vs 0.226s --> 0.986x faster (variance: 1.012x, 1.022x) html5ever-2016- 4.218s vs 4.186s --> 1.008x faster (variance: 1.008x, 1.010x) hyper.0.5.0 4.746s vs 4.661s --> 1.018x faster (variance: 1.002x, 1.016x) inflate-0.1.0 4.194s vs 4.143s --> 1.012x faster (variance: 1.007x, 1.006x) issue-32062-equ 0.317s vs 0.316s --> 1.001x faster (variance: 1.013x, 1.005x) issue-32278-big 1.811s vs 1.825s --> 0.992x faster (variance: 1.014x, 1.006x) jld-day15-parse 1.412s vs 1.412s --> 1.001x faster (variance: 1.019x, 1.008x) piston-image-0. 11.058s vs 10.977s --> 1.007x faster (variance: 1.008x, 1.039x) reddit-stress 2.331s vs 2.342s --> 0.995x faster (variance: 1.019x, 1.006x) regex.0.1.30 2.294s vs 2.276s --> 1.008x faster (variance: 1.007x, 1.007x) rust-encoding-0 1.963s vs 1.924s --> 1.020x faster (variance: 1.009x, 1.006x) syntex-0.42.2 29.667s vs 29.391s --> 1.009x faster (variance: 1.002x, 1.023x) syntex-0.42.2-i 15.257s vs 14.148s --> 1.078x faster (variance: 1.018x, 1.008x) ``` r? @alexcrichton
2 parents 4879166 + 94771a1 commit b5f6d7e

File tree

1 file changed

+10
-25
lines changed

1 file changed

+10
-25
lines changed

src/libflate/lib.rs

+10-25
Original file line numberDiff line numberDiff line change
@@ -94,11 +94,14 @@ extern "C" {
9494
-> *mut c_void;
9595
}
9696

97-
const LZ_NORM: c_int = 0x80; // LZ with 128 probes, "normal"
98-
const TINFL_FLAG_PARSE_ZLIB_HEADER: c_int = 0x1; // parse zlib header and adler32 checksum
99-
const TDEFL_WRITE_ZLIB_HEADER: c_int = 0x01000; // write zlib header and adler32 checksum
97+
const LZ_FAST: c_int = 0x01; // LZ with 1 probe, "fast"
98+
const TDEFL_GREEDY_PARSING_FLAG: c_int = 0x04000; // fast greedy parsing instead of lazy parsing
10099

101-
fn deflate_bytes_internal(bytes: &[u8], flags: c_int) -> Bytes {
100+
/// Compress a buffer without writing any sort of header on the output. Fast
101+
/// compression is used because it is almost twice as fast as default
102+
/// compression and the compression ratio is only marginally worse.
103+
pub fn deflate_bytes(bytes: &[u8]) -> Bytes {
104+
let flags = LZ_FAST | TDEFL_GREEDY_PARSING_FLAG;
102105
unsafe {
103106
let mut outsz: size_t = 0;
104107
let res = tdefl_compress_mem_to_heap(bytes.as_ptr() as *const _,
@@ -113,17 +116,9 @@ fn deflate_bytes_internal(bytes: &[u8], flags: c_int) -> Bytes {
113116
}
114117
}
115118

116-
/// Compress a buffer, without writing any sort of header on the output.
117-
pub fn deflate_bytes(bytes: &[u8]) -> Bytes {
118-
deflate_bytes_internal(bytes, LZ_NORM)
119-
}
120-
121-
/// Compress a buffer, using a header that zlib can understand.
122-
pub fn deflate_bytes_zlib(bytes: &[u8]) -> Bytes {
123-
deflate_bytes_internal(bytes, LZ_NORM | TDEFL_WRITE_ZLIB_HEADER)
124-
}
125-
126-
fn inflate_bytes_internal(bytes: &[u8], flags: c_int) -> Result<Bytes, Error> {
119+
/// Decompress a buffer without parsing any sort of header on the input.
120+
pub fn inflate_bytes(bytes: &[u8]) -> Result<Bytes, Error> {
121+
let flags = 0;
127122
unsafe {
128123
let mut outsz: size_t = 0;
129124
let res = tinfl_decompress_mem_to_heap(bytes.as_ptr() as *const _,
@@ -141,16 +136,6 @@ fn inflate_bytes_internal(bytes: &[u8], flags: c_int) -> Result<Bytes, Error> {
141136
}
142137
}
143138

144-
/// Decompress a buffer, without parsing any sort of header on the input.
145-
pub fn inflate_bytes(bytes: &[u8]) -> Result<Bytes, Error> {
146-
inflate_bytes_internal(bytes, 0)
147-
}
148-
149-
/// Decompress a buffer that starts with a zlib header.
150-
pub fn inflate_bytes_zlib(bytes: &[u8]) -> Result<Bytes, Error> {
151-
inflate_bytes_internal(bytes, TINFL_FLAG_PARSE_ZLIB_HEADER)
152-
}
153-
154139
#[cfg(test)]
155140
mod tests {
156141
#![allow(deprecated)]

0 commit comments

Comments
 (0)