From 679c5a5a8e330b892edf1ada34fe29a33fe9a975 Mon Sep 17 00:00:00 2001 From: Zach Harris Date: Fri, 22 Mar 2019 22:11:47 -0700 Subject: [PATCH] Fixed, and double checked, compression calculations --- main/results/Compression_results.md | 123 ++++++++++++++++++++-------- 1 file changed, 91 insertions(+), 32 deletions(-) diff --git a/main/results/Compression_results.md b/main/results/Compression_results.md index b24ea65..a0a490e 100644 --- a/main/results/Compression_results.md +++ b/main/results/Compression_results.md @@ -7,11 +7,18 @@ Huffman encoding network... >>> >>> encoded a total of 83 layers, and 3237726 weights ->>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created ->>> a total of 674.22 KB are required to store the encoded variables (8.12 KB per layer on average) ->>> original number of bits needed: 12950.90 KB ->>> new number of bits needed: 713.65 KB ->>> compression ratio: 18.1474 +>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created +>>> +>>> quantized values: 84.99 KB +>>> the quantized code representation: + 3237.73 KB +>>> bits needed to store quantized network: = 3322.72 KB +>>> +>>> bits in codebook: 39.44 KB +>>> the huff codes representation: + 674.22 KB +>>> bits needed to store huff network: = 713.65 KB +>>> +>>> compression ratio: 4.6559 +>>> ``` ### Lempel-Ziv-Welch encoding @@ -19,11 +26,18 @@ Huffman encoding network... LZ encoding network... >>> >>> encoded a total of 83 layers, and 3237726 weights ->>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created ->>> a total of 652.58 KB are required to store the encoded variables (7.86 KB per layer on average) ->>> original number of bits needed: 12950.90 KB ->>> new number of bits needed: 692.01 KB ->>> compression ratio: 18.7149 +>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created +>>> +>>> quantized values: 84.99 KB +>>> the quantized code representation: + 3237.73 KB +>>> bits needed to store quantized network: = 3322.72 KB +>>> +>>> bits in codebook: 39.44 KB +>>> the lziv codes representation: + 652.58 KB +>>> bits needed to store lziv network: = 692.01 KB +>>> +>>> compression ratio: 4.8015 +>>> ``` ### Huffman encoding on LZ codes @@ -31,19 +45,39 @@ LZ encoding network... LZ encoding network... >>> >>> encoded a total of 83 layers, and 3237726 weights ->>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created ->>> a total of 652.58 KB are required to store the encoded variables (7.86 KB per layer on average) ->>> original number of bits needed: 12950.90 KB ->>> new number of bits needed: 692.01 KB ->>> compression ratio: 18.7149 +>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created +>>> +>>> quantized values: 84.99 KB +>>> the quantized code representation: + 3237.73 KB +>>> bits needed to store quantized network: = 3322.72 KB +>>> +>>> bits in codebook: 39.44 KB +>>> the lziv codes representation: + 652.58 KB +>>> bits needed to store lziv network: = 692.01 KB +>>> +>>> compression ratio: 4.8015 +>>> Huffman encoding LZ codes... >>> ->>> encoded a total of 83 LZ codebooks, and 347652 LZ codes ->>> a total of 83 codebooks containing 137685 uint32 numbers (550.74 KB) was created ->>> a total of 539.23 KB are required to store the encoded variables (6.50 KB per layer on average) ->>> original number of bits needed to store LZ codes: 652.58 KB ->>> new number of bits needed: 1089.97 KB ->>> compression ratio: 0.5987 +>>> encoded a total of 83 lziv codebooks, and 347652 lziv codes +>>> a total of 83 codebooks containing 137685 values (550.74 KB) was created +>>> +>>> bits in lziv codebook: 39.44 KB +>>> the lziv codes representation: + 652.58 KB +>>> bits needed to store lziv network: = 692.01 KB +>>> +>>> bits in huff codebook: 550.74 KB +>>> the huff codes representation: + 539.23 KB +>>> bits needed to store huff network: = 1089.97 KB +>>> +>>> compression ratio: 0.6349 +>>> +Determining overall compression ratio... +>>> +>>> bits needed to store quantized network: 3322.72 KB +>>> bits needed to store huff network: 1089.97 KB +>>> compression ratio: 3.0485 +>>> ``` The compression ratio is very small because the LZ encoded data is very sparse. @@ -52,19 +86,39 @@ The compression ratio is very small because the LZ encoded data is very sparse. Huffman encoding network... >>> >>> encoded a total of 83 layers, and 3237726 weights ->>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created ->>> a total of 674.22 KB are required to store the encoded variables (8.12 KB per layer on average) ->>> original number of bits needed: 12950.90 KB ->>> new number of bits needed: 713.65 KB ->>> compression ratio: 18.1474 +>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created +>>> +>>> quantized values: 84.99 KB +>>> the quantized code representation: + 3237.73 KB +>>> bits needed to store quantized network: = 3322.72 KB +>>> +>>> bits in codebook: 39.44 KB +>>> the huff codes representation: + 674.22 KB +>>> bits needed to store huff network: = 713.65 KB +>>> +>>> compression ratio: 4.6559 +>>> LZ encoding Huffman codes... >>> ->>> encoded a total of 83 Huffman codebooks, and 3237726 Huffman codes ->>> a total of 83 codebooks containing 8851 uint32 numbers (35.40 KB) was created ->>> a total of 580.86 KB are required to store the encoded variables (7.00 KB per layer on average) ->>> original number of bits needed to store LZ codes: 6287.88 KB ->>> new number of bits needed: 616.27 KB ->>> compression ratio: 10.2031 +>>> encoded a total of 83 huff codebooks, and 3237726 huff codes +>>> a total of 83 codebooks containing 8851 values (35.40 KB) was created +>>> +>>> bits in huff codebook: 39.44 KB +>>> the huff codes representation: + 674.22 KB +>>> bits needed to store huff network: = 713.65 KB +>>> +>>> bits in lziv codebook: 35.40 KB +>>> the lziv codes representation: + 589.05 KB +>>> bits needed to store lziv network: = 624.46 KB +>>> +>>> compression ratio: 1.1428 +>>> +Determining overall compression ratio... +>>> +>>> bits needed to store quantized network: 3322.72 KB +>>> bits needed to store lziv network: 624.46 KB +>>> compression ratio: 5.3210 +>>> ``` ## CRNN compression @@ -82,4 +136,9 @@ LZ encoding Huffman codes... ### Huffman encoding on LZ codes ``` +``` + +### LZ encoding on Huffman codes +``` + ``` \ No newline at end of file