Skip to content

Commit

Permalink
Fixed, and double checked, compression calculations
Browse files Browse the repository at this point in the history
  • Loading branch information
@jzharris
jzharris committed Mar 23, 2019
1 parent ef87632 commit 679c5a5
Showing 1 changed file with 91 additions and 32 deletions.
123 changes: 91 additions & 32 deletions main/results/Compression_results.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,43 +7,77 @@
Huffman encoding network...
>>>
>>> encoded a total of 83 layers, and 3237726 weights
>>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created
>>> a total of 674.22 KB are required to store the encoded variables (8.12 KB per layer on average)
>>> original number of bits needed: 12950.90 KB
>>> new number of bits needed: 713.65 KB
>>> compression ratio: 18.1474
>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created
>>>
>>> quantized values: 84.99 KB
>>> the quantized code representation: + 3237.73 KB
>>> bits needed to store quantized network: = 3322.72 KB
>>>
>>> bits in codebook: 39.44 KB
>>> the huff codes representation: + 674.22 KB
>>> bits needed to store huff network: = 713.65 KB
>>>
>>> compression ratio: 4.6559
>>>
```

### Lempel-Ziv-Welch encoding
```
LZ encoding network...
>>>
>>> encoded a total of 83 layers, and 3237726 weights
>>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created
>>> a total of 652.58 KB are required to store the encoded variables (7.86 KB per layer on average)
>>> original number of bits needed: 12950.90 KB
>>> new number of bits needed: 692.01 KB
>>> compression ratio: 18.7149
>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created
>>>
>>> quantized values: 84.99 KB
>>> the quantized code representation: + 3237.73 KB
>>> bits needed to store quantized network: = 3322.72 KB
>>>
>>> bits in codebook: 39.44 KB
>>> the lziv codes representation: + 652.58 KB
>>> bits needed to store lziv network: = 692.01 KB
>>>
>>> compression ratio: 4.8015
>>>
```

### Huffman encoding on LZ codes
```
LZ encoding network...
>>>
>>> encoded a total of 83 layers, and 3237726 weights
>>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created
>>> a total of 652.58 KB are required to store the encoded variables (7.86 KB per layer on average)
>>> original number of bits needed: 12950.90 KB
>>> new number of bits needed: 692.01 KB
>>> compression ratio: 18.7149
>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created
>>>
>>> quantized values: 84.99 KB
>>> the quantized code representation: + 3237.73 KB
>>> bits needed to store quantized network: = 3322.72 KB
>>>
>>> bits in codebook: 39.44 KB
>>> the lziv codes representation: + 652.58 KB
>>> bits needed to store lziv network: = 692.01 KB
>>>
>>> compression ratio: 4.8015
>>>
Huffman encoding LZ codes...
>>>
>>> encoded a total of 83 LZ codebooks, and 347652 LZ codes
>>> a total of 83 codebooks containing 137685 uint32 numbers (550.74 KB) was created
>>> a total of 539.23 KB are required to store the encoded variables (6.50 KB per layer on average)
>>> original number of bits needed to store LZ codes: 652.58 KB
>>> new number of bits needed: 1089.97 KB
>>> compression ratio: 0.5987
>>> encoded a total of 83 lziv codebooks, and 347652 lziv codes
>>> a total of 83 codebooks containing 137685 values (550.74 KB) was created
>>>
>>> bits in lziv codebook: 39.44 KB
>>> the lziv codes representation: + 652.58 KB
>>> bits needed to store lziv network: = 692.01 KB
>>>
>>> bits in huff codebook: 550.74 KB
>>> the huff codes representation: + 539.23 KB
>>> bits needed to store huff network: = 1089.97 KB
>>>
>>> compression ratio: 0.6349
>>>
Determining overall compression ratio...
>>>
>>> bits needed to store quantized network: 3322.72 KB
>>> bits needed to store huff network: 1089.97 KB
>>> compression ratio: 3.0485
>>>
```
The compression ratio is very small because the LZ encoded data is very sparse.

Expand All @@ -52,19 +86,39 @@ The compression ratio is very small because the LZ encoded data is very sparse.
Huffman encoding network...
>>>
>>> encoded a total of 83 layers, and 3237726 weights
>>> a total of 83 codebooks containing 9859 32-bit floating point numbers (39.44 KB) was created
>>> a total of 674.22 KB are required to store the encoded variables (8.12 KB per layer on average)
>>> original number of bits needed: 12950.90 KB
>>> new number of bits needed: 713.65 KB
>>> compression ratio: 18.1474
>>> a total of 83 codebooks containing 9859 values (39.44 KB) was created
>>>
>>> quantized values: 84.99 KB
>>> the quantized code representation: + 3237.73 KB
>>> bits needed to store quantized network: = 3322.72 KB
>>>
>>> bits in codebook: 39.44 KB
>>> the huff codes representation: + 674.22 KB
>>> bits needed to store huff network: = 713.65 KB
>>>
>>> compression ratio: 4.6559
>>>
LZ encoding Huffman codes...
>>>
>>> encoded a total of 83 Huffman codebooks, and 3237726 Huffman codes
>>> a total of 83 codebooks containing 8851 uint32 numbers (35.40 KB) was created
>>> a total of 580.86 KB are required to store the encoded variables (7.00 KB per layer on average)
>>> original number of bits needed to store LZ codes: 6287.88 KB
>>> new number of bits needed: 616.27 KB
>>> compression ratio: 10.2031
>>> encoded a total of 83 huff codebooks, and 3237726 huff codes
>>> a total of 83 codebooks containing 8851 values (35.40 KB) was created
>>>
>>> bits in huff codebook: 39.44 KB
>>> the huff codes representation: + 674.22 KB
>>> bits needed to store huff network: = 713.65 KB
>>>
>>> bits in lziv codebook: 35.40 KB
>>> the lziv codes representation: + 589.05 KB
>>> bits needed to store lziv network: = 624.46 KB
>>>
>>> compression ratio: 1.1428
>>>
Determining overall compression ratio...
>>>
>>> bits needed to store quantized network: 3322.72 KB
>>> bits needed to store lziv network: 624.46 KB
>>> compression ratio: 5.3210
>>>
```

## CRNN compression
Expand All @@ -82,4 +136,9 @@ LZ encoding Huffman codes...
### Huffman encoding on LZ codes
```
```

### LZ encoding on Huffman codes
```
```

0 comments on commit 679c5a5

Please sign in to comment.