Exploring reversible hashing for k>32

[betatim]

Motivated by #1441 we investigated different ways to store hash values with more than 64bits. This would allow you to have reversible hashing for k>32 k-mers.

Trial PRs:

• A type to store large hash values (>64bit) #1444
• Beyond 64bit hash values #1455
• Beyond 64bit hash values - alternate implementation #1488

Explored adding a new type that uses std::bitset or an array of bytes, or an array of different numerical types. One lesson learnt is that bitset does exactly that, packs the bits into integers in an array. Except it is smart about picking the size of the integer and can use vector instructions etc.

The khmer hashfunction uses shifts and ORs so it is hard to see how to speed things up.

The overall conclusion after some benchmarking was that this slowed down the hashing by >10x even for small k. Therefore we ditched it. Now using irreversible hashing for k>32.

[ctb]

cc #1426 #27 #60. I think #624 can be closed, too.

[ctb]

Irreversible hashing initial implementation: #1511

[betatim]

Another issue with thoughts in it #1490

[ctb]

Closing now that #1511 has been merged.