Swap murmurhash in as primary hash function.

[ctb]

Starting from #1432 (simplification of hash function code), swap in murmurhash as the default hash function for khmer.

This will support k > 32. cc #1426

The tests that are broken are either checks on the exact hash values, OR are sensitive to collisions (a lot of the occupancy numbers are like this), OR are broken for unknown reasons - I'm not sure why the graph alignment code is broken, in particular. Still, we're down to 15 broken tests which is pretty good...

[betatim]

Does it make sense to investigate a general purpose string/byte hashing algorithm? MH is pretty speedy as far as hash functions go but it is still ~100x slower than master. Considering that khmer doesn't need a hash function that can handle all 256 values of a byte but only maybe 10, does it make sense to brew a custom hash function? It seems like you should be able to capitalise on this limitation. thinkingoutloud

[ctb]

Good question ;). Poor man's solution could be this:

• hash DNA into two bit form;
• use the bytes emerging from that two bit form as the input to a
  normal hash function.

Should be much faster, too!

I do think we should avoid developing our own hash function because it's
an area with a lot of already developed expertise.

[betatim]

I do think we should avoid developing our own hash function because it's
an area with a lot of already developed expertise.

... and it probably is a bit like crypto, you think it will be a one afternoon adventure and two years later you are still developing and discovering what others learnt many years ago.

[betatim]

I was thinking that the way you'd design your hash function would be different if you only have 10 values or if you have 256. I'd be surprised if MH runs faster on byte arrays that only contain 10 different values than on one that contains 256 different values.

[ctb]

On Mon, Sep 05, 2016 at 08:09:51AM -0700, Tim Head wrote:

I was thinking that the way you'd design your hash function would be different if you only have 10 values or if you have 256. I'd be surprised if MH runs faster on byte arrays that only contain 10 different values than on one that contains 256 different values.

Right - but if you hash four DNA characters at a time in an 8-bit string
you're effectively hashing the entire range of 256 characters.

[ctb]

This was a demonstration PR - closing now that #1450 and #1444 are under way.