Use a fork of gguf-tools to make it work on Windows

[zcbenz]

Refs #1513.

https://github.com/antirez/gguf-tools is not being maintained, valid pull requests are left there months without getting a reply. Making gguf-tools work on Windows is not hard but with current state I can't get my work merged to upstream, so I think using a forked repo should be reasonable for now, and I can always try to merge things back once the upstream is alive.

My changes to gguf-tools are in https://github.com/frost-beta/gguf-tools/commits/main/, with a bonus side effect to fix antirez/gguf-tools#18, a MLX issue I was surprised to find out.

[awni]

Thanks.. I'm not sure how critical it is to keep GGUF support in MLX at this point.

It's not used much, doesn't fit well with the rest of the serialization API since it's a higher-level model format, and if the package we rely on for it is not maintained it might be better to deprecate it or make it more of a third party tool that can load MLX models from GGUF directly.

CC @angeloskath @barronalex what do you all think about deprecating GGUF support / moving it out of MLX core?

[angeloskath]

I think it is a good idea. It is probably hard to gauge how much it is used but it is definitely not well supported which is a good reason to deprecate and then remove from core.

[awni]

@zcbenz I appreciate the effort on this but I don't think we should merge it right now though. I'm leaning towards deprecating GGUF support in MLX core and moving the GGUF support as a "third party extension" in our GGUF LLM example. My recommendation is to disable GGUF support for windows until we decide on that. If you are unhappy with the C++ test failing we can disable them when GGUF support is inactive.

[zcbenz]

We can disable the tests on Windows if some day you decided to run CI for it, I'm good just closing this PR.