Feat: Log progress/status to Prime Intellect PRI-442

[manveerxyz]

• Setup a status/progress logger and integrate it so that we can visualize training progress on Prime Intellect
• Needed to log node information as well as the data we push to w&b

TODO:
[ ] Treat this as a library by adding a flag to opt-out of uploading information to Prime Intellect

[samsja]

this looks great, I have couples of comments.

I am not too comfortable in adding a prime specific function directly to the hivemind_diloco.py code because we expect people to be able to use open diloco independently from us.

Couples of alternatives:

• only having pi metrics specific code in the training code train_fsdp.py
• switch from a push to a pull strategy where the worker just exposes metrics and we run some core periodically on the pi backend to get the metrics. maybe using something like https://github.com/prometheus/client_python
• make metric pi agnostic and expose it as a feature
• only keep it for in an internal version

We should avoid global call as much as possible as well:

• Maximum one function should do a global call something like get_prime_logger and all of the other just call this one
• or we just skip all global call and just pass the object as parameters