MFS Republisher

[schomatis]

I'm opening this issue to learn more about the republishing system on the MFS (which I currently understand nothing about).

I'm preemptively tagging @Stebalien to see if he can help me with some basic pointers about its main design goals, I'm assuming this system allows to recreate a different set of nodes that would represent any modifications to a file in the MFS.

[schomatis]

I'm especially interested in understanding what is going on in this function,

https://github.com/ipfs/go-ipfs/blob/7853e53860805e08a212d78c4baa5d59bff99ba8/mfs/system.go#L257-L291

[schomatis]

The core publish function is over here,

https://github.com/ipfs/go-ipfs/blob/7853e53860805e08a212d78c4baa5d59bff99ba8/core/core.go#L735-L738

It's not trivial to find and it seems very important in order to understand what is the publisher doing because there are many mentions of the publish verb but I'm not seeing any definition of exactly what does publish mean in this context.

[schomatis]

/cc @Stebalien

[schomatis]

/cc @Stebalien

[Stebalien]

I'm especially interested in understanding what is going on in this function,

That function is calling "pubfunc" whenever the hash at the MFS root changes. Currently, I believe we only use this to persist to disk. However, IIRC, we plan on using this to publish to IPNS (eventually).

The complexity comes from the fact that publishing an IPNS record is slow so we need to batch updates. Algorithm:

1. When we receive the first update after publishing, we set a "long" timeout.
2. When we receive any update, we reset the short timeout.
3. If either the short timeout or the long timeout elapses, we call publish(latestHash).

These long timeout ensures that we delay publishing by at most TimeoutLong. The short timeout allows us to publish sooner if it looks like there are no more updates coming down the pipe.

---

The core publish function is over here,

Publish is fully pluggable. That republisher records the MFS root to disk. We also have some cases where we don't publish anything (e.g., when using a temporary MFS for adding/modifying directory trees).

[schomatis]

Documented in ipfs/go-mfs#36.