need 'Add log' for ipfs add

[whyrusleeping]

One big problem I have is that i will add something, and then forget the hash. I need a way to go and look up things that i've already added. This also has the added bonus of being able to go through and see which pins you want to remove when cleaning up your disk.

[wking]

On Wed, Jun 10, 2015 at 09:22:12AM -0700, Jeromy Johnson wrote:

I need a way to go and look up things that i've already added.

First pass at a UI:

$ ipfs log add

…

sorted by increasing addition timestamps. I'd make the timestamps for
“when we started the addition call” but avoid logging the entry until
the addition successfully completed. We might also want to log the
completion time? And I'm not sure how to balance races (especially if
we're sorting by start-time) in writing out the entries, since it's
nice to have append-only logs, but it's also nice to be able to bisect
down to since/before limits.

Potential arguments:

• --date (like ‘git log --date=…’)
• --format (like ‘git log --format=…)
• --since (only list entries with addition-timestamps >
  (≥?) that timestamp)
• --before (only list entries with addition-timestamps <
  that timestamp)
• --argument (only list entries with glob(7) matches for this
  pattern)
• --hash (only list entries that created this hash. Maybe
  globbed?)
• --pinned= (only list entries that are currently pinned (or
  aren't currently pinned, depending on the boolean value)

[jbenet]

• git reflog

[wking]

On Wed, Jun 10, 2015 at 12:32:57PM -0700, Juan Batiz-Benet wrote:

• git reflog

I love reflog, but it's about recording the history of mutable
pointers (e.g. what has my ‘HEAD’ pointer been pointing to?).
Archiving that sort of information would hook into ‘ipfs name publish
…’ (and maybe also name resolution that gets external IPNS records?),
but it's not very similar to add logging.

We likely do want it's expiration handling though.

[rht]

ui issue aside, is this to be put as an eventlog or util/log? From the codebase, util/log's output is os.Stderr, not a file. Since they are both going to be unified, I suppose the final output is file.
And I recall from the discussion that any process using log.EventBegin will output through http instead.

Or I wonder if the ui discussion drives the consensus of what should be logged to file (e.g. "HEAD's log" / any operation that mutates local dag, if there is a HEAD hash to summarize the local blocks at a given time).
So there are 3 outputs (os.Stderr, file, http).

[jbenet]

@rht this is a different kind of "log" than the eventlog and util/log. those are code/implementation logs, this is more a logical user-operation log. we'd want this as a structured log (probably an ipfs object)

any operation that mutates local dag, if there is a HEAD hash to summarize the local blocks at a given time).

yeah, exactly. we'd have things like:

• adding objects
• adding/removing pins
• gc-ing objects out

[leni536]

(I'm new to IPFS and unfimiliar with golang). Isn't adding is basically building the ipfs objects then pinning the root? I think a pin log would be more useful in the future, maybe logging if the pinning was initiated from an add, a get or an explicit pin command too. Then of course the add log could be simply implemented by filtering the pin log.

[rht]

@leni536 want to make this?

What about rht@0fed045
(ipfs get shouldn't have been there because it doesn't pin).
The remaining things to do

1. put the logger in the necessary places
2. make the filter command itself.
3. (optional) version the pin states

[leni536]

Yeah, seems easy enouigh, it will be my first interaction with go though. I think I will start with 1. and 2., I know nothing about how the pin state is stored or if it could be a subject to change later.

Some random thoughts:
I see efforts to push many things to the dag anyway. Maybe logs shouldn't be exception. What about having a log node definition, containing the date (maybe in unix epoch format), link to the added hash, link to its parent log entry, an enum for the method how it's added (pin or add) and enum for pin type (which would contain 'unpinned' too to mark unlinks)?

/random thoughts

I will be happy to implement a simple text log right now though.

[whyrusleeping]

I think we will want it to be a dag eventually, but I also think that doing so will require a lot more design work to figure out how to do that correctly. For now, if we just do a text log i think it would be good.