The Memex #149
Comments
|
i.e. torrent-ifying arbitrary pieces of content? Where "arbitrary" might be informed by the Open Annotations specification? (Ignore the technical incorrectness of the claim and consider only the analogy in its accessibility to the layperson) |
|
@mekarpeles Anything that can be added to IPFS, yes :) |
Would something like https://onename.com work for identity? |
|
It might be good to be able to create "associative trails" as per Bush's original vision too. Linking items together to form narratives. Bush's trails seem to be linear-only, but I've been thinking about ways to create non-linear narratives using IPLD: |
|
Bush's "associative trails" are one of my ultimate goals (in life) to support. Books are essentially associative trails... but they are too rigid in their creation and presentation to benefit from the rapid-paced, technology-augmented evolutionary / recapitulation processes "Wikipedia" style crowd-sourced platforms enable. One area where this could be applicable is in my embarrassingly idea-stage concept for http://dissertate.org -- a wiki for quickly, collaboratively publishing directed sequences of themed academic papers towards a goal. Think coursera's "tracks" but imagine that these tracks can branch arbitrarily. Thus (Randy Pausch style "head-fake") we're really talking about step towards creating a dependency knowledge graph. The goal is saving researchers time by being able to search sequences of papers (the result of tens of hours of an expert reading papers, filtering the relevant and high quality works, and pruning/publishing a "shortest path" to knowledge. e.g. if I am reading one of Andrew Ng's papers on CNNs for voice recognition, that same paper may occur in a sequence of "how to tune a CNNs". arbital.com, metacademy, khan academy, and other services have similarly acknlowledge this philosophy in their design and interface choices, but it's challenging to build momentum on such fronts, as (even if we can get past the technical requirements), it requires (the right) curators / scholars to participate + seed value and expertise. And presently there isn't a whole lot of incentive for this -- academics have the option to get paid and receive sole recognition for monopolizing the paper format and silo'ing + publishing their ideas and insights. Such a format also allows them to remain more focused and avoid the distracting element of others changing/interrupting their work. This suggests that a successful mode might need to be more like git in its interface, to enable collaborators to also maintain personal / separate branches / trains of thought... without preventing pull only-access / cross-pollination of ideas. Needless to say, now, in this theoretical landscape of adoption, we have a tangled mess of tangential features complicating adoption. My thought on such things is, it's less about the technology and mechanism (pinning, etc) and more about the content: the ability to focus on a valuable domain and get buy-in from scholarly experts who can seed the right value. e.g. the phenomenon where 50% of all edits on Wikipedia ( @aeschylus are attempting a project (books.archivelab.org for greco-roman classics) to start with (a) content + data, (b) a community of experts, and (c) minimal interfaces (with minimal technological investment) to enable contributions. The emphasis is on exploring the best ways to organize ~100 greek classics (which version/translation of a book). Once we have enough overlapping sequence of books, we can enable some sort of "Choose Your Own Adventure" (For those who remember the Goosebumps series where you could switch between story lines/trees: https://images-na.ssl-images-amazon.com/images/I/61T0AHDQ33L._AC_UL320_SR218,320_.jpg). The long term goal is to enable people to create directed sequences not only out of books, but out of parts of books (cc: @edsilv) using the Internet Archive's collection of millions of IIIF backed books. I'd look forward to experimenting with this "pin" feature on such a collection, given the community, schema, and data is ready for it. |
|
@mekarpeles It sounds like we have similar ideas but are coming at the problem from different directions. I think if there were a simple standardised way to create decentralised n-linked lists that refer to one or more IPFS objects with additional metadata we can achieve these linear and non-linear "trails". I'd really appreciate it if you could have a read of my 'IDF' (bad name I know...) proposal and let me know your thoughts. I wrote it after going to the i-docs conference in Bristol this year. The idea of interoperable narratives occurred to me while watching a panel discussion between two i-doc authoring tool vendors with different data models. "Where's the IIIF for interactive documentaries?" The documentary makers I spoke to seemed to like the idea, but if we can make it as generic as possible I'm sure it would have a wide variety of applications. |
|
This thread may also be of interest to @trentmc. We chatted briefly at the DWeb Summit about this 'IDF' idea and how the author properties could make use of the ongoing work of the COALA IP group: http://coala.global/working-groups/ |
|
I knew I'd forgotten something: https://read-write-web.org/ |
As i read every comment here, my reactions are: |
|
I think we have all been greatly inspired and want to build the various pieces of the memex. I (naturally) think IPFS is perfectly suited for this, particularly for establishing the public record. But it's going to take a lot more too-- lots of UX tooling / applications on top. we can do it! Btw, i'm reminded of https://WebRecorder.io |
|
Randomly dropping in out of nowhere to say I love that this is getting discussed |
Thanks for the ping, Ed -- definitely interested :) Correct, you might be able to leverage the COALA IP protocol here to save much effort. About the protocol: it starts with LCC, a protocol designed to generalize rights expression & attribution across different media verticals such as DDEX (widely used in music) or PLUS (widely used in photos). COALA IP protocol is an opinionated implementation of LCC: it uses JSON-LD & IPLD for base data structure, schema.org for identifier naming, and Interledger protocol to connect across networks (IPFS, IPDB, more). It's been an effort by the COALA IP working group, with much input from the broader community. I've attached a couple slide decks that provide an intro. Short = 5 min intro, long = all the details. Please let me know if you have more Q's. |
|
@ascribe0
Would be a sensible mantra to adopt for a Memex project too I think. I've been looking at the Web Annotation Data Model. It looks like we could potentially model bookmarks and trails with this. The (We'd need to adhere to using IRIs as per the spec). Combining JSON-LD with IPFS has some snags: https://docs.google.com/document/d/1LsQCB2aznlFduE549AfIYR6Tbm95t2VmrjEiFnOhqVM/edit?usp=sharing I noticed in your example JSON objects you aren't using an Maybe we could also use a https://www.w3.org/TR/2016/CR-annotation-model-20160705/#rights-information |
|
I highly recommend reading Ed's docs on the OA specs. Pretty much every memex feature can be modelled in a reasonable (meaning non-semantic-web-fascist) way by OA (except for dereferencable IRI/URIs). |
|
Now that JSIPFS and soon IPLD are minted, what is a reasonable first step in the right direction? Ultimately I could see this replacing my web browser. I would much prefer to live life without a web browser. |
I like that you've used the COALA IP Party model here.
Yes. We've not been using IRIs in the COALA IP spec, but multiaddr (maybe @jbenet wants to give feedback on that). Off-topic: Did anyone ever experiment with signed IPLD objects. This would be something we'd need here for the implementation of the COALA IP spec.
Don't know what you mean. The id is there, you can simply not see it because your head is not able to hash the payload 😆. A machine can easily though :)
This won't work. It's close to impossible to find a hash of an object that contains that exact hash. |
Yep, this has been a struggle for me to get my head around too. In that Google doc above:
I'm reminded of this quote:
After a recent IIIF conference where mediachain was demoed, a colleague commented to me that IPFS is "forking the web". I think this is a source of confusion for many right now - myself included 😄
Your COALA IP json examples in the PDF don't have an Would it make sense to try remodelling the WADM in IPLD? |
@diasdavid Any thoughts? |
That's exactly my point. The
I think you've hit the nail on the head here. IPFS is forking the web.
I never looked into WADM, but if it has the assumption that data can for example:
then it's very likely that you'll have to remodel. Maybe some of the leading IPFS visionaries can comment on my thoughts here. I think what I'm describing is definitely a problem that needs to be addressed structurally by IPFS, since I think it holds back implementations from third-parties. |
@edsilv It would be useful to discuss the problems highlighted. This statement is useless without content. we are explicitly NOT forking the web. The entire point is NOT to fork the web, to rebase the existing web and enable everything (or as much as we can) to work as is. You see current webapps working on IPFS without modification. The point is to keep ONE web.
@TimDaub What do you mean by this? Non-compliant with what standards? You can typically use whatever you have originally and just use IPFS for caching, redundancy, and distribution. But, if you want to take advantage of the nicer features (like dropping down to IPLD), then yeah you may have to adjust to that layer. This means that a user can continue to use their webapp today unmodified save for serving static content on IPFS. that's fine and works just great with stock web browsers. nothing modified. For more advanced features to be used, more advanced implementations may be needed (potentially modifications, yes). This is just like what happens in the web today with regular standards: they need to be implemented and deployed. You can use some of them as polyfill, and some of them you have to change things to take advantage of them. And we're doing that. In general, it's pretty frustrating to hear "they're forking the web" when the entire point is NOT to, and we're working very hard to make everything work even in stock browsers of today (js-ipfs). Once we have all that working very well, we'll submit patches to browsers to add support natively. One Web.
No, they dont. they can be imported as they are, using a data structure that models a version history on a graph. this means you can wrap your bidirectional graph into something like an op-log that "builds up" your graph. Same for mutations. You need a notion of a time axis to make the web archivable, and that's what hash-linking gives you. Think about what Git did for file systems. You can still model a filesystem exactly the same, and even use it. there's "mounted" versions of Git, that commit on every write and implement the fs syscalls giving you perfectly compliant filesystem. Like has been discussed in these repos many times, for RDF and other bidirectional graph structures can be trivially modeled on top of IPFS without modification. It needs a "middleware" data structure, or maybe just transformations, between the raw IPLD and the raw RDF. cc @nicola who is working on something like this. Think about how you would represent RDF graphs in git. You wouldnt necessarly use git links to replace the RDF links. and neither should you when adding raw, existing RDF to ipfs. |
Without having looked at the specifics, that sounds good to me. Btw, you can always "IRIfy" a multiaddr by adding |
First step to what-- the Memex? or The Memex using OA? |
|
@nicola this thread, and the conversations we had at DWS and MIT make me think we need to accelerate work on "model your stock RDF on IPLD". I saw you're already on it (starting threads). Need any help? |
|
One general note for everyone: IPLD and IPFS both give you the ability to implement whatever you want on top of them, including exactly the same files and data streams you have today. One example is unixfs which IPFS uses to represent traditional POSIX files. It just depends how deep you want to go with the hash-linking, whether you want to use IPFS as just a transport of bytes (totally fine) or re-model your data with distributed authenticated data structures in mind (harder but way more powerful for you). Getting more powers does not remove your old powers, just drop back up to the familiar POSIX file world if it's annoying to change anything. Many applications don't need more than authenticated file distribution. It's like HTTP actually. HTTP allowed you to move the same files FTP did (TeX, ps, whatever). But it ALSO had native support for more interesting, linked files: HTML documents. This is strictly an upgrade. IPFS does the same, you can use it as a dumb file store, only using it to distribute and authenticate regular POSIX files. Or you can remodel with IPLD and leverage more power. It's up to you. |
|
Oh and on Mutability, you can add mutable links in IPLD if you want. Your applications can parse them and use them, they just wont resolve natively by IPFS just yet.
|
|
It seems there's two main issues here: the data storage aspect ("pin it", synchronisation, sharing, etc), and the annotation/metadata/interlinking aspect. Is that right? If so, it might be helpful to split them into two (or more) separate issues, preferably with concrete goals. |
|
@davidar - that's my interpretation. I 100% understand the data storage benefits.
This sounds promising - something like a beginner's guide would certainly help me :-) It might also give me the means to mount a reasonable defense of the interlinking aspects of IPFS as an upgrade as opposed to fork of the web (as some may perceive it). |
|
Dropping a few links here: There is a lot happening in the IPLD space! (here is a complete summary: ipld/specs#13).
There is an effort to bring simple rdf graphs to IPLD: #152 (which what really means is to authenticate RDF graphs) If you really want to shape IPLD, I really want you in the conversation, I set up a call for next week (here more info ipfs/team-mgmt#124) and a rewrite of the IPLD spec (here more info: ipld/specs#12) |
|
@nicola Hi, just looking for a link for the IPLD call today. Reading your PM docs:
|
|
Here's a proposal for a method to create linear and nonlinear narrative trails using the Web Annotation Data Model: |
I've had a few discussions with people about using IPFS as a "personal library", but can't find much in the way of public discussion about it, hence this issue.
In particular, users would be provided with a ":pushpin: Pin It" button (a la pinterest) that would pin a document to a user's local ipfs node, along with providing some system for synchronising pins across all of a user's devices. There could also be shared collections that are synchronised between multiple users, etc.
The 📌 button could be embedded directly into webpages, be added to the browser toolbar, added as an entry to the contextual menus of file browsers, etc. Ideally it would also show the (estimated) number of ipfs nodes seeding the content, much like GitHub stars (but better, because in addition to showing support for content, you're also helping to host and distribute it ;).
We could also integrate it with other systems, eg. @hypothesis for annotations (ipfs-inactive/archives#34), @mediachain for media resolution, etc.
CC: @jbenet @edsilv @aeschylus @JesseWeinstein @mekarpeles
The text was updated successfully, but these errors were encountered: