Links between entries

[oscargus]

There are some feature requests at SourceForge on being able to link between related entries. One obvious thing would be to link papers which are cited in another paper, but there could also be other reasons to mark entries as related. This could also be a foundation for other things, such as plotting graphs on the relation between papers, importing references and citation links from databases etc.

I thought a bit about it and the following is my draft idea:

• Introduce two new fields in the bibtex entry: references and related(?), which contains the keys for the entries referenced by the paper and keys for other papers which are related
• Add a new tab in the entry editor with three(?) lists: references, cited by, related
• Cited by is automatically filled based on the other entries references
• Entries can be dragged-and-dropped to all three fields. If an entry in dropped in Cited by, that entry is updated.

Issues:

• Probably one would need to keep the keys in the related field of every entry. This potentially introduce some redundancy, which is obvious in the case of two related items, but for three or more it is not obvious how to resolve it. Any ideas?
• It seems to make sense that once an entry is dropped in the related list, that entry should also be updated with related. If multiple entries are dropped at the same time, they should all be cross-related.
• Are there characters which can not be part of a key? (And are therefore suitable as separator.)

Current state of discussion

fields

supported

• cites - comma separated list of BibTeX keys which are cited by this entry
• crossRef - single entry which is cross referenced.
• related - comma separated list of BibTeX keys which are in some kind related to this entry. The type of all relations can be specified by a single relatedtype (see Multiple related entries having different relationship types plk/biblatex#475 (comment)). Note: Biblatex prints this information if related is active at the biblatex package.

not supported

• citedBy - this is the opposite of cites. That field has to be used.
• relations - this would introduce a complicated field similar to our save actions. A simple key/value is enough
• references - stores all references in plain text (PRVV plugin). Thus, we do not use it.

[koppor]

First issue: Do you mean that the keys have to be renamed if the key of the referenced entry changes? - See JabRef/cloudref#6 for a related discussion.

Second issue: 👍

Third issue: I found net.sf.jabref.util.Util.checkLegalKey(String) with all the characters allowed. Unfortunately, there is no reference provided. I would use , or space as separator as the former one is also used at latex's \cite command.

[koppor]

I think, the JabRef PRRV Plugin should be checked. They seem to parse a list of references automatically and add the pointers accordingly. OK, related won't be checked. - How are keywords, groups, and related related? Done. See #14 (comment).

[oscargus]

I was reminded of this.

First issue: yes, if the key is renamed, then all fields referring to that key should be updated. However, what I actually meant was the following.

Assume three entries: a, b, and c.

Say that b is related to a, i.e., related = {a} in b. Now, should this also mean that related = {b} in a (or should this be implicit in JabRef without writing it to the .bib-file)?

Now, if b is related to c, i.e., related = {a,c} in b, should a and c be related? Implicitly or explicitly?

[tobiasdiez]

Maybe JabRef could internally store these related references by there internal ID (and not the bibtex key). Upon serialization or display, it fetches the current bibtex key of the item and stores/display it.

I would say that related is bi-directional....

[oscargus]

Very good idea how to handle key changes!

Yes, bidirectional probably makes sense for related, but should one automatically relate entries which have a common related entry? I do not think so, although I can realize that there will be feature requests related to this eventually...

So, for a basic implementation the following seems to make sense:

• directly after loading the .bib-file the _related-field (or whatever the internal name would be) is populated with a Set containing the internal IDs.
• when relating an entry to another, it is also related in the other way around
• when serializing/displaying, the bibtex keys are returned
• another field references(?) includes, in the .bib-file, the current bibtex keys, internally they are handled similar to related
• internally, there is also a referencedby(?) field which is never serialized
• adding an entry to referencedby in the gui actually adds it to references of the other entry

[koppor]

Maybe, we need different semantics for the links? I need extendedversion to clearly state which journal paper is an extension of which conference paper. This is different from cited / citedBy or related. Furthermore, presentationFor would also be good to link a presentation for a paper. Maybe, the latter should go to the URL field somehow? Should we support multiple URLs as we support multiple files? (Maybe, I move that to a separate issue)

Should we go the Semantic Web way and introduce a (lightweight) ontology for different kinds of references? We could serialize the relations using BibTeX fields, but would also be able to serialize in XML or turtle.

[tobiasdiez]

Maybe Section 3.4 of the biblatex documentation might be helpful.

[oscargus]

@tobiasdiez : At least as evidence that related is a really bad choice for naming the field that I talk about. :-)

As i understand it, only one entry (or at least one type of relation, possibly for more than one entry) can be specified. I think we should support this way, e.g., when visualizing, but we would need something more flexible and general.

@koppor : presentationFor I would solve by adding the PDF/PPT/... to the entry (is it possible to have more than one PDF?). Does it really make sense to have a .bib-entry for a presentation at the first place?

[koppor]

I would really go for the related field. We should try to add support for different relation types in biblatex. Currently (according to Section 4.5.1 of biblatex doc and biblatex-examples.tex), there is no support for multiple relationtypes. We should ask the biblatex makers whether support for multiple realtiontypes would be possible.

First idea: If there are more than one keys written in the related field, either one relationtype has to be given or as many as keys are given in the related field.

[oscargus]

I disagree. The Biblatex related field is clearly for actual relations (where "extended version of" would probably qualify), i.e., bordering to that you automatically would (or would not) cite both items once you cite one (typically you wouldn't like to cite two different editions of the same book if that is caused by typing the wrong key, but there may be reasons to have both in your .bib-file).

Not "oh, this paper somehow deals with the same subject as this other one" nor citation relations as I primarily talk about. The first type one can almost solve with keywords if they are specific enough. Citations can probably be of interest to add in a standardized way in Biblatex (as well as citation count etc) though.

[koppor]

OK, biblatex also states "reprintof, translationof, and multivolume" as examples, which indicates a real close relationship.

PRRV uses references, but expects there all references in plain text. Thus, we should not use references to be compatible to PRRV.

Maybe a new field relations with a format similar to the files field (or the serialization of customized entry types)? I like simple customized formats and I think JSON is too oversized here. I would also not begin fields like relation1, relation2, ...

[oscargus]

With JabRef#1637 and JabRef#1645 defining at least some relations should be easier now. Not sure where to begin though. What about citations and the automatically generated field citedby?

As opposed to the previous discussion, with the keychangelistener it should be possible to do this with the bibtex keys directly.

[tobiasdiez]

I think I still prefer that relations are stored directly as BibEntries.
That is every BibEntry has a private field (+ get / some kind of setter)
List<BibEntry> related / cited / blafoo.
Upon serialization, the entry keys are fetched and written to the related = {...} field. Reason:
foreach(relatedEntry in entry.getRelated())
is type-safe and easy to read. In contrast,
foreach(key in entry.getRelatedKeys()) relatedEntry = database.getEntryByKey(key);
is ugly.
Summary: we are using an object-oriented approach and thus there is no need to rebuild a relational database system based on IDs.

[oscargus]

Ideally, yes. However, there are some consequences. The lists has to be generated first after all entries are loaded. How do we deal with invalid keys? One of the main problems in JabRef#1637 is was that the entry editors "automatically" serialize the field after every edit, and, hence, my "clever" serialization removing empty fields lead to very strange effects. If you look at the current implementation it is still closing in to it and you will get a list you want, except that the field parsed when getting it: public List<ParsedEntryLink> getEntryLinkList(String fieldName, BibDatabase database). ParsedEntryLink returns an Optional<BibEntry> and can be constructed either from a BibEntry or a key + BibDatabase combination.

[koppor]

Update: Moved to issue description

We should collect our decisions somewhere. I get lost. I will do it here in this comment. @oscargus Feel free to move the things up to your first comment - or I just update this list based on the following comments.

fields

supported

• cites - comma separated list of BibTeX keys which are cited by this entry
• crossRef - single entry which is cross referenced.
• related - comma separated list of BibTeX keys which are in some kind related to this entry. The type of all relations can be specified by a single relatedtype (see Multiple related entries having different relationship types plk/biblatex#475 (comment)). Note: Biblatex prints this information if related is active at the biblatex package.

not supported

• citedBy - this is the opposite of cites. That field has to be used.
• relations - this would introduce a complicated field similar to our save actions. A simple key/value is enough
• references - stores all references in plain text (PRVV plugin). Thus, we do not use it.

[oscargus]

I think related can handle more than one link (and relatedtype is a list as well).

[koppor]

Ah, yeah, on the second page of the biblatex manual Section 3.4, it is there:

@MVBook{key1,
...
related = {key2,key3},
relatedtype = {multivolume},
...
}

I'm not sure, how relatedType works. Currently, it reads that only one type can be given.

[koppor]

Based on plk/biblatex#475 (comment), I updated #14 (comment) accordingly.

[oscargus]

OK! Good! Then we can easily add a drop-down for relatedtype and do not need to add a column for related in JabRef#1637

[oscargus]

It is quite easy to just add support for cites in InternalBibTexFields and set FieldProperty.MULTIPLE_ENTRY_LINKS for the field. It will then be good to go. Later on one may add a virtual citedBy field (that builds on and edits cites) and visualizations and automatic generation and stuff.

[koppor]

The contents of the dropdown should be configrable. Alternative: textfield with autocompleter? 😇

[oscargus]

Yes, or we provide the common/defined alternatives as the user can always type whatever they want.

[koppor]

Regarding citedBy: It is a field in bibentry, not a real field. Otherwise, we are back to JabRef#574 and use DISPLAYABLE and WRITEABLE. Minor comment: I think, fields like citedBy were considered when defining these flags. I fully agree, however, that we should only store a field in our fields map if we write and display it to have the UI consistent with the serialization.

Some free thought: If we really go to have additional fields where we work an and which are not serialized, we really opt-out for changing our backend from in-memory to database-based. Thus, we will never (?) be able to use the full power of SQL. I think, this is currently no issue as we first need to separate logic and UI, then we can think about handling zillions of records (e.g., all available entries in dblp etc).

[koppor]

We should have a look at biblatex. It supports "translated to", ... How does it do it? Which fields are recommended? Support these fields.

[koppor]

Zotero supports related = {bibtex[relationtype=multivolume]}

See retorquere/zotero-better-bibtex#1102 (comment)

[koppor]

Maybe Section 3.4 of the biblatex documentation might be helpful.

It is now 3.5:

It can only host one relation type:

(See also plk/biblatex#475 (comment))

If we use "cites", there cannot be other relations.

[koppor]

We seem to already have implemented cites, related, and crossref: https://docs.jabref.org/advanced/entryeditor/entrylinks