Data and code availability statement zone

[kermitt2]

Recognition of data and code availability statements, either in the header or the end of an article and marking the section in a normalized place in the TEI result (similarly as the acknowledgement section).

This involves the segmentation model (to recognize the zone as an additional section after the main article body) and the header model (when the zone is located within the header).

Thanks @lfoppiano :D

[coveralls]

Coverage decreased (-0.005%) to 39.879% when pulling f427ad7 on feature/data-availability-statement into 54d1c29 on master.

[coveralls]

Coverage decreased (-0.03%) to 39.928% when pulling 1925296 on feature/data-availability-statement into f9dc68f on master.

[coveralls]

Coverage decreased (-0.006%) to 39.878% when pulling 40331eb on feature/data-availability-statement into 54d1c29 on master.

[kermitt2]

Hi @lfoppiano !

I reviewed everything and made some few changes - in particular put back the header labels into TaggingLabels.java to avoid a breaking change in all the current grobid modules (which uses labels from the class TaggingLabels - I actually wanted to put back the segmentation labels too in TaggingLabels to simplify a few years ago, but I didn't do it to avoid a breaking change).

Currently, when the availability statement is in the header, it is correctly labeled and stored in BiblioItem.java, but it is not serialized in the final TEI because the method used can only work with segmentation label:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/engines/FullTextParser.java#L2473

            // data availability statements
            StringBuilder dataAvailability = new StringBuilder();
            if (StringUtils.isNotBlank(resHeader.getDataAvailability())) {
                dataAvailability = getSectionAsTEI("availability", "\t\t\t", doc, TaggingLabels.HEADER_AVAILABILITY,
                    teiFormatter, resCitations, config);
            } else {
                dataAvailability = getSectionAsTEI("availability", "\t\t\t", doc, SegmentationLabels.AVAILABILITY,
                    teiFormatter, resCitations, config);
            }

getSectionAsTEI() then uses:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/engines/FullTextParser.java#L2517

SortedSet<DocumentPiece> sectionPart = doc.getDocumentPart(taggingLabel);

Only the SegmentationLabel work here with doc.getDocumentPart() (using TaggingLabels.HEADER_AVAILABILITY here will return nothing).

To retrieve the layout tokens relative to the availability statement from the header stored in BiblioItem.java, normally we use:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/data/BiblioItem.java#L4238

List<LayoutToken> headerAvailabilityStatementTokens = biblio.getLayoutTokens(TaggingLabels.HEADER_AVAILABILITY);

[kermitt2]

Latest commits fixed the header availability statement TEI serialization.
createTraining for header and segmentation now also pre-labeled availability stmt and funding sections.