Simple Python parser

[joelkuiper]

In case someone is interested in a simple parser

def read(ifile):
    obj = {"mention": []}


    HEADER = re.compile(r"(?P<pmid>[0-9]*)\|(?P<t>[t|a])\|(?P<content>.*)")
    MENTIONS = re.compile(r"(?P<pmid>[0-9]*)\t(?P<start>[0-9]*)\t(?P<end>[0-9]*)\t(?P<content>.*)\t(?P<tui>(T.+|UnknownType))\t(?P<cui>C[0-9]+)")


    with gzip.open(ifile, 'r') as fin:
        for line in fin:
            l = line.decode("utf-8")
            h = HEADER.match(l)
            if h:
                obj["pmid"] = int(h.group("pmid"))
                obj[h.group("t")] = h.group("content")
                continue
            m = MENTIONS.match(l)
            if m:
                mention = {"start": m.group("start"),
                           "end": m.group("end"),
                           "content": m.group("content"),
                           "tui": m.group("tui").split(","),
                           "cui": m.group("cui")}
                obj["mention"].append(mention)
                continue
            else:
                yield obj
                obj = {"mention": []}

Code isn't pretty, and not sure how useful outside my own use case; however, I thought it would be nice to share.

[GregSilverman]

@joelkuiper, thanks! This is extremely helpful.

However, there seems to be an issue with the text offsets on a lot of the manual annotations not aligning to what is in the text (that is, the title + abstract).