Speed up the transformation from KGX graph to RDF file

[0m1n0]

Hello Biolink team,

I recently found Biolink Model with KGX and I am deeply grateful for your work as I see a huge benefit for the scientific world. I also appreciate your effort in documentation and unit tests as well as your reactivity.

I am working on a microbial ecosystem project and would like to create a RDF that represents: GPR (gene-protein-reaction) and corresponding metabolic pathways.

As I did not find the function allowing the transformation of a tsv file into RDF, I created a network as an intermediate step: a test with ~900 nodes and ~900 edges.

1. Create nodes.tsv and edges.tsv with Biolink Model vocabularies
2. Transform nodes and edges data to KGX graph (<1 second)

input_args = {
    "filename": ["nodes.tsv", "edges.tsv"],
    "format": "tsv"
}
t = Transformer()
t.transform(input_args)

3. Convert graph to nt format file (~8 seconds)

output_args = {
    "filename": "rdf.nt",
    "format": "nt",
}
t.save(output_args)

Here are my questions:

1. I see that we can specify biolink:provided_by in nodes.tsv otherwise KGX takes the name of the nodes file by default. How can we remove it? Probably an argument to be specified in input_args ?
2. I may have missed it, but is there a function that allows direct transformation from tabular file to RDF (nt, ttl file)?
3. The conversion time from graph to nt file is long (~8 seconds for a small graph). Can you give me some advice on how to speed up this process?

Thank you for reading :)

Regards,
Min

[sierra-moxon]

Hi @0m1n0 :) - can you put a ticket in this repo for your first question? I think for most of our existing use cases, provenance is essential. But I'd love to hear a little more about why you'd like to remove it, and we can think about ways to allow removal.

For 3 - did you use the "stream=True" flag in your transform?

[0m1n0]

Thank you again @sierra-moxon for your replay.
I put a ticket as an issue: Set the provided_by of nodes.tsv as an option #400.

I had not seen stream=True! Indeed it accelerated about 40%. Thank you very much!