🐈 Task: Coordinate Test Module with new metadata

[GemmaTuron]

Summary

This task is related to these two issues:

#1484 - The test module v1 is being completed and almost ready
#1477 - Upgrade Airtable metadata

Once the Airtable is consolidated to the new format, the Test module will need to be revised, mainly:

• Ensure the metadata fields are properly accounted for
• Use new metadata fields like Output Consistency (Fixed/Variable) to avoid the test to fail when it should not
• Use the column header files and the example files that will be then available for all models in the test command

Objective(s)

No response

Documentation

No response

[GemmaTuron]

Hi @Abellegese and @DhanshreeA

Let me summarise here the main changes (not mentioning all fields only the ones that are relevant or changed):

Metadata

1. Mode (pretrained-retrained etc) disappears
2. Task (annotation-representation-sampling...) + Subtask (Activity prediction- Featurization...) appear
3. Input is always compound, Input Shape is always Single
4. Output remains as it was
5. Output dimension (1,2,3...1024 etc) is added
6. Output consistency: fixed or variable will let you know if the result should always be the same or not, allowing skipping the consistency result in those models that are variable
7. Output Type float-integer-string
8. Output shape: list-single
9. Biomedical Area: list of diseases if relevant (else: Any)
10. Target Organism: species if relevant (else: Not Applicable)
11. Publication type: peer reviewed, preprint, other
12. Publication year
13. Licenses: consolidating all to SPDX
14. Source: Local or Online (ie whether the model connects to an online server or it has all it needs to run locally, mostly are local)
15. Source Type: internal-external-replicated

And then the new metadata fields that will be filled in at test/build moment: environment size, image size, computational performance 10, computational performance 100, Docker Pack Method

Columns
Column headers are now fixed in a file on framework/columns/run_columns.csv The test command can use it or not, as you think best

Examples
The models have a framework/examples/input.csv and framework/examples/output.csv with three molecules. Not sure what Ersilia is using at the moment, not sure this is needed for the test command - maybe to test if the output is the same as originally?

I suggest the following:

1. Create a Python script (if it does not yet exist) that reads from the Airtable Reannotation and updates the model metadata either on .json or yml @DhanshreeA
2. Test the script in one model so Abel can see how the data looks like @DhanshreeA
3. Update the Test command accounting for these improvements (bare minimum is to check the metadata fields present and the Fixed/variable thingy, not sure if more is needed) @Abellegese
4. Try the new workflows using the test command on the example model only until we are happy with it (I am worried about overwriting stuff etc) @DhanshreeA
5. Update the workflows in the models in batches of 10 and work on those until all are working @DhanshreeA

[GemmaTuron]

We will use these three models as example @DhanshreeA 👍

• eos9gg2 - FastAPI
• eos80ch - legacy template
• eos8fma - variable output