Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Add some relationships, such as “trainedOn” or “finetuningOn”, to indicate the relationship between model and dataset #982

Open
gmscofield opened this issue Jul 6, 2024 · 1 comment
Labels
profile: ai AI Profile and related matters
Milestone

Comments

@gmscofield
Copy link

gmscofield commented Jul 6, 2024

The specification now seemed to lack properties about the relationship between models and datasets. All I can see about this information is the property "informationAboutTraining". However, there are two reasons for strengthening relationships between models and datasets:

  1. Data poisoning have been a common method to attack llm. It's necessary to make it clear what dataset llm is trained on or fine-tuned on to help ai-profile users protect against llm attacks more effectively. At least they can find some similar ways to defend attacks from other models trained on the same datasets.
  2. Data profile and ai profile have been separate profiles. Adding such relationships in spdx will make their connections more clear.(It looks like these two profiles have nothing to do with each other now)
    So I think that adding relationships between models and datasets in spdx is necessary. What's more, relationships between datasets are also valuable, such as sampledFrom.
@bact
Copy link
Collaborator

bact commented Jul 7, 2024

RelationshipType already has trainedOn.

See: https://spdx.github.io/spdx-spec/v3.0/model/Core/Vocabularies/RelationshipType/

The discussion to add finetunedOn is on going for 3.1.

See meeting 2024-06-05: https://github.com/spdx/meetings/blob/main/ai/2024/2024-06-05.md

We have AI and Dataset Profiles meeting every Wednesday.
You are welcome to join. See time and meeting link here: https://github.com/spdx/meetings/

@kestewart kestewart added this to the 3.1 milestone Aug 11, 2024
@bact bact added the profile: ai AI Profile and related matters label Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
profile: ai AI Profile and related matters
Projects
None yet
Development

No branches or pull requests

3 participants