Auto-approval for datasets

[zsaltys]

Is your feature request related to a problem? Please describe.
Sometimes teams publish public data for which they are happy to give everyone access without any approvals. Having to always request approval just adds unnecessary delays and interrupts the approving team as they have to monitor request emails and approve requests.

Describe the solution you'd like
There should be a simple editable checkbox on the dataset that allows to mark a dataset as auto-approve. If a dataset is marked an auto-approve then after submit is clicked the share should immediately go through without an approve step.

[dlpzx]

Very interesting feature @zsaltys! We can start working on ideas and designs directly in this issue. That will help us clarifying the specific tasks

[dlpzx]

We can split the work in 2 different tasks:

1. Include "auto-approve" option in Dataset creation and store it as metadata

This list of sub-tasks can be modified with the questions raised below:

• Modify Dataset creation UI view and add checkbox "auto-approve"
• Modify createDataset API with the new input field and modify the resolver to store the auto-approve info in RDS
• Modify Dataset GraphQL object and add new column auto-approve
• Modify Dataset RDS table and add new column auto-approve
• Add migration script to add the new column to RDS table

Some ideas/questions to start the discussion:

• Can we re-use existing metadata? For example, we classify the confidentiality level public, official , secret. At first I thought about re-using the public confidentiality tag as "auto-approve", but I don't think it is enough intuitive for users to infer that if a dataset is tagged that way it is automatically shared. Plus, we would need to deal with backwards compatibility.
• Should dataset owners be able to modify the auto-approve value for an existing dataset?

2. Modify share workflow to skip approval process on auto-approved datasets

This list of sub-tasks can be modified with the questions raised below:

• Modify the submitShareObject API resolver to execute 2 state machine transitions if auto-approve is enabled in the dataset (The sharing workflow is defined as a state machine in dataall.modules.dataset_sharing.db.share_object_repositories.ShareItemSM)

Some ideas/questions to start the discussion:

• The proposed sub-task implements the simplest solution for this feature without modifications to ShareItemSM . We can also consider improvements in the state machine definition to make it more flexible and configurable. @noah-paige I would like to know your thoughts.

[SofiaSazonova]

@dlpzx
Currently confidentiality levels are "unclassified", "official" , "secret".
We can add one "public (auto-approval)". Anyhow, I think, it would be useful to add a legend into GUI: what level means what. May be as a floating info-note or separate link.

But it is not the solution, if we want to keep 'auto-approval' as a separate property of data set, not connected with confidentiality. May I mark e.g. the 'secret' dataset with auto-approval flag? If not, than it's better be connected to confidantiality.

[anmolsgandhi]

To align with the described issue, I propose we make it unrelated to confidentiality levels since confidentiality standards may vary for customers. Rather than a blanket approval for say ‘unofficial/public,' let's introduce a new field (checkbox) for each dataset, enabling auto-approval. Some datasets may be public but the dataset owner may still want to have control over when it should be approved for use. Essentially. we should allow users to mark datasets as auto-approved during dataset creation/import. we store that flag and use it at the time of share request consumption to skip the approval process. Also, this should also retroactively work for already created datasets and the users should able to update metadata and check the auto approve box which would then make that existing dataset auto approved for new shares. Thought? @SofiaSazonova @noah-paige @dlpzx

[noah-paige]

Agree with the above - think keeping auto-approve separate from confidentiality may be best

Currently confidentiality affects the level of data preview that is allowed but also could be extended in the future to change the immutability/editability of the dataset parameters or similar.

I think this new feature of auto-approval can be a separate standalone boolean property of the dataset specific to share requests

Nonetheless, I think adding a tooltip or some legend in the UI to make sense of what each confidentiality level means could be a nice enhancement as well as it is not very intuitive to new data.all users as of now

[dlpzx]

To not repeat the same, I agree with the above. For simplicity and to make users aware of the "auto-approve" let's keep it a separate feature