-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow archiving histories and workflows #1734
Comments
To add to this, I'd actually find just a 'locked' attribute to be great to have separately from archived. This is particularly important for e.g. published histories (especially ones referenced in papers) -- it is a sad day when you accidentally add some cruft to that perfectly created history that you want to share (instead of the copy you made). |
Instead of just archived, could we create "Groups" for histories, where "Archived" could be a particular group? That way we can organize things even a bit better. |
@blankenberg sorry, missed your replies. A locked attribute would be amazing, I definitely agree w/r/t published histories. Groups is an interesting thought. What sort of other groups would you envision having? |
Update: added workflows to this list. Same problem. I want them to be locked down and moved to a different area, out of my sight. |
This sounds like a great idea. I fully support it. I'd like to add another suggestion: the ability to delete underlying datasets in a history (to conserve space), but save the tool. Pretty much the same as a deleted dataset, but without deleting the item in the history. It would be great for users to be able to archive (online and offline) certain outputs (counts, fold changes, plots, etc.) but not some of the redundant or big data (raw fastq stored elsewhere, bam files, etc.). Of course, preserving the provenance of those final outputs would be needed. Not sure if I made this totally clear, so please let me know if further clarification would be helpful or if there is a better place to record this. |
related: #3269 |
Ideas on how to archive data which is old, unused, ...
|
I think I heard some discussion somewhere that an "archive histories" function could also enable moving them to colder, tape storage (for admins that configure it) which would be super fantastic. |
This would be brilliant! I have major issues with accidentally modifying histories I previously shared as examples, I'm a big fan of this. Plus presumably this could make them citable as well. i.e. 'made this tool, history is here for its use, cite me for being awesome' or some such. |
Like the idea:
Wanted something like this for a long time. |
It's alive again ;-) |
This is the current history table.
We could add a new column @blankenberg would tags instead of groups work? We have the tagging infrastructure in place. We could think about a tag-centric UI where you see all histories with the same tag in a tree-view? Could the archived or |
@foellmelanie mentioned this during her talk, @Slugger70 DM'd me about it as well his users want this. Especially in the context of RO-Crate maybe we can finally do something about this. Archive the history, and move it to another system with storage (e.g. zenodo) |
Or other cheap storage like an object store (swift, s3 etc.) |
Yes! Cheap object store, exactly. Could be tape. Could be anything, just a method (especially using the File Sources) letting users archive something and have a URL they can point to that sufficiently describes it, while not taking up space. |
Pinging @davelopez who presented some ideas for this at a recent meeting; we're hoping to address this soon. |
Yes, will open a planning issue soon and link it here 👍 |
The part for histories is already done in #16003 I'll keep this open for the potential Workflow part but I'll move the milestone. |
My users report having "too many histories" to deal with effectively. I feel their pain, and there are histories that I'm not going to reference very often but would like to retain forever.
I'd like to propose an "archive" for histories, it's another state that histories can transition into.
Properties:
Update (2016-05-16): I've found the same issue with workflows. There are workflows which I should retain for historical reasons, but will never use again (or are broken, but we want to record exactly what was run)
The text was updated successfully, but these errors were encountered: