Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User-facing objectstore metadata. #10233

Merged
merged 4 commits into from
Nov 30, 2020
Merged

Conversation

jmchilton
Copy link
Member

#6552 implemented the ability for admins to assign job outputs to different object stores at runtime and #10221 implements the ability to assign quotas and perform tracking for different object stores and groups of object stores. So for instance, user preferences or injected tool resource parameters can be used to create files in different object stores and disk usage and quota limits will be tracked and can be used in that calculation.

Still the user has no way of knowing what data went where or what that means for their analysis. Beyond this retrospective use case, as we look forward to user-based selections - we are not currently tracking metadata that might help a user make selections about which object store to use. This PR adds the ability for admins to annotate objectstores with more metadata - enough to solve this first problem and provide some infrastructure for solving that selection problem in the future.

Concrete ObjectStores (defined in either XML or YAML) can now be named and can have a rich Markdown description. A dataset API endpoint has been added that exposes this metadata for a particular existing dataset and this information will now be rendered as part of the dataset info page.

Here are two examples - illustrating how admins might highlight long term, backed up storage and short term scratch storage. They also demonstrate the power of allowing admins to define these annotations using Markdown.

Long Term Storage:

Screen Shot 2020-09-16 at 2 05 25 PM

Short Term Storage:

Screen Shot 2020-09-16 at 2 15 11 PM

@jmchilton jmchilton force-pushed the data_inspector branch 2 times, most recently from 5be378d to 311ad5e Compare September 29, 2020 21:50
@jmchilton jmchilton changed the title [WIP] User-facing objectstore metadata. User-facing objectstore metadata. Sep 30, 2020
@galaxybot galaxybot added this to the 21.01 milestone Oct 6, 2020
This data is stored in a Galaxy ObjectStore named <b>{{ storageInfo.name }}</b
>.
</p>
<p v-if="storageInfo.object_store_id">
Copy link
Member

@mvdbeek mvdbeek Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If both name and id are defined this seems to take up more screen space than needed, maybe you could combine that into one sentence (or a small table?)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mean t to be an else-if I think. Thanks for the catch!

Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants