Store atlas metadata when importing and registering atlases #65
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Provide a function to backup a target location before copying in a file.
Provide an option to convert Enum keys and values to their corresponding class and names as strings. This format allows them to be reloaded through the custom YAML load function.
Labels metadata such as the labels reference specification and the original label IDs have been discovered by reloading them from the original atlas file when viewing an imported atlas. While this method reduces redundancy, it requires the original atlas to be available and specified through the `--labels` parameter. Loading the original label IDs also requires loading the original annotations image, which can have a performance impact if the image is large. To collect all the metadata in the imported atlas folder, create a labels metadata YAML file and copy in the reference file. The metadata file references the filename of the specification file since it may have a different name and format for any given atlas. Additionally, store the original label IDs directly in the metadata so that the original image does not need to be opened.
- Load metadata to identify the labels reference file - Load this reference file if available, but prioritize any reference file given directly through `--labels` - Load the original label IDs from the metadata, falling back to the original image if necessary
Save the labels metadata and associated labels reference specification from an atlas to the output folder.
Store labels metadata in a class structure to encapsulate for better encapsulation, including its own saving and loading mechanisms.
The labels metadata class has been accessed as a dictionary backed by Enum-based keys so that the keys are specified programmatically rather than simply as strings both both accessing metadata and saving/reloading it in a YAML file. As a simplification, the class could use attributes directly instead, with the keys only specified in the save and load functions for the YAML file. Also, prepend the reference path with its working directory when loading the path.
- Parameter to check whether the base path is an existing directory to be simply joined to the suffix when combining paths - Support multiple extensions
Labels metadata has accepted a custom directory, extended here to allow a full path prefix. Also, move the path constant into the class.
Continue to support giving an atlas directory alone during registration by providing a labels metadata symlink with only the suffix part of the filename in the target directory, excluding the rest of the path prefix. Add a library wrapper to the symlink function with basic error handling.
- Ensure that the metadata is loaded from a regular file rather than a directory - Use this load path to make the relative reference file path absolute rather than converting the prefix - Add labels metadata docs
Get the path from the metadata if available and not overridden by the CLI argument.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After atlas import, the original atlas has continued to be accessed for two pieces of metadata:
.jsonor.csvfile specifying the label IDs, names, and levelsThis PR integrates this metadata into the imported atlas so that the original atlas no longer needs to be referenced. The imported atlas directory thus becomes self-sufficient, without the need to use the
--labelsflag when loading the atlas. The original annotation image also no longer needs to be reloaded for a slight performance improvement.The metadata is stored in a
<base-path>_meta_labels.ymlfiles, currently with only two entries:path_ref: the name of the reference file, which is copied into the atlas directoryregion_ids_orig: a list of the original IDsThe
--labelsflag must now be used at the atlas import stage (--register import_atlas) to find the reference file. The reference file can be of any format since it is copied in directly, though currently only JSON and CSV files can be loaded for use in the package. Once the reference file is copied in and atlas metadata generated, the--labelsflag is no longer necessary to load the atlas with labels, and registering the atlas to another image will similarly transfer the metadata and reference file to the registered version of the atlas.