Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is specifically meant for the way data is stored in the MX beamlines, it can be expanded to other collections if needed.
The MX beamlines use analysis store headers to store the results of rasters. An example of a raster result is as follows:
For context, dumping a raster data like above with 195 cells into a file with indent=4 takes around 3500 lines. Not all of the data is necessary for purposes of reconstructing the raster. Specifically, there is no need for "rasterCellMap", or data within "rasterCellResults.resultObj" for certain situations. By using projections, one can remove keys like "rasterCellMap" and potentially reduce the processing time to get raster data, especially if we filter a large number of rasters in 1 query.
Removing keys from within "rasterCellResults" is not straightforward but could be, by changing the structure of data. But that is a PR for another repo.