94 save objects in qs #99
Conversation
add labkey functionality and update dp_init gitignore
|
This throws a wrench in things, but looks like qs has a planned deprecation. qsbase/qs#103 Can we test with qs2 instead? |
|
Great catch! We can definitely implement qs2 once it is supported by |
|
The changes introduced will be backward compatible with existing daaps |
There was a problem hiding this comment.
I tested with local and s3 data products and everything is working smoothly.
Note that with the current implementation daaps with multiple types (ie a daap started with rds and switched to qs) may have unexpected behavior with get_data_object_path but I think this can be handled via documentation in a future qs vignette.
| @@ -58,3 +55,22 @@ dp_commit <- function(project_path = fs::path_wd(), | |||
|
|
|||
| return(repo) | |||
| } | |||
|
|
|||
| get_data_object_path <- function(project_path) { | |||
There was a problem hiding this comment.
Do we need to take into account the case where a data product may have been started in rds format and switched to qs format? If so, this will always return the RDS path preferentially because of the order of the list
There was a problem hiding this comment.
There should be no issue if the data product was initially generated in one format and later switched to another, provided that the repository is freshly cloned or synchronized, therefore the contents of the output_files directory are ignored. However, it could be problematic if a user continues to develop the data product within an existing repository and decides to switch the format type. While this scenario is unlikely, it would be prudent to include a warning in the vignette explaining how the new type parameter functions.
Ability to write and read data objects in qs format will improve the speed with large daaps. There is a new
typeparameter indp_write, which is set to 'rds' by default. I had to create some helper functions to accommodate the convention of saving RDS objects under "output_files/RDS_format" directory. Please take a look closely and test it on your environment.