Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add flags to qri save for setting individual components of a save #690

Closed
b5 opened this issue Feb 11, 2019 · 5 comments
Closed

add flags to qri save for setting individual components of a save #690

b5 opened this issue Feb 11, 2019 · 5 comments
Labels
API JSON API issues CLI command line client issues feat A code change that adds functionality
Milestone

Comments

@b5
Copy link
Member

b5 commented Feb 11, 2019

I'm having a semi-frustrating time always having to create a dataset.yaml or dataset.json just to update a transform script. What I'd like to be able to do from the CLI is one of two things. The first would be providing json/yaml snippits for each component. In each of these cases the _file argument would be a file that only defines that component, not an entire dataset:

$ qri save --meta meta_file.json me/dataset
$ qri save --structure structure_file.json me/dataset

$ qri save --viz viz_file.yaml me/dataset
$ qri save --transform transform_file.json me/dataset

# we already support this:
$ qri save --body file.csv me/dataset

The second would be providing a raw script as a file as a shortcut for "default settings, but with this file":
qri save --transform tf_file.star me/dataset
ditto for viz scripts:
qri save --viz template_file.html me/dataset
This would just do the right thing, populating a component with defaults (like setting syntax to html or starlark) and setting the scriptPath to the supplied path argument

In a later PR it'd be great to leverage the qri values we embed into datasets to intelligently accept a dataset.json definition, but if passed to a specific component argument, we would traverse to and only use that component of the supplied dataset. For example this:

qri save --meta other_dataset.json me/dataset

would be the same as saying "I've exported some other dataset as other_dataset.json, please copy it's meta component onto me/dataset. This is future work. For now it'd be great if we errored if you passed a file that had "qri": "ds" in the top level to any component argument

@b5 b5 added feat A code change that adds functionality CLI command line client issues API JSON API issues labels Feb 11, 2019
@b5 b5 added this to the 0.7.1 milestone Feb 11, 2019
@dustmop
Copy link
Contributor

dustmop commented Feb 11, 2019

I'm not a huge of having a flag for each component, leads to too many flags that are similar things but are also mutually exclusive. I would prefer --component <component_name>. Both --file and --body are somewhat special in how they work, for example, either can be used to create a new dataset, while --component couldn't be.

@b5
Copy link
Member Author

b5 commented Feb 12, 2019

In the alternative you're outlining I would do qri save --component structure --file st_file.json? Just making sure.

Agreed on the many/too many flags issue. I want to make sure I understand the alternative, but for I can see this adding up to overloading the --file flag a whole bunch instead of flags for each component:

# do what I was mentioning above, inferring this is a default transform
$ qri save --file transform.star me/dataset

# skip the whole --component thing, instead let users specify components with "qri" : "st" top level values
$ qri save --file component_file.json me/dataset

This would have a few implications:

  • data, script, and viz file extensions would need to be mutually exclusive
  • can't compose multiple components from different source files in a single save invocation (b/c no moar flag slots)

What about dropping the --component business altogether, and just loading up --file with new tricks? After working with Qri a bunch I really just want this flow of applying stuff from one dataset to another, which overloading the --file flag would satisfy:

$ qri get transform.script me/dataset_a > transform.star
$ qri save --file transform.star me/dataset_b

I like this approach more b/c all I have to think about is the --file flag, no more bending the mental model around which flag to use when.

Toughts? cc @camhunt

@dustmop
Copy link
Contributor

dustmop commented Feb 12, 2019

In the alternative you're outlining I would do qri save --component structure --file st_file.json?

Yes, this was the possibility I was thinking of.

What about dropping the --component business altogether, and just loading up --file with new tricks?

I think this is an even better idea. Also, we can allow changing multiple components by passing multiple --flag values:

qri save --file change_meta.json --file change_structure.json me/my_dataset

To do this we would change --file to use a StringSliceVar: https://godoc.org/github.com/spf13/pflag#StringSliceVar

@b5 b5 added the ready label Feb 18, 2019
@b5
Copy link
Member Author

b5 commented Mar 17, 2019

so looks like we only have to get the multiple --file flags working to close this out. We've also added .star -> transform with a scriptPath and .html -> viz with a scriptPath.

@dustmop
Copy link
Contributor

dustmop commented Mar 28, 2019

This is working now, closing this issue.

@dustmop dustmop closed this as completed Mar 28, 2019
@ghost ghost removed the ready label Mar 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API JSON API issues CLI command line client issues feat A code change that adds functionality
Projects
None yet
Development

No branches or pull requests

2 participants