Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(save): Save a dataset with multiple file arguments #718

Merged
merged 3 commits into from
Mar 25, 2019

Conversation

dustmop
Copy link
Contributor

@dustmop dustmop commented Mar 25, 2019

No description provided.

@ghost ghost assigned dustmop Mar 25, 2019
@ghost ghost added the in progress label Mar 25, 2019
@dustmop
Copy link
Contributor Author

dustmop commented Mar 25, 2019

Fixes #690.

@dustmop dustmop requested a review from b5 March 25, 2019 19:31
Copy link
Member

@b5 b5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple quick nits, this is looking really good!

@@ -57,7 +57,7 @@ commit message and title to the save.`,
},
}

cmd.Flags().StringVarP(&o.FilePath, "file", "f", "", "dataset data file (yaml or json)")
cmd.Flags().StringSliceVarP(&o.FilePaths, "file", "f", nil, "dataset or component file (yaml or json)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's currently valid to also supply .html and .star files here directly. Relevant code that handles those extensions is still lib.ReadDatasetFile:

qri/lib/file.go

Lines 135 to 148 in b813d05

case ".star":
// starlark files are assumed to be a transform script with no additional
// tranform component details:
ds.Transform = &dataset.Transform{ScriptPath: path}
ds.Transform.SetScriptFile(qfs.NewMemfileReader("transform.star", f))
return
case ".html":
// html files are assumped to be a viz script with no additional viz
// component details
ds.Viz = &dataset.Viz{ScriptPath: path}
ds.Viz.SetScriptFile(qfs.NewMemfileReader("viz.html", f))
return

Any chance we can get cases that confirm both of those examples work? This is asking you to test code that I've written (😞), but we should confirm this behavior survives through this PR. The motivation is supplying no-configuration transform & viz components:

qri save --file transform.star me/dataset
qri save --file template.html me/dataset

I use both of these rather often, and find they make for easier tutorial writing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, along with an all-in-one combination of meta + transform + viz.

lib/file.go Outdated
f *os.File
data []byte
)
// ReadDatasetFiles decodes a dataset document into a Dataset
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated comment. It'd be great if this function comment broke down some of the nuances of reading multiple filepaths into a dataset, keeping it as high-level as possible. Something like this (adjusting for accuracy):

// ReadDatasetFiles combines resources at zero or more filepaths, deserializing them into a
// *dataset.Dataset. Files can separately specify different components of a dataset
// but two provided components cannot overlap (modify the same component)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

lib/file.go Outdated
data []byte
)
// ReadDatasetFiles decodes a dataset document into a Dataset
func ReadDatasetFiles(pathList []string) (*dataset.Dataset, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer

func ReadDatasetFiles(pathList ... string) (*dataset.Dataset, error)

To me ReadDatasetFiles would be very useful when using qri-as-a-library by hiding the []string from the caller:

//  variadic arguments allows calls for easier single-file loading:
ds, err := lib.ReadDatasetFiles("./path.json")

//  and chaining that doesn't require []string allocation:
ds, err := lib.ReadDatasetFiles("./structure.json", "./meta.yaml")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, this is much better.

@b5 b5 self-requested a review March 25, 2019 21:53
Copy link
Member

@b5 b5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOPE

@dustmop dustmop merged commit d6e5787 into master Mar 25, 2019
@ghost ghost removed the in progress label Mar 25, 2019
@dustmop dustmop deleted the multi-file-components branch March 25, 2019 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants