Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added suport for multiline variables from sh #64

Merged
merged 1 commit into from
Sep 4, 2017
Merged

Conversation

smyrman
Copy link
Contributor

@smyrman smyrman commented Sep 3, 2017

Fixes #63.

Instead of giving an error on multiline results from sh, the results are now concatenated to one line by default, separated by a single space. Trailing and leading white space are stripped like before.

In adition, some flags have been added for advanced use cases to turn off line concationaton and/or trimming off leading/trailing white-space.

Multiline results are now allowed, while the last newline is stripped to allow the output of most commands to work better as variable values. Some helper functions are added to make it easier to deal with multi-line variables in various ways.

@smyrman smyrman force-pushed the multiline branch 4 times, most recently from 06d2240 to 1c29cc9 Compare September 3, 2017 11:23
@andreynering
Copy link
Member

Thanks again.

In adition, some flags have been added for advanced use cases to turn off line concationaton and/or trimming off leading/trailing white-space.

I'm a bit in doubt about what approach to push.

I kind of dislike these boolean flags, they're a bit too verbose. Now we have two, in the future, if we have more use cases, then we'll have many?

This is why I thought about turning it in a slice. Then the use can programmatically do exactly what they want. No flag needed.

Replacing the Var.Static attribute with a []string is a bigger and also a breaking change. E.g. it would break both set and cases where the end-users define multiline values in task via yaml

I think in these cases we would just push this string as the first element of the slice, to keep the current behavior.


I want your opinion if that would solve your use case from #63.

@andreynering
Copy link
Member

andreynering commented Sep 3, 2017

OK, let's do the simplest thing. One can just split the result if they want, as you suggested before

  • Don't change to slice, it's simpler as is 👍
  • Get rid of the multiline flag and keep line break by default. The user can replace it if they want
  • What about trimming? Maybe just trim if there's a single line break in the end of the string?

@smyrman
Copy link
Contributor Author

smyrman commented Sep 3, 2017

Get rid of the multiline flag and keep line break by default. The user can replace it if they want

Alright. 👍 I can add some convenience functions to make it easy to concat lines or split them (including Windows styled newlines).

What about trimming? Maybe just trim if there's a single line break in the end of the string?

I guess that makes sense. That should make most command output work as expected (there is usually just one trailing newline), but leave some leeway for plausible use cases where you would want some of the whitespace kept, e.g. code generation.

@andreynering
Copy link
Member

I can add some convenience functions to make it easy to concat lines or split them (including Windows styled newlines).

Maybe something for the future, since even on Windows files often use \n instead of \r\n.

About the trimming, maybe always trim the last line break would be better because:

  1. Unix tools like cat adds a trailing line break anyway
  2. strings.Join adds a final empty string to slice with no trim: https://play.golang.org/p/K0clOebD-s

@smyrman
Copy link
Contributor Author

smyrman commented Sep 3, 2017

About the trimming, maybe always trim the last line break would be better because:

Yes Agreed.

Maybe something for the future, since even on Windows files often use \n instead of \r\n.

Yeah, but you just need to replace "\r\n" with "\n" and then do the split. This is similar to how e.g. the Go line scanner works, although it uses "peek" to be more memory efficent.

I wrote some, but a bit unsure about the best way to name them, as go-taks appears to use PascalCase and UPPERCASE, where as sprig uses camelCase for functions.

"splitLines": func(s string) []string {
s = strings.Replace(s, "\r\n", "\n", -1)
return strings.Split(s, "\n")
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👆

Copy link
Contributor Author

@smyrman smyrman Sep 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you prefer the PascalCase, I can rename them, or if you prefer consistency with sprig, we can rename all of them, but leave backwards compatibility:

 	taskFuncs := template.FuncMap{
		(...)
		"fromSlash": func(path string) string {
			return filepath.FromSlash(path)
		},
		"toSlash": func(path string) string {
			return filepath.ToSlash(path)
		},
		"exeExt": func() string {
			if runtime.GOOS == "windows" {
				return ".exe"
			}
			return ""
		},
		// IsSH is deprecated.
		"IsSH": func() bool { return true },
	}
 	// Deprecated aliases for renamed functions.
	taskFuncs["FromSlash"] = taskFuncs["fromSlash"]
	taskFuncs["ToSlash"] = taskFuncs["toSlash"]
	taskFuncs["ExeExt"] = taskFuncs["extExt"]

We can comment the backwards compatibility only very briefly in the README.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with renaming them. The reason this is not consistent is because spring was added after some of the other funcs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I assumed it was historical reasons:-)

key, val := keyVal[0], keyVal[1]
m[key] = Var{Static: val}
}
return m
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did some re-ordering in this file to get a better overview. Basically the ordering rule is to place methods and creator functions right below the type definitions.

Don't know if you agree with this ordering, or if you prefer a different style @andreynering


// UnmarshalYAML implements yaml.Unmarshaler interface
// UnmarshalYAML implements yaml.Unmarshaler interface.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trailing period is recommended by Effective Go, or at least the way I interpret it:

Doc comments work best as complete sentences, which allow a wide variety of automated presentations. The first sentence should be a one-sentence summary that starts with the name being declared.

The standard lib uses a period consistently throughout all libraries as far as I have seen. Note that I am only talking about doc comments, not code comments.

variables.go Outdated
var sh struct {
Sh string
}
var sh struct{ Sh string }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should revert this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@smyrman smyrman force-pushed the multiline branch 2 times, most recently from e44fa3d to e422ac3 Compare September 3, 2017 22:12
variables.go Outdated
// Trim a single trailing newline to make the result easier to use in shell
// commands.
if strings.HasSuffix(result, "\n") {
result = result[0 : len(result)-1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

result = strings.TrimSuffix(result, "\n")

If also not required.

@andreynering
Copy link
Member

Two notes, then you can merge this. Thank you for your work and patience.

Feel free to push directly to master if/when you have small fixes and are confident. Specially for documentation and comments.

Instead of giving an error on multiline results from sh, the results are
now stored as is, except that the last newline is stripped away to make
the output of most commands easy to use in shell commands.

Two helper functions have been added to help deal with multi-line
results. In addition, previous PascalCase template function names have
been renamed to camelCase for consistency with the sprig lib.
@smyrman smyrman merged commit 7a64530 into master Sep 4, 2017
@smyrman smyrman deleted the multiline branch September 4, 2017 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants