This repository has been archived by the owner on Aug 22, 2022. It is now read-only.
Go: Embedded Static Assets Proposal #114
Comments
svengreb
added a commit
to svengreb/wand
that referenced
this issue
Dec 9, 2020
The github.com/markbates/pkger [1] Go module provides the `pkger`
command, a tool for embedding static files into Go binaries.
To configure and run the `pkger` command, the new `task.GoModule` [2]
has been implemented in the new `pkger` [3] package that can be run
using the "gobin" command runner [4] or any other command runner [5]
that handles tasks of kind `KindGoModule` [6].
The task is customizable through the following functions:
- `WithEnv(env map[string]string) pkger.Option` - sets the task specific
environment.
- `WithExtraArgs(extraArgs ...string) pkger.Option` - sets additional
arguments to pass to the command.
- `WithIncludes(includes ...string) pkger.Option` - adds the relative
paths of files and directories that should be included.
By default the paths will be detected by `pkger` itself when used
within any of the packages of the target Go module.
- `WithModulePath(path string) pkger.Option` - sets the module import
path.
- `WithModuleVersion(version *semver.Version) pkger.Option` - sets the
module version.
The "elder" reference implementation provides the new `Pkger` method
including the handling of the "monorepo" workaround (see section below).
>>> Official "Static Assets Embedding"
Please note that the "pkger" project might be superseded and
discontinued due to the official Go toolchain support for embedding
static assets (files) [7] that will most probably be released with
Go version 1.16 [8].
Please see the official draft document [9] and markbates/pkger#114 [10]
for more details.
>>> "Monorepo" Workaround
"pkger" tries to mimic the Go standard library and the way how the Go
toolchain handles modules, but is therefore also affected by its
problems and edge cases.
When the `pkger` command is used from the root of a Go module
repository, the directory where the `go.mod` file is located, and there
is no valid Go source file, the command will fail because it internally
uses the same logic like the `list` command of the Go toolchain [11]
(`go list`).
Therefore a "dummy" Go source file may need to be created as a
workaround. This is mostly only required for repositories that use a
"monorepo" layout [12] where one or more `main` packages are placed in a
subdirectory relative to the root directory, e.g. `apps` or `cmd`.
For repositories where the root directory already has a Go package,
that does not contain any build constraints/tags, or uses a "library"
layout, a "dummy" file is probably not needed.
The new `Pkger` method of the "elder" reference implementation handles
the creation of a temporary "dummy" file that gets deleted automatically
when the tasks finishes in order to avoid the need for the user to add
such a file to the repository and commit it into the VCS.
Please see markbates/pkger#109 [13] and markbates/pkger#121 [14] for
more details.
[1]: https://pkg.go.dev/github.com/markbates/pkger
[2]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#GoModule
[3]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/pkger
[4]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gobin#Runner
[5]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner
[6]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#KindGoModule
[7]: golang/go#41191
[8]: https://github.com/golang/go/milestone/145
[9]: https://go.googlesource.com/proposal/+/master/design/draft-embed.md
[10]: markbates/pkger#114
[11]: https://pkg.go.dev/cmd/go/#hdr-List_packages_or_modules
[12]: https://trunkbaseddevelopment.com/monorepos
[13]: markbates/pkger#109
[14]: markbates/pkger#121
GH-52
svengreb
added a commit
to svengreb/wand
that referenced
this issue
Dec 9, 2020
The github.com/markbates/pkger [1] Go module provides the `pkger`
command, a tool for embedding static files into Go binaries.
To configure and run the `pkger` command, the new `task.GoModule` [2]
has been implemented in the new `pkger` [3] package that can be run
using the "gobin" command runner [4] or any other command runner [5]
that handles tasks of kind `KindGoModule` [6].
The task is customizable through the following functions:
- `WithEnv(env map[string]string) pkger.Option` - sets the task specific
environment.
- `WithExtraArgs(extraArgs ...string) pkger.Option` - sets additional
arguments to pass to the command.
- `WithIncludes(includes ...string) pkger.Option` - adds the relative
paths of files and directories that should be included.
By default the paths will be detected by `pkger` itself when used
within any of the packages of the target Go module.
- `WithModulePath(path string) pkger.Option` - sets the module import
path.
- `WithModuleVersion(version *semver.Version) pkger.Option` - sets the
module version.
The "elder" reference implementation provides the new `Pkger` method
including the handling of the "monorepo" workaround (see section below).
>>> Official "Static Assets Embedding"
Please note that the "pkger" project might be superseded and
discontinued due to the official Go toolchain support for embedding
static assets (files) [7] that will most probably be released with
Go version 1.16 [8].
Please see the official draft document [9] and markbates/pkger#114 [10]
for more details.
>>> "Monorepo" Workaround
"pkger" tries to mimic the Go standard library and the way how the Go
toolchain handles modules, but is therefore also affected by its
problems and edge cases.
When the `pkger` command is used from the root of a Go module
repository, the directory where the `go.mod` file is located, and there
is no valid Go source file, the command will fail because it internally
uses the same logic like the `list` command of the Go toolchain [11]
(`go list`).
Therefore a "dummy" Go source file may need to be created as a
workaround. This is mostly only required for repositories that use a
"monorepo" layout [12] where one or more `main` packages are placed in a
subdirectory relative to the root directory, e.g. `apps` or `cmd`.
For repositories where the root directory already has a Go package,
that does not contain any build constraints/tags, or uses a "library"
layout, a "dummy" file is probably not needed.
The new `Pkger` method of the "elder" reference implementation handles
the creation of a temporary "dummy" file that gets deleted automatically
when the tasks finishes in order to avoid the need for the user to add
such a file to the repository and commit it into the VCS.
Please see markbates/pkger#109 [13] and markbates/pkger#121 [14] for
more details.
[1]: https://pkg.go.dev/github.com/markbates/pkger
[2]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#GoModule
[3]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/pkger
[4]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gobin#Runner
[5]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner
[6]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#KindGoModule
[7]: golang/go#41191
[8]: https://github.com/golang/go/milestone/145
[9]: https://go.googlesource.com/proposal/+/master/design/draft-embed.md
[10]: markbates/pkger#114
[11]: https://pkg.go.dev/cmd/go/#hdr-List_packages_or_modules
[12]: https://trunkbaseddevelopment.com/monorepos
[13]: markbates/pkger#109
[14]: markbates/pkger#121
Closes GH-52
svengreb
added a commit
to svengreb/wand
that referenced
this issue
Apr 21, 2021
In #52 [1] a task for the github.com/markbates/pkger [2] Go module was added, a tool for embedding static files into Go binaries. The issue also includes the "Official Static Assets Embedding" section which mentions that the task might be removed later on again as soon as Go 1.16 [3] will be released which comes with toolchain support for embedding static assets (files) [4] through the `embed` package [5]. Also see markbates/pkger#114 [6] for more details about the project future of `pkger`. The `pkger` package [7] has been removed and the `//go:embed` directive should be used instead. [1]: #52 [2]: https://pkg.go.dev/github.com/markbates/pkger [3]: https://blog.golang.org/go1.16 [4]: https://golang.org/doc/go1.16#library-embed [5]: https://pkg.go.dev/embed [6]: markbates/pkger#114 [7]: https://pkg.go.dev/github.com/svengreb/wand@v0.4.1/pkg/task/pkger GH-74
svengreb
added a commit
to svengreb/wand
that referenced
this issue
Apr 21, 2021
In #52 [1] a task for the github.com/markbates/pkger [2] Go module was added, a tool for embedding static files into Go binaries. The issue also includes the "Official Static Assets Embedding" section which mentions that the task might be removed later on again as soon as Go 1.16 [3] will be released which comes with toolchain support for embedding static assets (files) [4] through the `embed` package [5]. Also see markbates/pkger#114 [6] for more details about the project future of `pkger`. The `pkger` package [7] has been removed and the `//go:embed` directive should be used instead. [1]: #52 [2]: https://pkg.go.dev/github.com/markbates/pkger [3]: https://blog.golang.org/go1.16 [4]: https://golang.org/doc/go1.16#library-embed [5]: https://pkg.go.dev/embed [6]: markbates/pkger#114 [7]: https://pkg.go.dev/github.com/svengreb/wand@v0.4.1/pkg/task/pkger Closes GH-74
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Recently the Go team presented the following to proposals for the Go language and tool chains.
Both of these, if adopted, would mean that packages such as Pkger, Packr, Statik, etc... will no longer be necessary.
So what does that mean for the future of Pkger? Great question! For that I want to seek community input and help.
Multiple Choice
There are many different ways that Pkger can react to these proposals. I would like to walk through a few different approaches that have come to me, but I would encourage others to leave alternate ideas, proposals, etc...
When I refer to "we" below, I mean not just me, but the community. Projects like Pkger are difficult to maintain. They hit many, many edges cases due to the nature of the problem. To fix those issues, or to implement any of the following proposals, will require help from the community.
So, with that said let's look at the two most obvious choices.
1: Ignore the Proposals
Definitely the simplest approach. Those proposals are draft proposals after all and we aren't obligated to take them into account at all. They, if accepted, would take several cycles to get into the language and command. Therefore a need will exist for third party static embedding tools.
2: Implement the Proposals
Definitely the hardest approach. The Pkger tool and library would be re-architected to implement the proposed changes from the Go team.
By its original design, Pkger, tries to mimic the standard library by creating wrappers around the
osandiopackages, amongst others. The Go team proposals present similar interfaces and concepts.A change like this would break compatibility with the current versions of Pkger. Considerations around module naming, i.e.
markbates/pkger/v2,gobuffalo/pkger,gobuffalo/embed, etc... would need to be decided.In this scenario the Go team would get early feedback of these proposals, and could act as prototype keeping with any changes in the proposals.
Implementing the proposals directly would also mean that Pkger could be swapped out directly with the official implementations, so they get accepted.
Like I said, there are a lot of other approaches that could be taken, and I would love to those as well as feedback on the two I outlined above.
The text was updated successfully, but these errors were encountered: