gonew: templates for new modules

[rsc]

A common request we hear from users is some kind of 'go new' functionality to start a new module with some kind of basic template.

It seems like a tool along these lines should not be limited to templates provided by the Go team: it should be possible for anyone to define templates. That is, templates should just be ordinary modules. At that point the core job of 'go new' is mainly to download a template, change its module path, and drop it in a new directory for editing. There may be more of course, but that's the core.

After literally years of occasionally batting this idea around with @Sajmani, a few months ago I wrote a very trivial text-substitution-based, untested tool at rsc.io/tmp/gonew that did this and mentioned it to a few other people.

Almost immediately, I found another use: a book author emailed asking how best to make book examples available without requiring book readers to puzzle through the vagaries of cloning a git repo. The obvious answer is gonew, which lets people use only Go tools:

$ gonew book.com/mybook-examples

(In this form there's no changing the module path, just clone and drop in a new directory for editing.)

I've also found that form useful for grabbing a copy of some module I am using, to grep through code or to hack up and add to a go.work, like a very lightweight version of github.com/rogpeppe/gohack.

A couple teams at Google were also interested to try gonew for templating. The ServiceWeaver team was interested enough that they put go install rsc.io/tmp/gonew@latest in their getting started template docs. That both confirmed interest and scared me enough (tmp is in the name for a reason!) that I wrote a small, real, tested version at golang.org/x/tools/cmd/gonew, which I've just submitted.

That tool, golang.org/x/tools/cmd/gonew, is experimental in the sense that we aren't sure what will happen next, but it is non-experimental in the sense that we promise to keep the core command line that exists today working. That way, people can confidently post scripts like ServiceWeaver now does:

$ go install golang.org/x/tools/cmd/gonew@latest
$ gonew github.com/ServiceWeaver/template example.com/foo

I've also updated the golang.org/x/example repo to work with gonew, so here are three sample examples:

$ gonew golang.org/x/example/hello
$ gonew golang.org/x/example/helloserver
$ gonew golang.org/x/example/outyet

I am confident there is plenty that template authors will want from 'gonew' beyond this basic functionality. I'd like to hear more about that in the comments. We will try to identify general mechanisms that will satisfy many of the things people need.

Another interesting thing about having these kinds of templates is that, with a clear way to identify them (TBD), you can imagine good support for searching for templates on pkg.go.dev and for instantiating templates in the IDE experience using gopls. If you can think of other ways that the overall Go tooling ecosystem might be changed to incorporate the idea of templates, please post them too.

This is the very very beginning of what could turn out to be an important concept or tool in the Go ecosystem. Perhaps eventually it will graduate to 'go new'. It's a bit unusual for us to publish such a tiny idea, but the core functionality seems useful enough to be worth taking the community's attention to see how it should evolve.

Thanks for any feedback, and please keep the discussion respectful.

[onionj]

And something like gonew with python: https://cookiecutter.readthedocs.io/en/stable/

[bluebrown]

I hacked something like this together in go, a while back https://github.com/bluebrown/blueprint

[Karitham]

Would it be reasonable to make this functionality part of the go mod init process?

Since they both serve bootstrapping process, go mod init example.com/x/abc --template github.com/ServiceWeaver/template could essentially have the same functionality.

[Delta456]

I agree then we don't need a new subcommand for this.

[zephyrtronium]

My concern with this is that tying it to initializing the module, even if only through the command name, might imply some limits on the structures that can be templated. E.g., it would feel awkward for go mod init -template bocchi.rocks/aws-cdk-template bocchi.rocks/aws-cdk-project to put the go.mod in a lambda/ subdirectory, or for it to create a go.work file anywhere.

[earthboundkid]

I wrote my own template tool inspired by Python Cookiecutter that uses txtar format a few years back. https://github.com/carlmjohnson/springerle

I think this makes sense as a thing to build into Go. For example, Node has a thing where npx create foo will automatically run a create-foo package (I forget the precise details of how it works).

Another tool to look at is degit from the creator of Svelte, which clones a git repo while stripping out certain info.

Anyhow, it’s good to hear Alan Donovan is working on GoPL2.0. 😊

[earthboundkid]

Yes, looking into how this works now, it’s most similar to Degit: https://github.com/Rich-Harris/degit

[earthboundkid]

Another idea besides using gofmt or eg to rewrite is to have some system to include or exclude files that use a certain tag. For example if you say -tag client it would include files with //go:build client and exclude //go:build server.

[earthboundkid]

A little more background on my usecase: before I made Springerle, I just had a GitHub repo that was marked as a “template” so I could press the new project from template button in GitHub. I switched to making my own Cookiecutter clone because there was too much manual work in adjusting a new project to be usable. Being able to set the module name would have been a good start, but it is also nice to setup the README and other things to follow the template. The thing I miss about the old setup was that it was a ready to use system and I didn’t need to copy stuff back to the template when I wanted to work out changes to it.

[earthboundkid]

Copier is Python project that has the interesting ability to upgrade a code scaffold. It saves your initial answers in a file, which in theory lets you run the generator again and get the updated version: https://copier.readthedocs.io/en/latest/comparisons/

[JeremyLoy]

I mentioned this elsewhere on the thread, but having project scaffolding that can be programmatically modified over time is the new trend

https://github.com/projen/projen

interestingly enough this project has go bindings…idk how good they are

[GuilhermeCaruso]

I think that regardless of how the new module will be generated (gonew or --template), it would be very interesting to be able to mark the code with some possible variables or notations that would be filled in by flags at the time of initialization of the new module

CLI example: gonew golang.org/x/example/abc --module=github.com/abc --project-name=myNewAbcProject

[heaths]

I wrote a GitHub CLI extension named gh-template that does something like this and, while simplistic, does some things worth considering. For one, it uses text/template to support not just variably replacement, but simple transformations e.g., string tolower, toupper, date functions, etc. If templates have to be modules, a separate .tmpl file as someone recommended above could also work if, indeed, they get published/indexed.

That my extension also clones a GitHub repo, of course, is completely orthogonal and not necessary for a pure go template tool.

[kylelemons]

We have a GitHub template for Go repos, and we're taking a different approach from the Python predecessors which used template files that included moustache-style templates. Those were very hard to get right, and fixing one feature usually broke another. So, our Go template repos are fully functioning services with carefully-selected variable names and file names so that we can use find and replace on them. We don't support optional features or anything right now though. We started with, and haven't really gotten much better than, a dumb go run ./scripts/customize_template_repo script that asks for a handful of variables including the repo name, the service name, the team name, etc. It then does a find and replace (on both file contents and file and directory names) based on your answers. This allows us to customize your gRPC service's .proto and generated code and such all in one fell swoop. So far this has been easier to keep fresh and working than the Python alternative, but it would be really nice if there were a structured way to do this that an official tool could understand.

[earthboundkid]

Yeah, having myself written a tool that uses {{ Go templates, I think the big disadvantage of those kind of tools is that they aren't valid syntax until after you run the transform, so you're in limbo land when you edit the template. I think a good workaround would be to have valid Go programs be the starting point, and then run a series of gofmt or eg scripts over it. The templating could just affect that script file and not the Go code itself.

[heaths]

That has been a source of frustration with my gh extension as well. I get CI failure notifications within seconds of running the command because it already cloned the GitHub repo and, as you pointed out, the template formatting makes it invalid so go fails immediately.

Alternatively, rather than text/template or something of that nature, perhaps just a set of source/replacements with some level of regex support, much like JavaScript's replace function supports. Or, in essence, like some l18n tools.

[isgj]

Can you give examples of what you would use these variables for?

Whenever we create a new http server we generate it with cookiecutter. Other than the usual project name, description, department..., we specify also if the project will use a sql db, cache type, stream service ect. The generated project will have only the required services configured, also the k8s files will have all setup only for the configured services.

[vedranvuk]

Here's something I've been writing last couple of days: https://github.com/vedranvuk/boil
The focus is much more on the parametrization of template files. Still very raw though.

[leaanthony]

I wrote a similar tool a while back for Wails. Would love to see this as part of the standard tooling: https://github.com/leaanthony/gosod

[JeremyLoy]

Neat idea!

a few thoughts:

versioning for templates with @ similar to the go get syntax would be great to pin templates keeping scripts that utilize go new from breaking.

Eg go new example.com/foo@v1.2.3.

after looking at the package itself, it already supports versioning! Just the projen comment then

——

have you heard of Projen before ? The TLDR is that it is a tool that eschews static one-time templates in favor of programmatic updates over the life of a project.

imagine your project was bootstrapped with v1.1 of example.com/fooTemplate. With projen, you can upgrade to v1.2 in an automated fashion. This could involve adding, deleting, or editing files.

I don’t think it’s necessarily something the go tool chain would support natively, but I just figured I’d mention it as it’s in the same orbit as templating, but takes the devils advocate approach of “templates are bad” (their words, not mine).

just a thought!

[cheikh2shift]

Is it possible to contribute to this tool? I was thinking of having template files that can be populated with user passed configuration variables (when the command is executed) and transpiled into Go files. Kind of like helm charts.

[adamdecaf]

We have a project called templater which uses Go templates to render Go source files. This works well enough but we often run into issues where a file had additional source code added to it (init db, repositories, event handlers, etc) so re-templating can lead to undoing partial hunks of files. Templater has a config file to render common elements and enable/disable features (as we upgrade patterns, libraries used, etc) across ~70 repositories.

Often we are supporting old/new code patterns and assisting developers in migrating their code.

    {{- if .Config.Kafka.Enabled}}
    {{- if .Config.Kafka.UseEventing}}
    Eventing eventing.Client
    {{else}}
    Producer producer.Producer
    Consumer consumer.ConsumerContext
    {{- end}}
    {{- end}}

gonew would be great to offer a generic configuration section and options to render complicated Go templates. We've got CRUD API templates, event handlers, the go-service itself, and more.

templater config file

ProjectPath: "."
Project:
  ProjectID: "file-saver"
  OrgID: "moovfinancial"
  ProjectName: "File Saver"
  Description: |
    Download files from remote locations to store, handoff, and organize within Moov.
  PrimaryBranch: "master"
  CodeOwners: "@moovfinancial/bank-rails"
DockerCompose:
  Enabled: true
SQL: true
MySQL:
  Enabled: true
  Port: 3306
Spanner:
  Enabled: false
Kafka:
  Enabled: true
  UseEventing: true
  Topics:
    - events.wrath.v1:1:1
Templates:
  GoService:
    ServicePort: 8691
    HealthPort: 9691
  GoGithubActionsPrivate:
    - Executable: file-saver

API controller template

// {{.GeneratedTag}}

package {{.Template.PackageID}}

import (
    "encoding/json"
    "net/http"
    
    "github.com/gorilla/mux"
    "github.com/moov-io/base/log"
    "github.com/moovfinancial/errors"
    "github.com/moovfinancial/go-zero-trust/v2/pkg/middleware"
    
    "{{.ImportPath}}/pkg/api"
)

type {{.Template.ModelName}}Controller interface {
    AppendRoutes(router *mux.Router) *mux.Router
}

func New{{.Template.ModelName}}Controller(logger log.Logger, service {{.Template.ModelName}}Service) {{.Template.ModelName}}Controller {
    return &{{.Template.ModelID}}Controller{
        logger:  logger,
        service: service,
    }
}

type {{.Template.ModelID}}Controller struct {
    logger  log.Logger
    service {{.Template.ModelName}}Service
}

func (c {{.Template.ModelID}}Controller) AppendRoutes(router *mux.Router) *mux.Router {
    router.
        Name("{{.Template.ModelName}}.create").
        Methods("POST").
        Path("{{.Template.BasePath}}{{.Template.ResourcePath}}").
        HandlerFunc(c.create)

    router.
        Name("{{.Template.ModelName}}.list").
        Methods("GET").
        Path("{{.Template.BasePath}}{{.Template.ResourcePath}}").
        HandlerFunc(c.list)

    router.
        Name("{{.Template.ModelName}}.get").
        Methods("GET").
        Path("{{.Template.BasePath}}{{.Template.ResourcePath}}/{ {{- .Template.ModelID}}ID}").
        HandlerFunc(c.get)

    {{if (eq .Template.NoUpdate false)}}
    router.
        Name("{{.Template.ModelName}}.update").
        Methods("PUT").
        Path("{{.Template.BasePath}}{{.Template.ResourcePath}}/{ {{- .Template.ModelID}}ID}").
        HandlerFunc(c.update)
    {{- end}}

    {{if (eq .Template.NoDelete false)}}
    router.
        Name("{{.Template.ModelName}}.delete").
        Methods("DELETE").
        Path("{{.Template.BasePath}}{{.Template.ResourcePath}}/{ {{- .Template.ModelID}}ID}").
        HandlerFunc(c.delete)
    {{- end}}

    return router
}

func (c *{{.Template.ModelID}}Controller) create(w http.ResponseWriter, r *http.Request) {
    middleware.WithClaimsFromRequest(w, r, func(claims middleware.TrustedClaims) {
        accountID := mux.Vars(r)["accountID"]

        create := Create{{.Template.ModelName}}{}
        if err := json.NewDecoder(r.Body).Decode(&create); err != nil {
            err = errors.Flag(errors.Wrap(err, "decoding"), errors.NotSerializable)
            api.ErrorResponse(w, err, c.logger)
            return
        }
        
        result, err := c.service.Create(r.Context(), claims, accountID, create)
        if err != nil {
            api.ErrorResponse(w, err, c.logger)
            return
        }

        api.JsonResponse(w, result)
    })
}

func (c *{{.Template.ModelID}}Controller) list(w http.ResponseWriter, r *http.Request) {
    middleware.WithClaimsFromRequest(w, r, func(claims middleware.TrustedClaims) {
        accountID := mux.Vars(r)["accountID"]
        
        result, err := c.service.List(r.Context(), claims, accountID)
        if err != nil {
            api.ErrorResponse(w, err, c.logger)
            return
        }

        api.JsonResponse(w, result)
    })
}

func (c *{{.Template.ModelID}}Controller) get(w http.ResponseWriter, r *http.Request) {
    middleware.WithClaimsFromRequest(w, r, func(claims middleware.TrustedClaims) {
        params := mux.Vars(r)
        accountID := params["accountID"]
        {{.Template.ModelID}}ID := params["{{.Template.ModelID}}ID"]
        
        result, err := c.service.Get(r.Context(), claims, accountID, {{.Template.ModelID}}ID)
        if err != nil {
            api.ErrorResponse(w, err, c.logger)
            return
        }

        api.JsonResponse(w, result)
    })
}

{{if (eq .Template.NoUpdate false)}}
func (c *{{.Template.ModelID}}Controller) update(w http.ResponseWriter, r *http.Request) {
    middleware.WithClaimsFromRequest(w, r, func(claims middleware.TrustedClaims) {
        params := mux.Vars(r)
        accountID := params["accountID"] 
        {{.Template.ModelID}}ID := params["{{.Template.ModelID}}ID"]

        update := Update{{.Template.ModelName}}{}
        if err := json.NewDecoder(r.Body).Decode(&update); err != nil {
            err = errors.Flag(errors.Wrap(err, "decoding"), errors.NotSerializable)
            api.ErrorResponse(w, err, c.logger)
            return
        }

        result, err := c.service.Update(r.Context(), claims, accountID, {{.Template.ModelID}}ID, update)
        if err != nil {
            api.ErrorResponse(w, err, c.logger)
            return
        }

        api.JsonResponse(w, result)
    })
}
{{- end}}

{{if (eq .Template.NoDelete false)}}
func (c *{{.Template.ModelID}}Controller) delete(w http.ResponseWriter, r *http.Request) {
    middleware.WithClaimsFromRequest(w, r, func(claims middleware.TrustedClaims) {
        params := mux.Vars(r)
        accountID := params["accountID"] 
        {{.Template.ModelID}}ID := params["{{.Template.ModelID}}ID"]

        err := c.service.Delete(r.Context(), claims, accountID, {{.Template.ModelID}}ID)
        if err != nil {
            api.ErrorResponse(w, err, c.logger)
            return
        }

        w.WriteHeader(http.StatusNoContent)
    })
}
{{- end}}

[madflojo]

I think a good call out from this is not only is templating for the contents of files important, but the inclusion/exclusion of files based on config/command-line flags is another key feature.

[chr1shung]

Is you templater project open-sourced ?
I have a similar requirement recently and want to have some good reference.

[naivary]

One use case that I can think of is to have a standard header in every file like for example the used license.

[nomad-software]

Aren't you simply enabling something that you criticised here: golang-standards/project-layout#117

I'm already seeing comments online about how this will enable a global standard for all projects, etc. Which is horrifying.

[JeremyLoy]

The criticism of that repo was about its content and the way it presented itself, not the fact that it was a template.

Adding tooling support for templates is completely separate from enforcing global standards for what a project should be structured. Even if there were templates added to the standard library, you’re not forced to use them. And nothing stops you from opening a PR to modify it should it need updating

[jimmyfrasche]

Since this goes through the module system it will not contain files/directories that do not end up in the mod zip, correct? If there's a way to tag modules as templates it would probably be handy to use that mechanism to also specify files/directories to keep for readmes or anything that you'd want the template user to have access to but not want to embed

[thediveo]

pretty much all ends up in the "modball", because a mod is something that go can download, such as a git repository. There are gotchas with respect to multi platform and special files and links, but that hopefully doesn't concern the templates here.

[Merovius]

@thediveo A module is not a git repository. A module is distributed as a zip file. The go tool can transparently synthesize that from a git repository, but the canonical distribution mechanism is the zip file.

The full constraints are listed here. Of particular relevance to a templating tool seems the restrictions that sub directories containing modules are excluded and that vendor is excluded. That seems to, in particular, make it impossible for this to check out a workspace.

[earthboundkid]

What would be the usecase for checking out a workspace? Would you really want to clone and modify multiple modules at once outside of a teaching context?

[thediveo]

I didn't say that a modball is a git repo, there's a clear semantic difference conveyed by "such as" but obviously not understood by some. Anyway, thanks for the clarification.

[Merovius]

@carlmjohnson When skimming this discussion, I've seen mentions of go.work and sort of extrapolated a situation where a small team might have a template to put all the modules they maintain in a workspace. I also know that gokrazy operates on multiple modules as a unit. It didn't seem far-fetched. Though on looking again, I might have misread or read too much into these mentions. If it's not interesting, then that's fine.

@thediveo If I misunderstood you, feel free to correct me - but please do not be passive-aggressive when doing so.

[jimmyfrasche]

A common use for a scaffolding tool is to fill in the boilerplate for an extension to a framework where an extension can either be a third party shareable module or a project-specific subdirectory.

This tool would work for either situation if it's okay for the extensions to be sub-modules or dependencies. That's certainly doable but I imagine not everyone would want to deal with that.

It would be useful if there were a way to "adopt" the template as part of the current module. The easy part would be rm go.mod and the more involved part would be first merging the template's dependencies into the adopting go.mod.

For more context, I'm specifically thinking of the sort of thing I mentioned in a previous reply: #61669 (reply in thread)

[empire]

Thank you for bringing this discussion up. We have heard that Google uses a mono repo and it's very impressive. However, due to the lack of tools and expertise, we as an external company can't do the same. Therefore, we have to duplicate some parts of our logic for each project. To handle this, we decided to set up a separate repo and copy common parts of code, such as working with NATS and Redis, in that repo and use it in our projects. With this approach, the go get command will help us to eliminate most of the duplicate code and import it into our projects.

But some issues can't be resolved with this approach and we still need to duplicate some logic for each project. For example, if we want to support Redis in a project, we have to manually add the initialization and shutdown parts, call the monitoring parts, and add the configuration and deployment files as well. The problem gets worse when we need to refactor some parts of our common module and replace the old code with a better alternative.

When I read this discussion, based on this experience, I think we are not alone and maybe this kind of issue can lead to better tools.

Let me think aloud, if the go get command modifies the external directory (~/.local/share/go/pkg/mod/ directory), the gonew command will modify and weave local files and directories. And if we have a standard tool with developer-friendly API, it can change the way we structure our projects. All of this can reduce some more manual work from us.

$ gonew example.com/ourteam/redis

$ tree . # if the gonew command modify the local files.
.
├── docker-compose.yaml
├── .gitlab
│   └── ci
│       └── templates
│           └── test.redis.yaml
├── src
│   ├── config
│   │   └── defaults.redis.go
│   │   └── redis.go
│   └── app
│       └── app.redis.go
└── deployment
    ├── templates
    │   └── config-map.redis.yaml
    └── values.redis.yaml

$ tree . # if the command can modify and weave the local files.
.
├── docker-compose.yaml
├── .gitlab
│   └── ci
│       └── templates
│           └── test.yaml
├── src
│   ├── config
│   │   └── defaults.go
│   └── app
│       └── app.go
└── deployment
    ├── templates
    │   └── config-map.yaml
    └── values.yaml


$ go generate ./...
$ go mod tidy

I apologize for the long description but I wanted to bring up the problem to find a better approach.
Thanks.

[zhb127]

gonew can keep template files default permissions?

this is my test template: https://github.com/zhb127/gonew-test
I have permission to exec my script:

./helloserver/scripts/init-local-dev.sh

start local db...
start local redis...
The local development environment is initialized.

create project from template:

gonew github.com/zhb127/gonew-test/helloserver github.com/zhb127/demo

gonew: initialized github.com/zhb127/demo in ./demo

but:

cd demo
./scripts/init-local-dev.sh

zsh: permission denied: ./scripts/init-local-dev.sh

[earthboundkid]

For my template tool, I didn't want to add a mode for setting file permissions, so I just hard coded that files ending in .sh are chmod +x. It's a tricky UI question.

[szabba]

It's not 100% clear what that should do on Windows. Both git and Git Bash had to come up with some answers thought so there's prior work.

[timrulebosch]

@rsc this could be interesting, if you can keep it simple.

Previously I've used Copier, its a templating approach which, while powerful, has the limitation that the template itself is not valid code. So, all the toolings you might use for QA and doc don't work. So, in that regard, I find gonew to be very interesting.

I would (like to) use this gonew to build projects based on existing patterns. Pull in a CLI implementation, then a service (and tests), perhaps a doc layout (which is compatible with our doc systems) and so on.

[timrulebosch]

Just tried this, pulling in a few of the exiting templates into a folder. The only observation is that gonew only works for go modules, so you can't just use it to pull in some plain content.

This is the workflow I might like to use:

mkdir greetings
cd greetings
go mod init example.com/greetings

gonew golang.org/x/example/helloserver example.com/greetings/internal/myserver
gonew golang.org/x/example/cli example.com/greetings/cmd/server
gonew golang.org/x/example/rest example.com/greetings/rest/server

# This bit did not work, and perhaps that is OK. You can also CURL such content.
gonew github.com/GoogleCloudPlatform/osconfig/THIRD_PARTY_LICENSES/cloud.google.com/go example.com/greetings/licenses/go

[earthboundkid]

Hugo uses go.mod to distribute its theme files, so there is some precedent for using go.mod for content. It is a little weird though.

[terryosayawe-cpi]

Really cool, I need to give it a try.

[trentrosenbaum]

It is really good to see gonew working over this initial interaction. @rsc am I right in thinking that the modules being used to create a new module are meant to be fully functioning standalone and not templated themselves? I see translating an example project into a templated syntax to be a barrier to maintenance. I see the original module as being a fully working standalone project and not having to be adapted for the gonew tool.

It feels that working over the go.mod and the directory name would give enough information to adapt references to internal imports that might be in the module.

It feels like in the go module landscape all modules should rename useable and not a combination of traditional modules and templated sytntax modules.

Hopefully I have been able to explain myself. In short I think gonew is a positive start 😄

[jimmyfrasche]

go new

1. extracts the module files locally
2. renames the module

Both of these are valuable on their own. Renaming a module obviously so. Extracting files less so, but it would be useful.† If those were separate commands then go new as it is would just be a helper to run those two in the correct order with the correct args.

† Consider a web framework that by convention embeds all its template files on an exported Templates variable and has a naming system that lets you override a dependency's template file. Handy on its own but you usually want to start from the original template instead of from scratch. It's already in the module cache so being able to ask go to grab just that file from the module being used is simpler than going to github and finding the file at the correct version.

requests for extensions so far generally fall into two categories

1. templating
2. refactoring

I think it's unlikely that templating will be included. Some basic package renaming would be nice but also seems unlikely given how awkward to specify this would be.

In both cases you could use some tool after you call go new to get the desired results but you may not know that you have to so there would probably be some kind of "read me first" file though you may not know to look for it. If the file name were standardized, maybe go new could cat it before exiting.

Alternately, while generally go doesn't run arbitrary code, maybe this could be the exception as it's meant to be run once by a developer. If there were a standard go tool/generate command name to run at the end it could collect any additional information and run any templating/refactoring.

[sysulq]

Suddenly seeing this discussion, it is very interesting that our team also developed a similar tool in the past two years and used it extensively internally. This greatly contributes to improving development efficiency, especially in the case of microservice development.

It's worth mentioning that we also supported caching and generating project structure based on module paths. However, the caching mechanism implemented in "gonew" is more elegant, we will also consider learning from it and optimizing our own implementation.

Running this command below would generate a project in github.com/douyu/exampleapp directory with github.com/douyu/exampleapp module name.

jupiter new github.com/douyu/jupiter-layout github.com/douyu/exampleapp

Once gonew is merged into the official Go repository, we will immediately replace and start using it. 😄

Refer:

• https://github.com/douyu/jupiter/tree/master#quick-start
• https://github.com/douyu/jupiter/blob/master/pkg/core/cmd/cmd_new.go

[anilsenay]

I was excited to use gonew but I had a issue that I want to share it with you.
I have a repository in bitbucket.org/company/project-name but in project module name is company.com/project-name which is using in imports and go.mod file. So when I use gonew bitbucket.org/company/project-name new-project, it will not replace imports as new-project because it use module name as bitbucket.org/company/project-name and try to replace it with new one.

I made a quick solution for myself by cloning gonew and adding new command line flags in this repo but maybe you can think of this issue and find a better solution.

[jeffmccune]

To echo others who have requested input parameters, it'd be great if I as the template author could tell gonew to unmarshal yaml or json provided by the template user into a struct accessible from the templates. The use case is to have the user provide values like a hostname or url.

[atdiar]

A quick question/feedback as I am currently implementing a project launcher CLI which is similar to gonew.
(Note: I haven't tried with gonew yet as I am waiting for the tool to get merged in)
Thinking that it might be pertinent here.

When I create a new project that will live in a workspace, seems like I need to update the go.work file before I build the project, otherwise I get:

Successfully initialized go module: github.com/atdiar/zuitest
current directory is contained in a module that is not one of the workspace modules listed in go.work. You can add the module to the workspace using:
        go work use ..

The first line is a custom error message, the second occurs because I am trying to build a main.go into a main.wasm

I probably have to check whether I'm in a workspace and if so, update the workspace programmatically, unless there is a better way?

cc @bcmills

[zephyrtronium]

FWIW, if you're already certain you have a Go toolchain, then detecting that you're currently in a workspace is just checking whether the output of go env GOWORK is non-empty. It also seems reasonable to simply always run go work use . and discard the error message.

Either way, generally speaking, it's preferable to invoke the go tool to manipulate the Go environment, rather than trying to reimplement its behavior.

[atdiar]

Good look! So I guess the remaining question is whether gonew should do that, I don't think that currently the code is workspace aware.

[atdiar]

Oh and still on the topic of integration with workspaces, maybe there is a way to update the workspace once a module is removed?

Like a go work tidy?

[pellared]

If the tool would not change, then I would consider renaming the tool to gofork as it is a neat tool for making forks of existing repositories so that it they be easily published as separate Go modules.

Without the ability to pass anything more than dstmod as input, I think that calling the solution as "template" is an overuse.

I think that with gonew you should be able to use templates that would allow you to quickly bootstrap repositories like https://github.com/golang/pkgsite (I think it could be a good test subject).

Still thanks for your great work 👍

[bcmills]

gofork would be misleading, I think. Instantiating a template does not preserve the commit history leading up to that template, wherease a “fork” of a repo usually does include the history.

[pellared]

What about repositories which have multiple Go modules?

For instance it is quite common to follow https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module. You can take a look at https://github.com/goyek/goyek which has tools as a separate Go module. This pattern is especially important for libraries so that its users do NOT depend on the libraries' build tools. However, it could also profit for applications (e.g. CLI apps) as it should speed up the time of building the applications using go install. Here is an example GitHub repository template that creates two Go modules which is good for libraries https://github.com/goyek/template.

[aelse]

I want to be positive about experiments and contributions to the go ecosystem. I realise that we need to cater to developers of all levels and sometimes a problem that seems trivial to someone familiar with go may be a challenge to a developer just arriving from another language. That said, I don't see what problem gonew solves in its current state.

Looking at the gonew source it's just replacing the old module name with the new name in source code. This is not a challenging problem to solve for someone who wants to copy an existing repo to a new namespace.

When i saw the announcement I had hoped for something similar to cookiecutter, which i've been using for years to help bootstrap new projects. This has been incredibly useful to my teams in crafting and reusing project templates for go and other languages.

I think the current version of gonew is effectively a conversation starter (and this discussion is the conversation!). What I hope is that gonew becomes something similar to what we're doing with cookiecutter so that we have a go native tool for bootstrapping projects and a common language for defining project templates. This would seem to be something easily built on top of text/template in conjunction with a project template specification.

Is this the direction we're likely to head?

[ghasemloo]

I think this can be useful. It can save people coding time if there are maintained standard code templates for common things like a web server, etc. similar to how some Google-internal server frameworks create the initial files.

One con to keep in mind is that making things easier to write is not always the best thing. Copying over a large amount of boilerplate code over and over without understanding and real need can make things less maintainable over time (if there is not a refactoring to some dependency that is maintain, the folks who copied the code would need to maintain the copy).

So it might be good to think about the ecosystem and maintainability impact of having such a tool.

[mattventura]

I am looking at using this in the context of a monorepo. In this case, the templates and the destination would be in the same repo. Is there a good way to use this tool to reference a local path as the module template? Or am I over-complicating something that should just be a simple directory copy?