Make go modules-compatible and switch to mono-repo

[myitcv]

I'm planning to make all of my Go code hosted on Github vgo-compatible and in the process switch to a mono-repo.

Rough plan at the moment is:

• house all myitcv.io/... packages in a mono-repo at https://github.com/myitcv/x
• have multiple modules within the mono-repo, myitcv.io/react being one of them
• drop the _vendor dir, and instead have a reference "cache"/"proxy" in a separate repo that will be used by developers of the code (not users)
• categorise issues/PRs in much the same way as the Go project does, e.g. myitcv.io/react issues would be prefixed by react, react/cmd/reactGen etc

Any one have any thoughts/concerns with this?

cc @mpl

[ghost]

sounds like the future. If a developer is NOT using the new VGO vendoring approach are they going to be up the creek without a paddle though ?

…

On Wed, 4 Apr 2018 at 12:37 Paul Jolly ***@***.***> wrote: I'm planning to make all of my Go code hosted on Github vgo-compatible <https://research.swtch.com/vgo> and in the process switch to a mono-repo. Rough plan at the moment is: - house all myitcv.io/... packages in a mono-repo at https://github.com/myitcv/x - have multiple modules <https://research.swtch.com/vgo-module> within the mono-repo, myitcv.io/react being one of them - drop the _vendor dir, and instead have a reference "cache"/"proxy" in a separate repo that will be used by developers of the code (not users) - categorise issues/PRs in much the same way as the Go project does <https://github.com/golang/go/wiki/HandlingIssues>, e.g. myitcv.io/react issues would be prefixed by react, react/cmd/reactGen etc Any one have any thoughts/concerns with this? cc @mpl <https://github.com/mpl> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#116>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATuCwnywdJpuhVVJobby9q7uHN5z7LGgks5tlKJxgaJpZM4TGj35> .

[myitcv]

For developers of myitcv.io/react, i.e. those who make changes to the myitcv.io/react/... code base; yes, they will be up the creek.

For users of myitcv.io/react, i.e. those who import/use myitcv.io/react/..., everything will continue to work the if you use "old go"; just as it will also work with vgo

Edit: slight tweak to wording from original post.

[mpl]

@myitcv Nice of you to try vgo out and experience the bugs for the rest of us.
I have barely scratched the surface of the vgo posts, so I don't think I have anything interesting to contribute. And if I understand correctly, as a user of myitcv.io/react , nothing much would change for me, right?
What's the pros/cons of keeping your current pkgs separate VS making them a monorepo with modules?

[myitcv]

Nice of you to try vgo out and experience the bugs for the rest of us.

🔨

And if I understand correctly, as a user of myitcv.io/react , nothing much would change for me, right?

Correct. Indeed nothing should change, regardless of whether you use regular go or the new vgo.

The one thing that will change is the remote repo for myitcv.io/react. I'll have a think about how to document that change, how to clean up etc.

[myitcv]

What's the pros/cons of keeping your current pkgs separate VS making them a monorepo with modules?

As a starting point (which I will update over time)

Pros:

• less overhead in terms of maintaining multiple repos, issue lists, CI setups etc

Cons:

• it's a change from where we are today; loss of continuity, history etc (although the history of the old repos will remain intact in the original repo)

[myitcv]

For anyone interested, I'm maintaining notes as I go https://github.com/myitcv/x/wiki/Notes-on-migrating-to-a-Go-mono-repo

For now myitcv.io/react remains in its current location; a couple of blocking issues.

[mpl]

Have you started moving some things around? Because 'dep ensure' (in Perkeep) is whining that myitcv.io/gogenerate should be renamed to myitcv.io or something like that now. Or maybe it's unrelated and I hadn't noticed before?

[myitcv]

Hmm @johanbrandhorst also reported an issue via dep.

And it seems:

go get myitcv.io/react/...

is also now failing.

Apologies - I was trying to make these changes in an entirely non-breaking way. But it seems I failed to get sufficient testing on that. Will revert the changes to get things back and working.

[myitcv]

meant @johanbrandhorst - apologies @jbrandhorst

[myitcv]

Ok - I've just fixed things by repointing myitcv.io/gogenerate back to its original repo.

Also added a safety go get check in myitcv.io/react whilst the vgo migration is in flux.

And added a Travis cron job to build myitcv.io/react@master every day to catch things that fall through the net.

Thanks for the report @mpl and @johanbrandhorst.

[mpl]

Thanks, looks good now. (https://camlistore-review.googlesource.com/c/camlistore/+/15126)

[myitcv]

Quick status update here.

I'm still attempting to make this switch. Understandably given that vgo is "alpha" there are a number of things to be ironed out along the way.

Most critically from @mpl's perspective (indeed anyone who uses myitcv.io/react) is that myitcv.io/react and related GopherJS repos will not move to my vgo-enabled mono repo until GopherJS becomes vgo-aware.

So the initial migration will be "everything non-GopherJS related". Followed, at some time later, by "everything GopherJS related."

This split state of affairs should be fine so long as people are using Go 1.9.7 or Go 1.10.3.

[myitcv]

Ok, this migration has just happened.

Everything should continue to work as before, regardless of your Go version. The only issue you might hit is that the repo "position" within the myitcv.io/... namespace has moved. Whereas there was a repo at myitcv.io/react before, there is now just a single repo at myitcv.io.

Making the mono-repo go module aware will happen on a separate branch for now.

Would appreciate if people could ping here in case of any breakages. Will leave the issue open until I'm satisfied nothing is broken.

[euank]

This does cause dep ensure to consider a previously valid Gopkg.toml invalid since it no longer allows you to refer to myitcv.io/react unqualified.

For example, the following:

[[constraint]]
  name = "myitcv.io/react"
  revision = "bca7c66b77ed8a5b86fb77cff70914c4a7cc3ce5"

will result in

$ dep ensure
The following issues were found in Gopkg.toml:

  ✗ the name for "myitcv.io/react" should be changed to "myitcv.io"

ProjectRoot name validation failed

A user that encounters this error can fix it in one of the following ways:

1. Updating to use myitcv.io as it suggests, including an update to the commit hash.

... and as far as I can tell, that's the only solution. Even explicitly specifying an old source causes it to complain (e.g. the following does not make dep happy).

[[constraint]]
name = "myitcv.io/react"
revision = "bca7c66b77ed8a5b86fb77cff70914c4a7cc3ce5"
source = "https://github.com/myitcv/react"

[euank]

@myitcv To phrase my above comment as a specific question:

Should we be rolling forwards our Gopkg.toml to the new project root configuration, or do you expect to go back and use a different approach given this one does turn out to break some configurations?

Since it has already been a week with no one else popping up on this thread, it seems to me like just rolling forwards with it makes sense, but if you can give an ack that that's the plan, that would be appreciated!

xref perkeep 17526

[myitcv]

@euank apologies for the radio silence.

Should we be rolling forwards our Gopkg.toml to the new project root configuration, or do you expect to go back and use a different approach given this one does turn out to break some configurations?

Yes, please do roll forward.

Apologies for the breakage here, but in the long run it will make things much easier for me, selfishly, to manage a mono repo.

Since it has already been a week with no one else popping up on this thread, it seems to me like just rolling forwards with it makes sense,

Exactly.

I'll leave this issue open for now just in case of any other issues.

cc @mpl

[myitcv]

I haven't heard anything on this so I'm going to close this now.

Not least because I've also just merged module support into my mono repo:

myitcv/x#35

When running Go 1.11 (a build from tip to be specific), the dependency includes experimental Go 1.11 version of GopherJS that is available from https://github.com/myitcv/gopherjs/tree/go1.11 that is itself module-aware.

I'll also be archiving these now-redundant repos over the weekend, moving wikis etc to the mono repo.

[mpl]

@myitcv we've started moving Perkeep to modules as well (https://perkeep-review.googlesource.com/c/perkeep/+/17946). If we have any issues with myitcv/react, we'll probably see them soon.