Proposing to form a monorepo

[joyeecheung]

This idea started in #1 (comment), and seems to receive some popularity within our team, so I think we can open an issue dedicated to this topic.

The current proposal

1. Put all automation related projects into a single repo. We can keep the existing git history with git subtrees.
2. At the moment the most viable solution for managing a monorepo seems to be Lerna, there is a WIP using it in Add branch-diff #9 to move branch-diff into this repo.

The upside of a monorepo

(...that I can think of)

1. More code-sharing, we don't have to define a formal API and can test every dependencies to make sure one change in a tool does not break others
2. More visibility, can get more people involved into the maintenance of these projects
3. Easier to uniform the practices and environment setup for these tools, including contributing, governance, coding styles, Github API authentication, Jenkins token setup
4. Easier to bundle them into one toolkit and distribute/update together

The downside of a monorepo

1. Complicates release and commit processes, we are very likely to need to rely on a tool for that
2. More messages for the collaborators because they all live in one repo (though I don't think this would be a big problem considering the sum of existing traffic is not that terrifying)
3. Need to figure out how to get infrastructure e.g. CI working
4. Would need to audit/transfer existing issues/PRs of these projects.

Things to discuss if we do adopt the monorepo

1. Should the github bot be placed there? It's more like a server-side application, not an one-off command. Proposing to form a monorepo #12 (comment)
2. Should we use the automation repo, or another repo, to place the code? Proposing to form a monorepo #12 (comment)
3. How should we handle the transition? Maybe progressively adding them to the monorepo instead of doing it at once to avoid premature decisions. Proposing to form a monorepo #12 (comment)

Please indicate your approval/objection towards this idea in this thread, and discuss about how we are going to structure the monorepo if this idea sounds good to you.

[joyeecheung]

An important thing to discuss is whether we want the code placed in this repo, or another one. Personally I am in favor of a separate repo, because it helps us triage notifications better. We can discuss about high-level things here, and discuss about the actual code changes and bug reports/feature requests in another repo (similar to how TSC repo and nodejs/node repo work).

[MylesBorins]

I have two distinct and opposing feelings about this

On one hand I am concerned about trying to take all of these tools and prematurely move them all in the one place.

On the other hand I've been very interested in exploring some of the benefits of working with mono repos. A number of fairly popular project (e.g. babel) are maintained using lerna and this could act as a great smoke test the project can do for that workflow.

I think slowly merging a number of tools into the monorepo would be a great way to do this!

If we are going to start with branch-diff we should also include changelog-maker and commit-stream. The three are deeply connected and there is currently pain in working on them as you need to do some npm link gymnastics. I've commented as such in the issue.

[Tiriel]

I already said it, I like this monorepo idea.

I get what you say about having two repos and I tend to agree with you. But just for the sake of the argument, do we currently need to have the distinction ? Do we have enough meta discussions on one side and enough technical issues on each tool?

As for the progressive approach to the monorepo, I'm totally for it. It's a big change in the way we manage the tools, we should go carefully.

[alopezsanchez]

Personally I am in favor of a separate repo, because it helps us triage notifications better.

@joyeecheung I'm with you. Keeping them separated makes the management easier (in my opinion). But as I said in #11, we should consider to have common resources in a "centralized" repositorie.

Diversity of opinions, it's cool.

[phillipj]

Put all automation related projects into a single repo.

This is purely for projects published as npm packages, right? In other words, the github-bot would still be in its own repo?

[joyeecheung]

@phillipj This is something that needs discussion as well: do we want to put everything in the monorepo, or just the npm packages?

Personally I think we can just put them all in one place, because the upside 1-3 in the OP would apply. As for 4(distribute/update), we can ignore the bot when doing releases anyway so could not see any more harm there.

[joyeecheung]

..which reminds me of another downside:

4. Would need to audit/transfer existing issues/PRs of these projects.

but that's not an impossible thing to do anyway.

[phillipj]

The first challenge that comes to mind for the github-bot, is that it automatically deploys to production when commits lands in the master branch.

I'm sure it's technically possible to figure out whether or not those commits has any effect on the github-bot code before actually triggering the deployment process. Though it's not obvious to me that putting the github-bot into a monorepo of mostly packages, is worth the complexity added elsewhere for the github-bot in particular.

To me packages and running processes are very different in nature, maybe that could be a reason to keep them in separate repos?

[joyeecheung]

@phillipj Hmm..yes, technically github-bot runs (or is intended to run) on the server while other project runs on dev machines. I think that is a very valid argument.

[Tiriel]

After a bit thinking, having nodejs/automation for triage and meta discussions and another repo like nodejs/automation-tools for the monorepo does seem a good idea (despite my previous message). Keeping in mind that all tools won't be migrated at once in the monorepo, and some won't ever be migrated there.

[gibfahn]

I think it definitely makes sense to leave the github-bot discussion until later.

I actually like node-core-utils, for the name of the monorepo. Seems like the most exact description.

Not using automation seems reasonable as a starting point, we can always merge later.

[joyeecheung]

A bit off-topic: I think node-core-utils is close to getting transferred into the organization now that we have enough test coverage, a few active collaborators, and an al-right README..

[Tiriel]

I'm all for transferring node-core-utils.

As for the rewrite of the main issue, I'll give clarify my positions here:
Monorepo: Yay!
Using nodejs/automation or another repo for code: Another repo for code
github-bot: should have its owne repo, it's a special case
Progressive transition? Yay!

[Trott]

Unarchived this repo so I could close all the PRs and issues. Will re-archive when I'm done.