use pub workspaces

[jakemac53]

There is one weird quirk here - our usual way of pinning package versions doesn't work, because we are pinning a version which doesn't yet exist, using the non-wip number. I resolved that by adding version constraints like this instead >=x.x.x-wip <x.x.(x+1). But, it is a bit weird. When creating a release, we can change these to exact versions, because they will then match the version in the repo. But, we might not remember to. WDYT @natebosch ?

Also cc @sigurdm and @jonasfj in case they have a different solution to suggest.

[jonasfj]

• Don't pin versions 🤣
• If there are reasons for pinning (reconsider them, like could you just merge tightly coupled packages into a single package).
• If that fails, maybe reconsider -wip, like is that a good convention?
• If that doesn't work, consider using dependency_overrides to avoid problems with version constraints. Of course this means constraints won't be checked.
• If that unacceptable: the consider accepting that maybe this doesn't have a good solution 🙈

---

I don't necessarily have an answer here. But I think it's worthwhile questioning some of the decisions that led us here.

If we want to check version constraints in workspace (and we do) using a -wip is going to be a problem.
It's also an annoying convention. It might be possible to do smarter things, like maybe keep a WIP entry in the CHANGELOG, and only update version number there when publishing...

Or we could use GitHub Actions to bump version numbers and publish using workflow_dispatch. If we wanted to automate some of the tedious manual bits.

I don't have an answer off the top of my head, I'm just not 100% sure we shouldn't reconsider our workflow / conventions around -wip.

[natebosch]

• Don't pin versions 🤣

It's not clear how much effort that would take. We made a lot of progress in decoupling these packages, but not enough to use a looser dependency yet.

• like could you just merge tightly coupled packages into a single package

We split the packages to allow some use cases to have pruned transitive deps, so we can't merge them without changes in pub.

• maybe reconsider -wip, like is that a good convention?

It's our documented convention. https://github.com/dart-lang/sdk/blob/main/docs/External-Package-Maintenance.md#making-a-change

I find it's convenient to have a quick place to check whether there are unpublished changes, and reliable enough even without enforcement. Without this we have more often made mistakes like double-incrementing the version number with a single publish.

If we adopt tooling which does the edits for us we could implement it to check for published versions and warn about double incrementing maybe. I'm uneasy about dropping the practice without tooling though.

[jakemac53]

• Don't pin versions 🤣

We might consider this at this point, but it wouldn't solve the problem. Any time we have a new feature in one package, that isn't released yet, we will have to use one of these -wip constraints.

• If there are reasons for pinning (reconsider them, like could you just merge tightly coupled packages into a single package).

It used to be one giant package, but was split out for good reasons (fewer transitive deps). Flutter only depends on the smaller (test_api) package for the most part (at least in terms of non-dev dependencies), which means a lot fewer pinned deps in total.

• If that fails, maybe reconsider -wip, like is that a good convention?

Yes, it is a good convention, and it is the general established convention for dart-lang packages.

Packages are never published with that version, it just indicates "there are unpublished changes, and based on those this is the next anticipated version number". It makes it easy to tell at a glance which packages have unpublished changes and also allows you to evolve the next pending release version along with the changes actually being made.

• If that doesn't work, consider using dependency_overrides to avoid problems with version constraints. Of course this means constraints won't be checked.

Workspaces don't allow dependency_overrides of the packages in the workspace (this is what we previously did).

• If that unacceptable: the consider accepting that maybe this doesn't have a good solution 🙈

Yeah, the solution I came up with is sort of ok, not ideal but works. I prefer it to dependency overrides, because we can at least ensure our version constraints are compatible. We have before forgotten to bump pinned versions because of the overrides.

I don't necessarily have an answer here. But I think it's worthwhile questioning some of the decisions that led us here.

If we want to check version constraints in workspace (and we do) using a -wip is going to be a problem. It's also an annoying convention. It might be possible to do smarter things, like maybe keep a WIP entry in the CHANGELOG, and only update version number there when publishing...

We could do that but it feels wrong to have the changelog/pubspec out of sync. It is a work in progress version, and so having that in the pubspec seems correct to me. It is also error prone to not do that. We want to be able to update the pubspec in other workspace packages to depend on WIP features (see last comment below).

Or we could use GitHub Actions to bump version numbers and publish using workflow_dispatch. If we wanted to automate some of the tedious manual bits.

Maybe, not sure exactly what this would look like.

I don't have an answer off the top of my head, I'm just not 100% sure we shouldn't reconsider our workflow / conventions around -wip.

Yeah I am not suggesting changing how the version solve works, or hard coding knowledge about -wip. But I do think it is worth considering even just the "new feature" use case.

• You are developing a feature that stretches across multiple packages in the repo
• You don't want to release any package until it is working end to end
• You want to put a proper constraint on the packages in the same repo, to only allow the (unpublished) version, instead of relying on remembering to bump it later on when you are ready to publish.

I don't see a good workflow for this today. I don't have a solution to pitch yet either, but it might be worth thinking about.

[natebosch]

cc @devoncarew - WDYT about adding a check in firehose for a package which has a more than +1 increment in any semver position over what is published? I can imagine edge cases with retracted packages. I'm not sure if there are other reasons to have a version bump that skips a number.

[jakemac53]

Oh, I see Nate beat me too it :).

[jakemac53]

Going to hold off just a bit on landing this, we have some changes that would be good to get out prior to dropping new feature support for anything <=3.4.

Nate also brought up the idea via chat of just pinning the -wip version exactly, which totally works and also requires us to update the constraint when we actually do a release, which is good. Pretty happy with that solution.

[devoncarew]

cc @devoncarew - WDYT about adding a check in firehose for a package which has a more than +1 increment in any semver position over what is published? I can imagine edge cases with retracted packages. I'm not sure if there are other reasons to have a version bump that skips a number.

I think adding additional safety checks would be valuable. If they are checks that are mostly always things we want to enforce - but that have infrequent, legitimate, violations, we can either handle that via:

• the occasional manual publish (not ideal but feasible)
• having a category of checks that can be disabled via adding a label on the PR; allow people to opt-out if they think it doesn't allow for a specific change