Multiple packages with the name my_package found in the workspace

[btrautmann]

We're running into an issue where melos:

1. Hangs during bootstrap, causing the mirror directory at .dart_tool/melos_tool to remain populated (but no logs indicate why it's hanging. It hangs for > 10 minutes at which point CI times out)
2. When re-invoked, it is attempting to include packages from the mirror directory at .dart_tool/melos_tool when running commands, such as bootstrap, which leads to the following error:

melos.yaml: Multiple packages with the name `my_package` found in the workspace, which is unsupported.
To fix this problem, consider renaming your packages to have a unique name.

The packages that caused the problem are:
- my_package at .dart_tool/melos_tool/my_package
- my_package at my_package

It (seemingly) randomly started happening to us on 1.0.0-dev3 but bumping to 1.2.0 did not fix the issue. It's not fully reproducible locally (even a fresh clone of the repo will not reproduce it) but is happening consistently on CI.

Wondering if there's anything that would cause this that we may have unknowingly done, or if there's any guidance in debugging you might be able to provide.

[btrautmann]

I'm able to reproduce this reliably on our CI machine and by adding logs to a locally built melos, figured out that pub get is never completing. The only thing about the package that is "odd" is that it's package name includes a number (it's suffixed with _2)... But that's been true for a while and hasn't been an issue.

Will continue to figure out what's going on by piping the pub get logs to a file or melos' stdout

[btrautmann]

So I finally figured out that the underlying reason for the hanging during bootstrap was that one of our packages had a GitHub ref to a branch that had gotten deleted. That's why only CI was able to reproduce this issue (because it didn't have this dependency cached in ~/.pub-cache). I was able to pipe the output from pub get back to melos and found this:

IO  : Finished git. Exit code 128.

    | Nothing output on stdout.

    | stderr:

    | | fatal: Path 'packages/storybook_flutter/pubspec.yaml' does not exist in '4484fda004dfc2f6ddcdb3558d85abaf91edaab7'
SLVR: Version solving took 0:00:01.750471 seconds.

    | Tried 1 solutions.
FINE: Resolving dependencies finished (1.8s).

ERR : Could not find a file named "packages/storybook_flutter/pubspec.yaml" in https://github.com/jeroen-meijer/storybook_flutter 4484fda004dfc2f6ddcdb3558d85abaf91edaab7.

FINE: Exception type: ApplicationException

FINE: package:pub/src/utils.dart 507:5                               fail

    | package:pub/src/source/git.dart 315:7                          BoundGitSource._describeUncached

    | ===== asynchronous gap ===========================

    | dart:async                                                     Future.catchError

    | package:pub/src/utils.dart 109:52                              captureErrors.wrappedCallback

    | package:stack_trace                                            Chain.capture

    | package:pub/src/utils.dart 122:11                              captureErrors

    | package:pub/src/command.dart 180:13                            PubCommand.run

    | package:args/command_runner.dart 209:27                        CommandRunner.runCommand
    | package:pub/src/command_runner.dart 173:24                     PubCommandRunner.runCommand

    | package:pub/src/command_runner.dart 158:20                     PubCommandRunner.run

    | package:dartdev/dartdev.dart 45:56                             runDartdev

    | /b/s/w/ir/cache/builder/sdk/pkg/dartdev/bin/dartdev.dart 11:9  main

---- End log transcript ----

pub get failed (1; ---- End log transcript ----)

exit code on my_package is 1
Caught an error! BootstrapException: failed to install my_package at /root/project/flutter/my_package.

So the failure is definitely making its way back to melos from the pub get command but for some reason _logPubGetFailed never completes, so the exception is never rethrown. I can look more into that but wanted to leave an update here.

[Salakar]

Not sure why the duplicate package issue happened since .dart_tool should be excluded entirely based on this:

melos/packages/melos/lib/src/package.dart

Line 325 in 80332f3

createGlob('**/.dart_tool', currentDirectoryPath: workspacePath);

- are you running on Windows perhaps?

Did you manage to look into _logPubGetFailed hanging? I wonder if pub tool never exits hence the hang, I have noticed pub infinitely retrying to get a package before 🤔

[blaugold]

Issue #216 looks like the same problem with _logPubGetFailed. I think I identified the root issue in #216 (comment), but haven't had time to create a fix.

[btrautmann]

Not sure why the duplicate package issue happened since .dart_tool should be excluded entirely based on this:

melos/packages/melos/lib/src/package.dart

Line 325 in 80332f3

createGlob('**/.dart_tool', currentDirectoryPath: workspacePath);

• are you running on Windows perhaps?

I am not, I was able to reproduce this on a Mac as well as the CI machine which I believe is running Ubuntu.

You can probably(?) reproduce this simply by doing a bootstrap, hitting CTRL+C before it completes, then running again. I had to rm -r .dart_tool/melos_tool every time a bootstrap failed while I was debugging this.

Did you manage to look into _logPubGetFailed hanging? I wonder if pub tool never exits hence the hang, I have noticed pub infinitely retrying to get a package before 🤔

I haven't had a chance to test this unfortunately but what @blaugold mentions sounds promising... Although that assumes the isWindows check comes back true.. I may have time to play around with that next week to see if that's the case.

[blaugold]

I now suspect that not fully consuming the stdout and stderr might be a problem not just on windows. This is according to the docs for Process.start:

Users must read all data coming on the stdout and stderr streams of processes started with Process.start. If the user does not read all data on the streams the underlying system resources will not be released since there is still pending data.

I've submitted PR #240 as a fix for #216, but it might fix this one, too.

[danilofuchs]

I also have this issue of infinite loading during bootstrap. Can this fix be published on 1.2.2 so I can test it?

[Salakar]

I also have this issue of infinite loading during bootstrap. Can this fix be published on 1.2.2 so I can test it?

Publishing now, give me a few mins

[Salakar]

@danilofuchs v1.2.2 is now on pub - please try it out, thanks!

[danilofuchs]

1.2.2 solved the infinite bootstrap. I now receive the correct error messages from pub get, thanks!

[Salakar]

1.2.2 solved the infinite bootstrap. I now receive the correct error messages from pub get, thanks!

Great, thanks for confirming, and huge thanks @blaugold for the fix!

Will close this as resolved

[danilofuchs]

Hey @Salakar. 1.2.2 solved the infinite loading problem, but the issues are unrelated.

When cancelling a bs (ctrl + c), the next bs will detect the folder inside .dart_tool as a package.