Test Support: Get Packager from Dependencies

[alexlafroscia]

As discussed in #759, we need some way to expand the tests to cover the behavior of packagers besides the existing Webpack one. We could do this by having the build-a-real-app tests load their packager dynamically based on locating a known packager in their dependencies, then use scenario-tester to configure scenarios with different packagers installed.

This PR sets up that infrastructure and configures the app-template tests to use it.

[alexlafroscia]

Once thing worth visiting as part of reviewing/working on this code; @ef4 mentioned something specific when I suggested this idea last week

Also, I don't think we want vite vs webpack to multiply against all the different supported ember versions.

This does expand the packager matrix by the "ember version" matrix, though that makes sense given the fact that the packager matrix only contains Webpack currently.

Once the Vite packager is added to the mix, we'd need some way to avoid generating every possible job -- does Scenario Tester have an API for ignoring particular combinations?

[alexlafroscia]

Point maybe worth bike-shedding; using packager_webpack rather than just webpack means that the whole packager_webpack string is in the name of the CI job, etc. That's nice and specific but kind of verbose, in case it matters to anyone 🤷‍♂️

[thoov]

Once the Vite packager is added to the mix, we'd need some way to avoid generating every possible job -- does Scenario
Tester have an API for ignoring particular combinations?

@alexlafroscia It's possible to filter the matrix before we hand it off to github:

embroider/test-packages/support/suite-setup-util.ts

Lines 83 to 97 in d8edf75

	// while we convert over from test-packages to test-scenarios here we merge both together
	let { stdout } = await execa('scenario-tester', ['list', '--require', 'ts-node/register', '--files', '*-test.ts'], {
	cwd: resolve(__dirname, '..', '..', 'tests', 'scenarios'),
	preferLocal: true,
	});
	let testScenarios = stdout.split('\n');
	testScenarios.forEach(scenario => {
	suites.push({
	name: scenario,
	command: 'yarn',
	args: ['test', '--filter', scenario],
	dir: resolve(__dirname, '..', '..', 'tests', 'scenarios'),
	});
	});

[ef4]

I think this is overkill and we should just add vite to this list:

export function supportMatrix(scenarios: Scenarios) {
  return scenarios.expand({
    lts_3_16,
    lts_3_20,
    lts_3_24,
    release,
    vite
  });

The vite scenario would use the default ember version. If we later find some particular combinations that are problematic and need regression coverage, we can explicitly add those combinations.

[alexlafroscia]

Oh okay, cool -- I can tweak it to work that way instead!

Should each of the "Ember Version" variants explicitly add the Embroider packager, and we'll remove the Webpack "variant" altogether?

[ef4]

Going to close this, because while I expect to be adding additional packagers pretty soon, I don't anticipate doing it inside the broccoli pipeline. We want to invert control, so that you run them directly and they have plugins that call out to the embroider-specific things. That would change the way this test setup would work.