Plugin E2E: Pass testinfo to models

[sunker]

What this PR does / why we need it:

The Playwright testinfo class contains information about the currently running tests. It's passed as the second argument in tests, and the third argument in fixtures. Among many other things, testinfo allows you to conditionally fail a running test.

Today, all APIs of plugin-e2e are backwards compatible to Grafana 8.5. This is great, but I'm afraid we won't be able to commit to that for every new feature we build into plugin-e2e in the future. If for example we'd like to add support for testing Correlations at some point, we can't keep backwards compatibility as this feature was introduced pretty recently. We then need some way communicate to the user of plugin-e2e that those APIs don't work for the current Grafana version.

This PR injects the testinfo to all models. Even though we don't need it today, it will give us more flexibility when adding support for new APIs in the future. Adding testinfo to the models is a breaking change, so I rather do it now than after we've released 1.0.

Here are a few examples of how it can be used:

export class DataSourceConfigPage extends GrafanaPage {
  async addCorrelation() {
      this.ctx.testInfo.fail(
        semver.lt(this.ctx.grafanaVersion, '10.0.0'),
        'Correlations are not supported in this version of Grafana. Please skip this test for this version of Grafana.'
      );
      ...
  }
}

const createDsReq = await request.post('/api/datasources', datasource);
  if (!createDsReq.ok()) {
    this.ctx.testInfo.fail('Could not create data source');
  }

If we don't have a way to force a test to fail, it will fail on something else with possibly weird error messages that we don't control. We can write to the console, but that easily gets lost in the test report.

Breaking change notice

You now need to pass testInfo to page models.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

📦 Published PR as canary version: Canary Versions

✨ Test out this PR locally via:

npm install @grafana/create-plugin@4.3.0-canary.769.0b01966.0
npm install @grafana/plugin-e2e@0.20.0-canary.769.0b01966.0
# or 
yarn add @grafana/create-plugin@4.3.0-canary.769.0b01966.0
yarn add @grafana/plugin-e2e@0.20.0-canary.769.0b01966.0

[github-actions]

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged and will trigger a new minor release.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

[mckn]

LGTM!

I guess this will cover the scenario where a plugin developer is conditionally providing functionality in tier plugin based on Grafana version. So for me this makes sense. The only thing is that I think we should fail the test and let the plugin author (who writes the test) add logic to skip the test since they have added logic to conditionally use the feature/functionality.

[sunker]

I guess this will cover the scenario where a plugin developer is conditionally providing functionality in tier plugin based on Grafana version. So for me this makes sense. The only thing is that I think we should fail the test and let the plugin author (who writes the test) add logic to skip the test since they have added logic to conditionally use the feature/functionality.

Yep totally agree!

[grafana-plugins-platform-bot]

🚀 PR was released in @grafana/plugin-e2e@0.20.0 🚀