[Epic]: Visual Regression Tool Migration

[TristanWatanabe]

Problem

We are looking to move off using Screener as our Visual Regression testing solution for the entire monorepo by the end of the year.

Solution

Near Term

• Near term plan is to integrate and use an internal 1JS VR tool to replace screener for v0, v7, v8, and v9. The intent is for v0, v7 and v8 to stay with this solution going forward since they are legacy libraries.
  • Integrate pipeline jobs to run 1JS VR tool (v9) PR to add tasks in pipeline for v9 VRT integration #25606
    • Only run VR test when package is affected Add a check whether to run vr tests based on affected packages check #25676
  • Integrate pipeline jobs to run 1JS VR tool (v8) 1JS VRT tooll add pipeline to generate V8 screenshots #25643
    • Only run VR test when package is affected Add a check whether to run vr tests based on affected packages check #25676
  • Add v0 stories to be used by 1JS VR tool chore(v0 docs): Add storybook stories that reference docsite examples for 1JS VR tool migration #25663
  • Integrate pipeline jobs to run 1JS VR tool (v0) feat (CI): Enable VR Tool to run react-northstar tests for PRs #25922
  • Intergrate pipeline jobs to run 1JS VR tool (v7) feat (CI): Enable 1JS VR tool to run on v7 #25946
  • Remove Screener workflow files (master branch) chore (CI): delete Screener workflows #25977
  • Remove Screener pipeline job (v7 branch) chore(CI v7): Remove Screener job from pipeline #25990
  • Replace any Screener imports with StoryWright for v8 chore (v8-vr-tests): Remove usage of Screener #25979
  • Replace any Screener imports with StoryWright for v9 chore (v9 vr-tests): Remove usage of Screener #25980
  • Replace any Screener imports with StoryWright for v7 chore (v7 vr-tests): Completely remove usage of Screener #26049
  • Remove any Screener related files from v0 chore (v0 vr-tests): Remove usage of Screener #26053
  • Remove Screener dependencies from package.json chore: remove screener dependencies and various screener removal cleanups #26070
  • Cleanup new 1JS VR tool yaml files (remove convert scripts,etc) chore: remove screener dependencies and various screener removal cleanups #26070
  • Update any VR tool related docs in repo
  • Delete Screener related github secrets Remove screener related secrets now that they are not being used #26086

Long Term

• Long term plan is to migrate v9 VR tests to use Chromatic (Internal Doc writeup). This will have to be done in multiple steps:
  • Make react-storybook-addon configurable to be used by vr stories: feat(react-storybook-addon): make withFluentProvider decorator configurable to be used for VR tests #25162
  • Refactor all v9 VR tests to use storybook Component Story Format (CSF)
    • Accordion chore(vr-tests-v9): Convert Accordion VR tests to CSF #25525
    • Button chore(vr-tests-v9): Convert Button VR tests to CSF #25108
    • Card chore(vr-tests-v9): Convert Card VR tests to CSF #25526
    • Dialog chore(vr-tests-v9): Convert Dialog and Image VR tests to CSF #25527
    • Image chore(vr-tests-v9): Convert Dialog and Image VR tests to CSF #25527
  • Integrate Chromatic project for v9 into repo and make available for use
  • Slowly migrate each v9 component VR test to chromatic and refactor Screener interaction and hover tests to appropriate CSF equivalent.
  • Colocate VR stories to their respective packages (i.e. move Menu VR tests to the react-menu package etc.)

[ling1726]

@TristanWatanabe since we are moving to CSF, would it be feasible to also colocate the vr-test stories like we do for other stories in the docsite ?

[Hotell]

@TristanWatanabe since we are moving to CSF, would it be feasible to also colocate the vr-test stories like we do for other stories in the docsite ?

One topic we already talked about is to co-locate those vr stories/tests within package, same for performance etc. This approach should be taken into consideration if a new VR solution supports this or not and based on that tweak the decision of the tool to be used.

I'm not sure what you meant by "other stories in the docsite" @ling1726 ?

[ling1726]

I'm not sure what you meant by "other stories in the docsite" @ling1726 ?

@Hotell all our docsite stories are colocated. i.e Menu docs stories are in the react-menu package

[Hotell]

I'm not sure what you meant by "other stories in the docsite" @ling1726 ?

@Hotell all our docsite stories are colocated. i.e Menu docs stories are in the react-menu package

gotcha, so we are on same page -> "about is to co-locate those vr stories/tests within package"

[TristanWatanabe]

Added vr stories colocation to the "todo" steps in the issue

[microsoft-github-policy-service]

Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes.

Still require assistance? Please, create a new issue with up-to date details.

[microsoft-github-policy-service]

Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes.

Still require assistance? Please, create a new issue with up-to date details.

[microsoft-github-policy-service]

Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes.

Still require assistance? Please, create a new issue with up-to date details.

[microsoft-github-policy-service]

Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes.

Still require assistance? Please, create a new issue with up-to date details.

[microsoft-github-policy-service]

Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes.

Still require assistance? Please, create a new issue with up-to date details.