1JS VRT tooll add pipeline to generate V8 screenshots #25643
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
storywright tests
[skip ci]
Update README.md
Update README.md
Update README.md
Asset size changesSize Auditor did not detect a change in bundle size for any component! Baseline commit: d419dfa7124fb15ebae8a27b30bdad869964301b (build) |
|
This pull request is automatically built and testable in CodeSandbox. To see build info of the built libraries, click here or the icon next to each commit SHA. Latest deployment of this branch, based on commit 7af6ad0:
|
Perf Analysis (
|
| Scenario | Render type | Master Ticks | PR Ticks | Iterations | Status |
|---|---|---|---|---|---|
| Avatar | mount | 1148 | 1177 | 5000 | |
| Button | mount | 884 | 840 | 5000 | |
| FluentProvider | mount | 1438 | 1425 | 5000 | |
| FluentProviderWithTheme | mount | 555 | 570 | 10 | |
| FluentProviderWithTheme | virtual-rerender | 561 | 522 | 10 | |
| FluentProviderWithTheme | virtual-rerender-with-unmount | 554 | 578 | 10 | |
| MakeStyles | mount | 1692 | 1805 | 50000 | |
| SpinButton | mount | 2298 | 2334 | 5000 |
📊 Bundle size reportUnchanged fixtures
|
Perf Analysis (
|
| Scenario | Current PR Ticks | Baseline Ticks | Ratio |
|---|---|---|---|
| ImageMinimalPerf.default | 312 | 268 | 1.16:1 |
| LabelMinimalPerf.default | 316 | 273 | 1.16:1 |
| AvatarMinimalPerf.default | 150 | 131 | 1.15:1 |
| HeaderMinimalPerf.default | 291 | 253 | 1.15:1 |
| ListWith60ListItems.default | 527 | 460 | 1.15:1 |
| IconMinimalPerf.default | 558 | 485 | 1.15:1 |
| TextMinimalPerf.default | 280 | 244 | 1.15:1 |
| CardMinimalPerf.default | 446 | 392 | 1.14:1 |
| LayoutMinimalPerf.default | 293 | 257 | 1.14:1 |
| AnimationMinimalPerf.default | 436 | 386 | 1.13:1 |
| AttachmentSlotsPerf.default | 909 | 814 | 1.12:1 |
| MenuMinimalPerf.default | 705 | 631 | 1.12:1 |
| DialogMinimalPerf.default | 630 | 567 | 1.11:1 |
| RadioGroupMinimalPerf.default | 360 | 324 | 1.11:1 |
| SegmentMinimalPerf.default | 275 | 253 | 1.09:1 |
| SplitButtonMinimalPerf.default | 3633 | 3368 | 1.08:1 |
| EmbedMinimalPerf.default | 2989 | 2785 | 1.07:1 |
| PortalMinimalPerf.default | 151 | 142 | 1.06:1 |
| ButtonOverridesMissPerf.default | 1097 | 1045 | 1.05:1 |
| ItemLayoutMinimalPerf.default | 930 | 882 | 1.05:1 |
| ButtonMinimalPerf.default | 137 | 132 | 1.04:1 |
| GridMinimalPerf.default | 280 | 268 | 1.04:1 |
| ListCommonPerf.default | 528 | 508 | 1.04:1 |
| MenuButtonMinimalPerf.default | 1361 | 1310 | 1.04:1 |
| ProviderMinimalPerf.default | 309 | 297 | 1.04:1 |
| AlertMinimalPerf.default | 221 | 215 | 1.03:1 |
| BoxMinimalPerf.default | 283 | 276 | 1.03:1 |
| ChatDuplicateMessagesPerf.default | 224 | 217 | 1.03:1 |
| FormMinimalPerf.default | 309 | 299 | 1.03:1 |
| HeaderSlotsPerf.default | 597 | 579 | 1.03:1 |
| ListMinimalPerf.default | 430 | 421 | 1.02:1 |
| TooltipMinimalPerf.default | 1993 | 1962 | 1.02:1 |
| AttachmentMinimalPerf.default | 106 | 106 | 1:1 |
| DividerMinimalPerf.default | 288 | 289 | 1:1 |
| ProviderMergeThemesPerf.default | 1066 | 1064 | 1:1 |
| ChatMinimalPerf.default | 587 | 595 | 0.99:1 |
| ChatWithPopoverPerf.default | 299 | 302 | 0.99:1 |
| ListNestedPerf.default | 447 | 453 | 0.99:1 |
| ReactionMinimalPerf.default | 305 | 308 | 0.99:1 |
| DropdownMinimalPerf.default | 2196 | 2250 | 0.98:1 |
| SliderMinimalPerf.default | 1278 | 1306 | 0.98:1 |
| TableMinimalPerf.default | 322 | 330 | 0.98:1 |
| TreeWith60ListItems.default | 136 | 139 | 0.98:1 |
| FlexMinimalPerf.default | 228 | 236 | 0.97:1 |
| InputMinimalPerf.default | 858 | 887 | 0.97:1 |
| PopupMinimalPerf.default | 475 | 491 | 0.97:1 |
| CustomToolbarPrototype.default | 2191 | 2256 | 0.97:1 |
| DatepickerMinimalPerf.default | 4884 | 5089 | 0.96:1 |
| TableManyItemsPerf.default | 1468 | 1522 | 0.96:1 |
| TreeMinimalPerf.default | 661 | 685 | 0.96:1 |
| StatusMinimalPerf.default | 532 | 562 | 0.95:1 |
| AccordionMinimalPerf.default | 104 | 112 | 0.93:1 |
| ButtonSlotsPerf.default | 419 | 455 | 0.92:1 |
| DropdownManyItemsPerf.default | 499 | 545 | 0.92:1 |
| TextAreaMinimalPerf.default | 348 | 377 | 0.92:1 |
| ToolbarMinimalPerf.default | 696 | 756 | 0.92:1 |
| CarouselMinimalPerf.default | 340 | 375 | 0.91:1 |
| RosterPerf.default | 1589 | 1773 | 0.9:1 |
| SkeletonMinimalPerf.default | 251 | 279 | 0.9:1 |
| CheckboxMinimalPerf.default | 1534 | 1715 | 0.89:1 |
| VideoMinimalPerf.default | 531 | 594 | 0.89:1 |
| RefMinimalPerf.default | 160 | 183 | 0.87:1 |
| LoaderMinimalPerf.default | 230 | 271 | 0.85:1 |
Perf Analysis (
|
| Scenario | Render type | Master Ticks | PR Ticks | Iterations | Status |
|---|---|---|---|---|---|
| BaseButton | mount | 1068 | 1136 | 5000 | |
| Breadcrumb | mount | 2877 | 2876 | 1000 | |
| Checkbox | mount | 2556 | 2448 | 5000 | |
| CheckboxBase | mount | 2285 | 2216 | 5000 | |
| ChoiceGroup | mount | 4184 | 4225 | 5000 | |
| ComboBox | mount | 1197 | 1147 | 1000 | |
| CommandBar | mount | 9214 | 9233 | 1000 | |
| ContextualMenu | mount | 10416 | 10497 | 1000 | |
| DefaultButton | mount | 1287 | 1276 | 5000 | |
| DetailsRow | mount | 3492 | 3439 | 5000 | |
| DetailsRowFast | mount | 3509 | 3329 | 5000 | |
| DetailsRowNoStyles | mount | 3160 | 3209 | 5000 | |
| Dialog | mount | 2873 | 2982 | 1000 | |
| DocumentCardTitle | mount | 562 | 582 | 1000 | |
| Dropdown | mount | 3014 | 3008 | 5000 | |
| FocusTrapZone | mount | 1956 | 1957 | 5000 | |
| FocusZone | mount | 1945 | 1884 | 5000 | |
| GroupedList | mount | 1878 | 2138 | 2 | |
| GroupedList | virtual-rerender | 1015 | 1097 | 2 | |
| GroupedList | virtual-rerender-with-unmount | 1570 | 1590 | 2 | |
| GroupedListV2 | mount | 541 | 555 | 2 | |
| GroupedListV2 | virtual-rerender | 528 | 503 | 2 | |
| GroupedListV2 | virtual-rerender-with-unmount | 539 | 520 | 2 | |
| IconButton | mount | 1790 | 1748 | 5000 | |
| Label | mount | 644 | 723 | 5000 | |
| Layer | mount | 4114 | 4135 | 5000 | |
| Link | mount | 800 | 806 | 5000 | |
| MenuButton | mount | 1655 | 1521 | 5000 | |
| MessageBar | mount | 2080 | 2290 | 5000 | |
| Nav | mount | 3112 | 3170 | 1000 | |
| OverflowSet | mount | 1342 | 1285 | 5000 | |
| Panel | mount | 2482 | 2546 | 1000 | |
| Persona | mount | 1266 | 1259 | 1000 | |
| Pivot | mount | 1571 | 1537 | 1000 | |
| PrimaryButton | mount | 1476 | 1513 | 5000 | |
| Rating | mount | 6826 | 6813 | 5000 | |
| SearchBox | mount | 1497 | 1504 | 5000 | |
| Shimmer | mount | 2856 | 2803 | 5000 | |
| Slider | mount | 2091 | 1946 | 5000 | |
| SpinButton | mount | 4492 | 4421 | 5000 | |
| Spinner | mount | 800 | 752 | 5000 | |
| SplitButton | mount | 2977 | 3091 | 5000 | |
| Stack | mount | 807 | 817 | 5000 | |
| StackWithIntrinsicChildren | mount | 2407 | 2390 | 5000 | |
| StackWithTextChildren | mount | 4786 | 4814 | 5000 | |
| SwatchColorPicker | mount | 10046 | 9801 | 5000 | |
| TagPicker | mount | 2705 | 2604 | 5000 | |
| TeachingBubble | mount | 80424 | 85070 | 5000 | |
| Text | mount | 753 | 725 | 5000 | |
| TextField | mount | 1425 | 1542 | 5000 | |
| ThemeProvider | mount | 1395 | 1415 | 5000 | |
| ThemeProvider | virtual-rerender | 1050 | 1026 | 5000 | |
| ThemeProvider | virtual-rerender-with-unmount | 2077 | 2028 | 5000 | |
| Toggle | mount | 1088 | 978 | 5000 | |
| buttonNative | mount | 488 | 473 | 5000 |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
(re-ran pipelines to see if screener timeout will resolve) |
| mkdir -p screenshots | ||
| cp -rf apps/vr-tests-react-components/dist/screenshots/*.png screenshots/ | ||
| displayName: Collate Artifacts | ||
| - bash: node node_modules/vrscreenshotdiff/lib/index.js release --clientType "fluentuiv9" --buildId $(Build.BuildId) |
There was a problem hiding this comment.
Just for context, can you explain what this step is doing? And are all the secrets being referenced from the fluent-vr-tool Azure resource group?
There was a problem hiding this comment.
This step is responsible to
For baseline pipeline
- Upload the screenshots to azure blob storage
- Add screenshot metadata in cosmos database like buildid, pipelineid, screenshotpath in blob storage.
For pr pipeline
- Download baseline screenshots
- Diff baseline screenshot with pr screenshots
- upload diffed screenshots to blob storage
- Add details about diffing result in cosmos database
- Post comment on PR
Currently the secret are stored as secret variables in pipeline. The resource at the moment are pointing to our tests resurce group. As soon as piyush is back from sick leave we will point to resources in fluent-vr-tool resource group
| - job: TestVRTool | ||
| - job: VisualRegressionTest_V9 | ||
| variables: | ||
| pipelineId: '211' |
There was a problem hiding this comment.
Is this an arbitrary pipelineId or does it have some connection to an actual pipeline somewhere?
There was a problem hiding this comment.
Yes this is arbitrary pipelineId. It's an id to identify particular job. Earlier since there was only job we just used the default pipelineid. But since now we have 2 jobs for V8,V9 in same pipeline we created two id's. As long as every job has unique id it should be fine
TristanWatanabe
approved these changes
Nov 14, 2022
27 tasks
NotWoods
pushed a commit
to NotWoods/fluentui
that referenced
this pull request
Nov 18, 2022
Hotell
pushed a commit
to Hotell/fluentui
that referenced
this pull request
Feb 9, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current Behavior
1JS VR tool is configured to run only for fluentui v9
New Behavior
Changes to add support for generating screenshots for fluentui v8
Related Issue(s)
Part of #25078