feat: ACI Notification Banners #23256

mike-plummer · 2022-08-10T18:04:21Z

Closes
Dashboard Integration Smart Login Notification #22933
Dashboard Integration Smart Organization Creation Notification #22934
Dashboard Integration banner state for "no projectId"/"connect your project" state #22798
Dashboard Integration Smart Record Run Notification #22935

User facing changelog

Add new banners in the app to help get your project connected to the dashboard and complete a first run recording

Additional details

Four main areas here:

Adding new section of project preferences to track the "last shown" and "dismissed" timestamps for each banner being added here
Create new TrackedBanner component to allow reuse of storing "last shown" and "dismissed" info for banners
Add four new "Smart Notifications" that trigger based on users progress towards connecting their project to the dashboard
- Login
  - Has been using Cypress for at least 4 days
  - Is not logged in
  - Not previously dismissed in this project
- Create Organization
  - Has been using Cypress for at least 4 days
  - Is logged in
  - Is not a member of any cloud organizations
  - Not previously dismissed in this project
- Connect Project
  - Has been using Cypress for at least 4 days
  - Is logged in
  - Is member of at least one cloud organization
  - Project is not successfully connected to dashboard
  - Not previously dismissed in this project
- Record First Run
  - Has been using Cypress for at least 4 days
  - Is logged in
  - Is member of at least one cloud organization
  - Project is successfully connected to dashboard
  - Project does not have any cloud recording runs
  - Not previously dismissed in this project
Connect new banners to SpecsList in the SpecsListBanners component - this is currently serving as a "banner controller" of sorts to control which banners to display and when (a refinement of this construct is coming in future work related to A/B testing of banner content)

Steps to test

Note: To fully test it will be necessary to either create a new user or have an existing user removed from all organizations. You may want to create a new "throwaway" user rather than remove yourself from the shared Cypress org in staging.

Note: launch with CYPRESS_INTERNAL_ENV=staging for staging.

Create a new empty Cypress project (name it something distinctive) with at least one spec (content is unimportant)
Open the project in Cypress. Ensure you are logged out. Validate that "Smart Login Banner" does not display.
Close the project and find its state.json file. On Mac this is stored in a project-specific directory under ~/Library/Application Support/Cypress/cy/staging/projects/
Modify state.json to have a firstOpened value more than 4 days in the past: Date.now() - 8.64e+7 * 4
Reopen Cypress, validate the "Smart Login Banner" now displays
Dismiss the banner by clicking the "X" button. Refresh app and validate banner does not display
Reopen state.json, observe an entry for the login banner has been created. Remove it, and reopen the project.

    "aci_082022_login": {
      "lastShown": 1660075007022,
      "dismissed": 1660075012749
    },

Observe login banner once again displays. Use banner to execute login flow, validate it succeeds.
- Note: Next three prompts use same logic for respecting dismissal
"Smart Create Organization Banner" should now display (if your user has been removed from all cloud orgs). Use banner to create an organization, validate UI updates to no longer show create organization banner
"Smart Connect Project Banner" should now display. Use banner to create new cloud project, validate UI updates to no longer show connect project banner
"Smart Record Banner" should now display. Validate record command looks correct (should include --component flag for CT, valid record key) and can be copy-pasted
Optional: Record a cloud run, validate that "Smart Record Banner" does not display on next app launch

How has the user experience changed?

Login Banner

Login.mov

Create Org Banner

CreateOrg.mov

Connect Project Banner

ConnectProject.mov

Record Banner

Record.mov

PR Tasks

Have tests been added/updated?
Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
[na] Has a PR for user-facing changes been opened in cypress-documentation?
[na] Have API changes been updated in the type definitions?

…com/cypress-io/cypress into mikep/22933-aci-smart-notifications

cypress-bot · 2022-08-10T18:04:23Z

Thanks for taking the time to open a PR!

Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

cypress · 2022-08-10T20:38:16Z

Test summary

37828 • 0 • 615 • 0 • 7

Run details


Project	cypress
Status	Passed
Commit	`da8ae24`
Started	Aug 16, 2022 7:26 PM
Ended	Aug 16, 2022 7:40 PM
Duration	14:12 💡
OS	Linux Debian - 11.3
Browser	Multiple

View run in Cypress Dashboard ➡️

Flakiness

	cypress/e2e/commands/net_stubbing.cy.ts		5
1	network stubbing > intercepting request > can delay with deprecated delayMs param
2	network stubbing > intercepting request > can delay with deprecated delayMs param
3	network stubbing > intercepting request > can delay with deprecated delayMs param
4	network stubbing > intercepting request > can delay with deprecated delayMs param
5	network stubbing > intercepting request > can delay with deprecated delayMs param
This comment includes only the first 5 flaky tests. See all 7 flaky tests in the Cypress Dashboard.

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

* Add utm params to login & create org links * Fix org refresh on create * Fix styling of record command

lmiller1990

Did a basic review and ask a few questions, sparked some conversation... I will give it a proper test now and report back.

lmiller1990 · 2022-08-12T01:09:14Z

packages/app/src/specs/SpecsListBanners.vue

@@ -96,7 +96,7 @@
    </Button>
  </Alert>
  <Alert
-    v-if="showProjectRequestAccess"
+    v-else-if="showProjectRequestAccess"


I think many v-else-if combination is fine, but one thing I started using in my personal projects lately is v-switch, I wonder if it would be useful in Cypress, too: https://www.npmjs.com/package/@lmiller1990/v-switch

The source code is only about 10 lines, we could just copy-paste and register as a global component.

packages/app/src/specs/SpecsListBanners.vue

packages/app/src/specs/banners/ConnectProjectBanner.vue

packages/app/src/specs/banners/RecordBanner.vue

packages/app/src/specs/banners/index.ts

packages/frontend-shared/src/gql-components/HeaderBarContent.vue

packages/types/src/config.ts

marktnoonan · 2022-08-16T15:10:39Z

Still reviewing code and tests but wanted to share that functionality all works for me!

@pstakoun Product feedback - I wasn't quite expecting the "guided tour" feeling where if I don't dismiss the banner, its content changes on the next step - eg, "Login" content -> "Connect" content -> "Record" content. UX wise the internal content of a banner changing like this feels like it could use some sort of "journey" UI ("Step 1 of 3") or some element that visibly "progresses" as well as simply the text content being updated. I think the "guided" feeling is good & leaning into it a bit could be a future enhancement. It also feels like a temporary banner might not be the exact right UI element to reflect this, it's a bit counterintuitive to me that the contents of a dismissable banner change over time, it's different to all our other banners that look similar.

pstakoun · 2022-08-16T15:14:36Z

@marktnoonan totally agree with you and that feedback resonates with me. I think banners are not the form-factor that this experience will be in the near future - an onboarding checklist similar to that in the Dashboard is also under consideration.

marktnoonan · 2022-08-16T15:15:06Z

We have some overflow issues anyway on this page but maybe we could still confine this messaging to max size of "viewport width minus nav size" or something?

astone123

I tested each of the banner situations with staging. Each one worked correctly for me. One thing was that in order to get into the state where I was logged in but wasn't in an org, I had to delete my existing org. Not sure how a user would get into this state since I think you need to be in an org to complete the login flow, but good that we're handling that either way.

Nice work 🎉 I'm excited to see these banners get more styled in the future!

packages/app/src/specs/SpecsListBanners.cy.tsx

mike-plummer · 2022-08-16T15:58:02Z

We have some overflow issues anyway on this page but maybe we could still confine this messaging to max size of "viewport width minus nav size" or something?

@marktnoonan I don't think we should add custom styling for these at this time. From a UX perspective the app has a hard minimum width, and I think it would look kinda funky for parts of the UI to try to be responsive while the majority overflows. I think we should be more responsive, but it should be tackled holistically so there is a consistent experience

marktnoonan · 2022-08-16T16:14:28Z

packages/app/src/specs/SpecsListBanners.vue

@@ -30,7 +30,7 @@
    </p>
  </Alert>
  <Alert
-    v-if="showFetchError"
+    v-else-if="showFetchError"
    v-model="showFetchError"


Hmm. The combination of v-if and v-model pointing to the same value seems like a hint that v-model is not doing what we want and hiding/showing the Alert on its own?

This was an attempt to simplify the "only show one banner" concern within SpecsListBanners.vue. Currently, showFetchError just cares about whether the conditions for its display are met independently, but if we remove the v-if/v-else-if then I would need to add conditions to check whether any higher-priority banners are shown when determining the model value. This felt like a cleaner approach

Yeah that makes sense, at least for this iteration. I know more is coming 👍

marktnoonan · 2022-08-16T16:15:58Z

packages/app/src/specs/SpecsListBanners.vue

+  />
+  <LoginBanner
+    v-else-if="showLoginBanner"
+    v-model="showLoginBanner"


Similar comment here. Seems like we could push the if inside these components (or use a wrapper) and drive the presence/absence of these things from the state being tracked via the v-model.

packages/app/src/specs/banners/CreateOrganizationBanner.vue

marktnoonan · 2022-08-16T17:58:13Z

packages/app/src/specs/banners/CreateOrganizationBanner.cy.tsx

+
+    cy.get('a')
+    .should('have.attr', 'href')
+    .and('contain', linkHref)


Should we assert the specPage.banners.Join or create an organization in Cypress Dashboard to access your projects and recorded test runs here? Since grabbing that content is also a responsibility of this component.

marktnoonan

No actual blockers but requesting changes to discuss merge timing.

marktnoonan · 2022-08-16T18:23:28Z

packages/app/src/specs/banners/LoginBanner.cy.tsx

+  it('should render expected content', () => {
+    cy.mount({ render: () => <LoginBanner modelValue={true} /> })
+
+    cy.percySnapshot()


Similar to an earlier comment, I'd like this test to fail if the content were missing. Percy will catch this too but is a lot slower/sometimes gets ignored.

Or maybe we have an e2e test or higher level component test that checks the contents, and I haven't see it yet.

marktnoonan · 2022-08-16T18:43:29Z

packages/frontend-shared/src/gql-components/HeaderBarContent.cy.tsx

@@ -378,6 +378,10 @@ describe('<HeaderBarContent />', { viewportWidth: 1000, viewportHeight: 750 }, (
            },
            render: (gqlVal) => <div class="border-current border-1 h-700px resize overflow-auto"><HeaderBarContent gql={gqlVal} show-browsers={true} allowAutomaticPromptOpen={true} /></div>,
          })
+
+          cy.tick(2000)


Could we add a comment with the intention behind this tick? Would tests fail if we removed it?

marktnoonan · 2022-08-16T18:45:14Z

packages/frontend-shared/src/gql-components/HeaderBarContent.vue

+const isOpenDelayElapsed = ref(false)
+
+onMounted(() => {
+  setTimeout(() => isOpenDelayElapsed.value = true, 2000)


nice, this makes sense of the tick in the test code for mounting.

Sidenote: we could simplify a little and remove the mounted hook updating the ref with useTimeout (https://vueuse.org/shared/usetimeout/)

marktnoonan

I've left a few comments that are mostly nits, I think it worth adding assertions to cover the remaining new content in en-US.json in the component specs for the components that render those strings though.

Will leave my change request up for that reason, but @mike-plummer nothing else is blocking (and feel free to point of if I'm missed something about the content).

* Add test assertions for banner text content * Add comment to cy.tick

mike-plummer added 18 commits August 8, 2022 10:16

feat: ACI Notification Banners

22ea332

Add tests

07a2adb

Add tests, delay docs prompt after banners

8783a86

Fix icons, implement create org link

4539c7d

Fix savedState path

9c58238

Fix banner triggers, use existing org creation modal

bdabb99

feat: ACI Notification Banners

7efcf8d

Add tests

8ebb12a

Add tests, delay docs prompt after banners

df4076a

Fix icons, implement create org link

5b5a844

Fix savedState path

67c61b8

Fix banner triggers, use existing org creation modal

1777488

Fix tests

21015f3

Fix test

00e70bb

Merge branch 'mikep/22933-aci-smart-notifications' of https://github.…

8c60759

…com/cypress-io/cypress into mikep/22933-aci-smart-notifications

Use setPreferences api

62d3425

Simplify org creation to not use cloud connect modal

8e83a0f

Cleanup

861ec22

mike-plummer added 2 commits August 10, 2022 13:29

Fix create orgs link

bd14556

Merge branch 'develop' into mikep/22933-aci-smart-notifications

01d40fa

mike-plummer requested review from marktnoonan, warrensplayer and emilyrohrbough August 11, 2022 12:49

mike-plummer marked this pull request as ready for review August 11, 2022 12:52

mike-plummer requested review from a team as code owners August 11, 2022 12:52

Fix issues identified during acceptance testing

096ee09

* Add utm params to login & create org links * Fix org refresh on create * Fix styling of record command

lmiller1990 reviewed Aug 12, 2022

View reviewed changes

astone123 approved these changes Aug 16, 2022

View reviewed changes

marktnoonan reviewed Aug 16, 2022

View reviewed changes

packages/app/src/specs/SpecsListBanners.cy.tsx Outdated Show resolved Hide resolved

mike-plummer added 2 commits August 16, 2022 11:12

Move tests to align with execution order

4beb957

Merge branch 'develop' into mikep/22933-aci-smart-notifications

49cf741

marktnoonan reviewed Aug 16, 2022

View reviewed changes

packages/app/src/specs/banners/CreateOrganizationBanner.vue Outdated Show resolved Hide resolved

mike-plummer added 2 commits August 16, 2022 12:39

Fix e2e test, use Button instead of ExternalLink

bf73ad7

Merge branch 'develop' into mikep/22933-aci-smart-notifications

0ea3a0a

marktnoonan reviewed Aug 16, 2022

View reviewed changes

marktnoonan requested changes Aug 16, 2022

View reviewed changes

marktnoonan reviewed Aug 16, 2022

View reviewed changes

mike-plummer added 2 commits August 16, 2022 14:18

Address review comments

e85657a

* Add test assertions for banner text content * Add comment to cy.tick

Merge branch 'develop' into mikep/22933-aci-smart-notifications

da8ae24

marktnoonan approved these changes Aug 16, 2022

View reviewed changes

mike-plummer merged commit 7412548 into develop Aug 16, 2022

mike-plummer deleted the mikep/22933-aci-smart-notifications branch August 16, 2022 22:05

warrensplayer mentioned this pull request Aug 28, 2022

feat: show cloud connect options when hoving spec list fields #23584

Merged

3 tasks

warrensplayer mentioned this pull request Sep 9, 2022

feat: add cohorts for A/B testing in Specs List Banners #23735

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: ACI Notification Banners #23256

feat: ACI Notification Banners #23256

mike-plummer commented Aug 10, 2022 •

edited by lmiller1990

Loading

cypress-bot bot commented Aug 10, 2022

cypress bot commented Aug 10, 2022 •

edited

Loading

lmiller1990 left a comment

lmiller1990 Aug 12, 2022

marktnoonan commented Aug 16, 2022 •

edited

Loading

pstakoun commented Aug 16, 2022

marktnoonan commented Aug 16, 2022

astone123 left a comment

mike-plummer commented Aug 16, 2022

marktnoonan Aug 16, 2022

mike-plummer Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan left a comment

marktnoonan Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan Aug 16, 2022

marktnoonan left a comment

feat: ACI Notification Banners #23256

feat: ACI Notification Banners #23256

Conversation

mike-plummer commented Aug 10, 2022 • edited by lmiller1990 Loading

User facing changelog

Additional details

Steps to test

How has the user experience changed?

PR Tasks

cypress-bot bot commented Aug 10, 2022

cypress bot commented Aug 10, 2022 • edited Loading

Test summary

Run details

Flakiness

lmiller1990 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktnoonan commented Aug 16, 2022 • edited Loading

pstakoun commented Aug 16, 2022

marktnoonan commented Aug 16, 2022

astone123 left a comment

Choose a reason for hiding this comment

mike-plummer commented Aug 16, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktnoonan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktnoonan left a comment

Choose a reason for hiding this comment

mike-plummer commented Aug 10, 2022 •

edited by lmiller1990

Loading

cypress bot commented Aug 10, 2022 •

edited

Loading

marktnoonan commented Aug 16, 2022 •

edited

Loading