Add a notice within the Optimize setup screen, increasing awareness of Analytics configuration conflicts

[jamesozzie]

Feature Description

As one user mentioned in the WP Support Forums Optimize experiments will fail if a user is logged into their site and the are not counting logged in users within their Analytics configuration.

As this configuration is an option within Site Kit it would be useful to inform users of this likely cause of failed Optimize Experiments setup.

A notice similar to the below may suffice:
Notice: When setting up and testing an experiment at optimize.google.com please ensure you are not logged into your WordPress site so your Analytics tags will fire as a non logged in use. Alternatively change your Analytics configuration to track all users.

Alternative check the state of a users SK Analytics configuration and provide a more appropriate notice.

If all users are counted:
Notice: You're currently tracking all users within your Site Kit Analytics configuration. If you modify this configuration please test your experiment from optimize.google.com while not logged in to your site

If only non logged users are counted:
Notice: You're currently tracking only non logged in users within your Site Kit Analytics configuration. Please test your experiment from optimize.google.com while not logged in to your site

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

In the Optimize setup flow (https://imgur.com/a/exqE4Oa) and also in Optimize settings:

• If the GA settings for disabling Analytics tracking for certain users has a non-empty value (i.e. any user group, which definitely includes admins at this point, won't be tracked), show the following notice:
  Analytics is currently set to not track some logged-in users. If you're setting up or testing experiments on optimize.google.com, make sure you're not logged in to your WordPress site, otherwise the experiment will fail.

Implementation Brief

• Using assets/js/modules/optimize/components/settings/SettingsForm.js,
  • Query the Analytics data store via the getTrackingDisabled selector to check whether Analytics tracking is disabled for a set of users.
  • If the result of above call contains TRACKING_LOGGED_IN_USERS or TRACKING_CONTENT_CREATORS values from assets/js/modules/analytics/components/common/TrackingExclusionSwitches.js, display the text as per the AC, below the container ID section.
• Apply same logic to assets/js/modules/optimize/components/setup/SetupForm.js

Test Coverage

• No new tests to be added.

Visual Regression Changes

• N/A

QA Brief

• Connect Analytics and disable Analytics tracking for logged in users and/or content creators in the Analytics settings.
• Connect Optimize and check if there is the text displayed as per the AC, below the Optimize ID field.
• Disable tracking for logged in users and try to connect Optimize again. The text should no longer be displayed.
• Once Optimize is connected, go to the Optimize settings and see if the text is displayed.
• Disable Analytics tracking for logged in users
• Go back to Optimize settings. The text should no longer be displayed.

Changelog entry

• Add a notice to Optimize users warning them that Analytics requests are not being tracked for signed-in users.

[mxbclang]

As discussed in today's meeting, add notice in both Settings and during the setup flow, similar to the GA4 notice (https://imgur.com/2YdSKIC), and link the Learn More link to Optimize documentation.

[marrrmarrr]

It would be good to make this more context aware, so people don't have to wonder whether they're tracking all users or only the non-logged in users.

@bethanylang do you have a link to the relevant Optimize help doc? I did a bunch of searching in their help centre, but can't seem to find anything about non-logged in users..

[jamesozzie]

@marrrmarrr @bethanylang Happy to chime in here. I don't think there really is anything that provides sufficient context, I recall searching myself and this is as close to an official guide as I could find. There are some examples of users opening support topics in the Optmize Resource Hub where we could provide an appropriate notice (example).

As Optimize depends on Analytics firing when performing a check of an Optimize snippet maybe something along the lines of the below may be suitable:

To ensure you don't encounter the "Analytics pageview not detected" notice please logout from your website when performing a diagnostics test of your Optimize snippet.

There could then be a "Learn more" link to a website article describing how non tracking of logged in users or Ad Blockers can result in Analytics not reporting and therefore Optimize snippets checks failing.

[felixarntz]

@marrrmarrr I've updated the copy in the ACs combining yours and James's suggestion. What do you think? cc @jamesozzie

[marrrmarrr]

@felixarntz updated the text to be more clear about the consequences (the experiment will fail).

[felixarntz]

@marrrmarrr Thanks, that sounds solid to me. Moving to IB

[eugene-manuilov]

IB ✔️

[wpdarren]

QA Update: ✅

Verified:

Within Google Analytics settings, All logged-in users is enabled so all they will be excluded from Analytics. When I start to connect Optimizer the message appears Analytics is currently set to not track some logged-in users. If you’re setting up or testing experiments on optimize.google.com, make sure you’re not logged in to your WordPress site, otherwise the experiment will fail. as per the AC. I then connect the service and in settings the same message appears.

When the setting is disabled so that all users are tracked in Analytics, the message above does not appear when connecting the service or, within Site Kit settings.