StepperTracking: Why calypso_signup_step_start for Domains is triggered less?

[escapemanuele]

p1732520994038579/1732293584.799889-slack-C073776NJ66

What explains fewer Domain Starts in Stepper?

• As seen in the metric result table pbmxuV-3OC-p2, we see more Signup Starts and more User Registration in the stepper, but a decline starts at Domain Starts.
• This decrease appears to drive the overall underperformance of the stepper flow. For details, refer to the “pbmxuV-3OC-p2” section.
• We want to verify if there are any potential issues for users reaching the Domain Step in Stepper, or if there are any tracking issues in /Start that could have inflated Domain Starts. I unfortunately don’t have further insights at the moment..

[github-actions]

OpenAI suggested the following labels for this issue:

• [Feature Group] Signup & Site Onboarding: The issue highlights concerns regarding user onboarding and interactions with the signup flow, specifically around the triggering of an event during the process.
• [Feature] Domain Management: This feature is relevant because the issue pertains to the step where users set up domains during the signup process.
• [Feature] Tracking Metrics & Monitoring: The issue is about analyzing user behavior related to the triggering of a specific event, which falls under tracking user interactions and events.
• [Feature] User Interaction & Engagement: The matter discusses user engagement during the initial setup phase of the signup process, particularly around the domains step.

[skim1220]

[2024-11-25]
As requested by @dzver , I queried the experiment participants assigned on or after 2024-10-24, and grouped them by browserfamily and devicefamily of their first assignment events. Here's the export.

I'm not sure if there are any obvious patterns, but it seems like mobile environments have a higher share of "no domain starts"... 🤔

[chriskmnds]

@ebuccelli there's a backUrl to domains from the plans step that's only utilised in the Start system. My local is pretty messed up right now with the TS refactors of the Plans step. Can you confirm if we have something relevant? We can then explore how to wire it into the typed form.

https://github.com/Automattic/wp-calypso/blob/trunk/client/signup/steps/plans/index.jsx#L338

[escapemanuele]

We found something that could explain the issue.

Look at this code:
Start

if ( isComingFromUseYourDomainStep ) {
				queryParams = {
					...props.queryParams,
					step: 'transfer-or-connect',
					initialQuery: previousStep?.siteUrl,
				};

				if (
					( 'onboarding' === flowName || 'onboarding-pm' === flowName ) &&
					undefined === previousStep?.providedDependencies?.domainItem
				) {
					backUrl = getStepUrl( flowName, 'domains' );
				}
			}

Stepper

if ( redirectedToUseMyDomain ) {
						if ( Object.keys( useMyDomainQueryParams ).length ) {
							// restore query params
							const useMyDomainURL = addQueryArgs( 'use-my-domain', useMyDomainQueryParams );
							return navigate( useMyDomainURL );
						}
						return navigate( 'use-my-domain' );
					}
					return navigate( 'domains' );

We have a different behavior in the two frameworks when Domains -> use my domain -> Plans -> Back, so when a user goes from Domain to Use my Domain to Plans and then decides to go back.

• Start: We are redirected to domains. We trigger `calypso_signup_step_start { step: domains }
• Stepper We are redirected to use-my-domain We trigger calypso_signup_step_start { step: use-my-domain }

We know that p1732722695480959/1732529755.704809-slack-C073776NJ66:

• 1500 users/day go back from Plans
• A total of 2600 users/day go through use-my-domain and reach the Plans step from there.

This may explain the discrepancy between Start and Stepper.

Stepper is probably doing the right thing, tracking use-my-domain.