Task: Implement additional Jetpack tests.

[worldomonation]

Details

Taking into account outcome for pdWQjU-rL-p2, begin implementing new Jetpack E2E tests from the highest priority & easiest to implement.

Checklist

Task list taken from the Jetpack - Calypso test coverage mapping spreadsheet.

No response

Must Cover Tasks

Beta Give feedback

1. Jetpack Stats - navigate to /stats, navigate. Jetpack E2E: increase coverage of the existing Stats spec. #80891
2. Jetpack Social - configure sharing buttons in Tools > Marketing > Sharing Buttons Jetpack E2E: add Social Connection spec using Tumblr. #81073
3. Jetpack Social - sharing buttons on published content Jetpack E2E: add step to check Social Sharing icons on published page. #80897
4. Blocks/Design - Layout (It appears it was already covered in blocks__core.ts)
5. Jetpack/Social - toggle on/off Blaze. Jetpack E2E: add spec for Jetpack Blaze. #81252
6. Jetpack/Traffic - SEO preview Jetpack E2E: expand the Marketing: SEO spec. #81199
7. Blocks/Earn - Ad Jetpack E2E: new flow for the Ad block. #81127
8. Blocks/Earn - Payment Button Blocked by Payment Button Block: Toolbar inserter is disabled because there are no child blocks supported jetpack#30785
9. Blocks/Earn - Donation Form -- this seems to be done!
10. Blocks/Misc - AI Assistant Jetpack E2E: add flow for AI Assistant block. #80899 Jetpack E2E: update expectations for the AI Assistant flow. #81065
11. Blocks/Misc - Writing Prompt Jetpack E2E: add Writing Prompt spec. #80957
12. Blocks/Media - VIdeoPress Jetpack E2E: add spec for standalone VideoPress block. #81026
13. Blocks/Form - Jetpack Button Update Forms block smoke testing for broader coverage #81044
14. Blocks/Form - Name Update Forms block smoke testing for broader coverage #81044
15. Blocks/Form - Email Update Forms block smoke testing for broader coverage #81044
16. Blocks/Form - Multi-line Update Forms block smoke testing for broader coverage #81044
17. Form block submission Add full form submission e2e test #81301
18. Form block AI configured Add forms AI e2e block smoke test #81298
19. Jetpack Search Add Jetpack Instant Search E2E Test #80991
20. Blocks/Earn - Paywall Jetpack E2E: Paywall block. #81256
21. Estimated to be difficult below --------------------------------------------------------
22. Jetpack Newsletter - Subscription Management Jetpack E2E: add coverage for the newsletter subscription & removal flow. #81437
23. Jetpack Newsletter - Newsletter settings @worldomonation
24. Jetpack Newsletter - editor sidebar access settings
25. Blocks/Earn - Payments Jetpack E2E: enable Paid Contents block for AT environments. #81024
26. Blocks/Earn - Donations Jetpack E2E: add Donations Form flow. #81010
27. Jetpack - smoke test pages in the dashboard Jetpack E2E: smoke test of the dashboard and settings pages. #81443

Important Tasks

Beta Give feedback

1. Blocks/Embed - Related Posts
2. Blocks/Misc - GIF
3. Blocks/Misc - Image Compare
4. Blocks/Misc - Map
5. Blocks/Misc - Story
6. Blocks/Misc - Markdown
7. Estimated to be difficult below --------------------------------------------------------
8. Jetpack Social - Set up connections
9. Jetpack Social - Auto-share to social on publish
10. Jetpack Social - Social preview
11. Can we check for the Stats pixel.wp.com request on the front-end of WPCOM sites somehow? Jetpack E2E: add test for the stats tracking pixel. #82913

[dpasque]

I'll take point on Forms. Asked a question below about designing a potential dedicated E2E spec (as opposed to block smoke test) that focuses on Forms. Let me know what you think!

pdWQjU-rL-p2#comment-537

[dpasque]

After some more designing today, I had the idea to split up up the block smoke testing for forms and the full flow.

This means, two specs:

• blocks__jetpack-forms, which will itself include three flows...
• • One where you add a pre-canned Contact Form
• • One where you add all the possible fields
• • One where you transform a Form block using the custom pattern options.
• jetpack__full-form-flow, which will use a very basic form (like RSVP) and go through submission and validation.

[worldomonation]

⬆️ Sounds good to me.

One nitpick suggestion: what if the full spec is named like jetpack__rsvp-flow?

[dpasque]

One nitpick suggestion: what if the full spec is named like jetpack__rsvp-flow?

Oh that's fine too! Although, I'm not sure I want to necessarily couple it tightly to a specific type of form, because we might change that based on whatever is easiest --- the focus isn't the form content, but that you can submit forms on webpages and see the responses in Admin!

[dpasque]

Adding some test plan mapping notes / ideas:

Jetpack Social - Social preview

This test actually shouldn't be that hard, and would be a good test! We could just...

1. Make a new post
2. Open the Jetpack Sidebar
3. Add an "SEO Title"
4. Launch the preview and validate one or two social media previews.

Jetpack Newsletter - editor sidebar access settings

I added this because I think this is a big money maker for us, and I think any coverage here could be good.
We could set up a paid newsletter plan on these accounts, but we don't necessarily need paid subscribers. Even a test that sets it to paid subscribers and makes sure that any ol' person on the internet can't view the post. We can maybe also have a dedicated user that has a free subscription on all of these sites.

Jetpack Social - Auto-share to social on publish

It looks like the simple sites may have a tumblr connection already set up, so we might be able to use that to test a very simple/basic version of this case!

[worldomonation]

Can we check for the Stats pixel.wp.com request on the front-end of WPCOM sites somehow?

It looks like we have a tracking pixel and its details are here: p7pQDF-5HJ-p2.

Basically, we want to see if the request for a tracking pixel (at pixel.wp.com) succeeded, and that should be enough to verify that at least the frontend loaded the tracking pixel. From our last Jetpack <> KitKat meeting, I'm pretty sure it's going to be difficult to then query whether the tracked stat made its way back to the database.

@jeherve does the above sound okay?

[jeherve]

Yes, I think checking for the https://pixel.wp.com/g.gif should be enough. You would want to check for specific parameters with that request, since other a8c tracking solutions such as Tracks can also use g.gif but with different parameters.

Checking for the blog and post parameters should be good to ensure this is a Stats tracking pixel.

[worldomonation]

Yes, I think checking for the https://pixel.wp.com/g.gif should be enough. You would want to check for specific parameters with that request, since other a8c tracking solutions such as Tracks can also use g.gif but with different parameters.

Checking for the blog and post parameters should be good to ensure this is a Stats tracking pixel.

Edit: I figured it out for AT sites - the tracking pixels don't load if viewing the post as the publishing user, unlike on Simple where it does. Solution is to view the post as a logged out user.

Oddly, it seems on the AT blogs the tracking pixel isn't being loaded. These are the only requests I see:

• https://pixel.wp.com/g.gif?v=wpcom-no-pv&x_notes-empty-message=all_shown&baba=0.24164228172509428
• https://pixel.wp.com/g.gif?x_stats-initial-visibility=visible&v=wpcom-no-pv&rand=0.6719089094645461
• https://pixel.wp.com/g.gif?v=wpcom-no-pv&x_notes-menu-impressions=zero-jetpack&baba=0.5357180544050155
• https://pixel.wp.com/g.gif?x_stats-initial-visibility=visible&v=wpcom-no-pv&rand=0.8408110455779745

@jeherve

[jeherve]

it seems on the AT blogs the tracking pixel isn't being loaded.

Is the Stats module enabled on those sites?

Solution is to view the post as a logged out user.

Another option could be to change the Stats options to allow tracking logged in users as well, but that may be more steps.

[worldomonation]

it seems on the AT blogs the tracking pixel isn't being loaded.

Is the Stats module enabled on those sites?

Yes, it is enabled. The issue as I found out is a behavior difference between AT and Simple sites.

• Simple: loads tracking pixels even if viewing as the publishing user.
• AT: does not load tracking pixel if viewing as publishing user.

Solution is to view the post as a logged out user.

Another option could be to change the Stats options to allow tracking logged in users as well, but that may be more steps.

To clarify, the tracking pixel is being loaded as logged out user on AT and Simple. The only time the tracking pixel isn't loading on AT is when viewing the published post as the publishing user.

In any case, we want to view the post not as the publishing user.

[dpasque]

Going to go ahead and close this now, as we are at the end of our window for implementing these test. We've gotten the vast vast vast majority of the items here, and increased the coverage a lot!