Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky Jetpack E2E: when using AI Assistant block, validate only whether paragraph block exists. #81716

Merged
merged 1 commit into from
Sep 15, 2023

Conversation

worldomonation
Copy link
Contributor

@worldomonation worldomonation commented Sep 14, 2023

Related to #80730, #81580.

Proposed Changes

This PR is a follow-up to #81580 and updates the expectation in light of a flakeout by ChatGPT noted in p1694596605435499-slack-CDLH4C1UZ.

There will still be an initial attempt to verify the keyword(s) are present in the post, but if that fails, the block flow will fall back to simply verifying whether a paragraph block exists.

Unfortunately, because there is no reliable way to tell what generated the paragraph block in the published post, it is not possible to further narrow the selector.

Testing Instructions

Ensure the following build configurations are passing:

  • Calypso E2E (desktop)
  • Jetpack E2E Simple (mobile)
  • Jetpack E2E Simple (desktop)
  • Jetpack E2E AT (desktop)

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • https://wpcalypso.wordpress.com/devdocs/docs/testing/index.md for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-ajp-p2)?

- fall back on validating there is paragraph block
@github-actions
Copy link

github-actions bot commented Sep 14, 2023

@worldomonation worldomonation self-assigned this Sep 14, 2023
@matticbot
Copy link
Contributor

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

@worldomonation worldomonation marked this pull request as ready for review September 14, 2023 09:17
@worldomonation worldomonation requested a review from a team as a code owner September 14, 2023 09:17
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Sep 14, 2023
Copy link
Contributor

@dpasque dpasque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for sure the right call -- thanks for leaving such a helpful comment in the code! 💛

@worldomonation worldomonation merged commit f8eeb16 into trunk Sep 15, 2023
9 of 10 checks passed
@worldomonation worldomonation deleted the fix/jetpack-e2e-ai-assistant-expectation branch September 15, 2023 06:15
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Sep 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants