Automated screenshots for Parabol demo

[NikAiyer]

This pull request attempts to resolve #3767 and uses the cy.screenshot() command to take pictures of the demo while it is running.

Steps to get running:

• Started server (either in dev or prod mode)
• Opened cypress (./node_modules/.bin/cypress open --project packages/cypress)
• Ran the following tests: demo_reflect.ts, demo_group.ts, demo_vote.ts and demo_discuss.ts
• Reviewed screenshots under packages/cypress/screenshots
• Confirmed whether they match what is desired for marketing purposes

[ackernaut]

Hello @NikAiyer

Implicitly, all code changes are subject to review, but the minimum requirement for reviewing a PR is to take steps provided by the PR author to get to the goal state.

The ‘tests’ for the reviewer are not to be confused with actual tests in code. These can be as simple as the steps for a user to try a new feature and see that it’s producing the desired goal state (e.g. 1) start a check-in meeting 2) pin an agenda item 3) after the meeting is over, the pinned items persist. In this case, it might look like steps to run cypress locally and review the demo screenshots that are produced, probably just:

Tests

• Ran cypress tests locally (which tests/commands?)
• Reviewed demo screenshots and confirmed they look right

Also, you could add specific items that you want reviewed e.g. a new pattern in code, a database migration, etc.

[NikAiyer]

Sorry I'll add that info to the pull request.

[ackernaut]

@NikAiyer no worries at all, it’s just helpful process to the team and the reviewer — having explicit, clear items to review and steps to check, thanks for all the attention here!

[ackernaut]

Hey @NikAiyer I’m running this locally, but for some reason I’m not seeing generated screenshots. I’ve tried the Cypress interface and yarn cypress — the only one I see is for me.ts — I’m probably doing something wrong?

[NikAiyer]

Hey @ackernaut, are all the tests running? Is it skipping over the demo tests?

[ackernaut]

Hello again, it’s working now that I’ve pulled your latest commits.

[ackernaut]

Feedback here:

• Nice work!
• I like that the left navigation is collapsed since each section in the 101 content is focusing on the main activity of each phase. The only things that may be highlighted in the left navigation, eventually, is the ability to change the meeting name, the ability to change the facilitator, and the ability to sort topics using drag and drop. We can deal with that later if it comes up.
• We should use a consistent size of 1280x720. Some of the shots are 1000x600. I assume that’s an easy fix?
• Is there a good way to grab a shot of the reflect phase and the lobby?
• In the vote screenshots, I can see a card in flight that was being dragged in the group phase. I think this probably happens when the phase advances quickly while the card is being dragged. I wonder if some delay after the last card is moved would help? This is probably not a deal-breaker but might seem confusing
• In the demo summary, I see ‘No New Tasks…’, 0 reflections, 0 topics, etc. is there a way to capture that after the demo is completed and stats/tasks are available?
• For the demo summary, I’m wondering if there’s a way to remove the section for creating a free account so that the visible section is a user with tasks. In the 101 content, I probably doctored that screenshot if I used the demo view. I think 1280x720 is an okay size here, too. Showing the CTA section is probably not a deal-breaker
• Here’s a link to the screenshots that were produced locally (shared zip via Google Drive)

[NikAiyer]

Thanks for the feedback! I think I've got most of the suggested changes working correctly. I think there might be a bug regarding the summary stats section so I'll try to fix that as well.

[NikAiyer]

Hey @ackernaut, I was able to figure out that issue with the summary stats in Cypress. I believe everything is now properly working.

[ackernaut]

hey @NikAiyer apologies for the delay here, I’ll give it another look today. Thanks for addressing the feedback!

[ackernaut]

These are looking good! Nice work!

(Edit: see takeaways at the end of this comment)

Reflect

Any of these Reflect screenshots look good (here showing add-stop-column-reflection.png). I like this one because it shows 2 prompts having been addressed, but there’s still progress to be had in the third prompt. The stretch-goal detail that would be nice here is to see the other cards being edited. The small card chits at the bottom would have the animated gradient highlight. Not a big deal if that’s hard to achieve. I don’t mind a mix of the sidebar being open or closed for some phases.

Group

I especially like visit-group.png when the card is flying midway over the columns. Sometimes the timing is different and it’s not obvious when a card is in flight. Compare these two:

Vote

I like after-voting.png from this batch. However, sometimes there’s a visible scrollbar that’s distracting to me. It’s not a deal-breaker. Compare these two:

Discuss

I really like reply-comment-discuss-task.png. This one looks perfect. You can see a bot task, a ‘you’ task, and a reply comment. The only strange thing is a person replying to their own task with ‘this is a great idea’ but nobody’s probably going to catch that. I like that the discussion topics are visible in the left navigation. The image publish-to-jira.png is a nice option, too.

Meeting Summary

The meeting summary image in the Discuss folder turned out well. I like that you removed the sign-up CTA section from the demo summary. It’s probably long to stick in an evergreen 101 article. I wonder if you could capture a 2nd image in a 1280x720 browser window where you only see the top portion and the first bit of the first user’s section.

Takeaways

• We’d need clear instructions for when and how to generate these for the roles that want to create them. 1) set up the repo locally 2) run commands 3) pull images from directories. I guess one could run the tests until they like a certain image, but pulling all the images from the same test batch would keep the avatars and bot names consistent.
• Remove "ignoreTestFiles": "demo_reflect.ts" from cypress.json so we get that test back in if it’s not already merged elsewhere
• A reminder of how these images are currently used. Nothing prevents us from shaping the content differently in the future and using images differently. Other images we need to think about are the Meeting Lobby and the Icebreaker views.
• Do you have a suggestion for how to handle the meeting lobby and icebreaker? Should we create these images manually for now? Here’s my first bad idea for a test script to write:
  1. user signs up
  2. user creates a second team called ‘Demo Team’
  3. user can navigate to the lobby
  4. user can select Retro type
  5. user can select Demo Team
  6. user can select Start Stop Continue (at this point capture a 1280x720 screenshot of the lobby view)
  7. user can start meeting
  8. user can see icebreaker (at this point capture an image)
• If there’s nothing further to do, then you can merge this!

[NikAiyer]

Hey @ackernaut, thanks for reviewing my changes!

1. For the instructions, is this something we want to add to the README.md? I can go ahead and add that now.
2. I believe "ignoreTestFiles": "demo_reflect.ts" is removed from the cypress.json now.
3. For the meeting lobby and the icebreaker, is there a way to access that from the demo itself? Otherwise, I think what you proposed would be a good solution. I'll go ahead and start creating that part.

[ackernaut]

For the instructions, is this something we want to add to the README.md? I can go ahead and add that now.

I’m not sure this would live in our main README for the repository. Is it safe to start a README in packages/cypress where we could add some minimal documentation for now? Later we could move it to an internal doc in Notion if that’s what’s best for the team. The first step is to write the steps for a less-technical person as a litmus test.

Re: lobby and icebreaker images, I don’t want to blow up the scope of this work. I’ll let you judge if you want to shoot for that stretch goal.

Also, I seem to recall us trying to minimize the instances of Cypress properties in the actual app components. I’m not up to speed on what the minimal need is there, as I assume sometimes it’s necessary. Was there a rule of thumb that you and Matt agreed on?

[ackernaut]

I realize you probably aren’t able to merge, so let me know what additional work you want to do here, and I can look at this again when you’re ready. Thanks!

[NikAiyer]

Re: lobby and icebreaker images, I don’t want to blow up the scope of this work. I’ll let you judge if you want to shoot for that stretch goal.

Hmm, now that you mention it, it might be better to have that in a separate pull request. Just to keep things simpler. I'll get back to that when I finish the other issues I'm working on.

Also, I seem to recall us trying to minimize the instances of Cypress properties in the actual app components. I’m not up to speed on what the minimal need is there, as I assume sometimes it’s necessary. Was there a rule of thumb that you and Matt agreed on?

There wasn't an exact rule regarding this. Basically he told me to try to limit the amount of Cypress data-cy properties I used whenever I could.

Here's what he said:

Moving forward, a fun challenge would be to try and write tests without modifying the application code (e.g. injecting data-cy). Although it isn't always possible, this is usually preferable because it means tests are less tightly coupled to the app internals & it's closer to how a human would read the page.

[avivapinchas]

This overall LGTM! Agree with all of @ackernaut's feedback.

Some general feedback:

• I don't think we'll use screenshots where the Tips menu is open - can we focus only on capturing moments when that isn't open?
• For the Discuss phase, I'd love to see a group with multiple reflections, I think that shows the richness of the tool better. Is that possible?
• Can we add parabol-retrospective- to the start of all the file names? Including keywords in file names is a practice for getting those images found and upping our SEO value for key terms, and if we have that automatically here, it'll reduce the human error potential.

We’d need clear instructions for when and how to generate these for the roles that want to create them. 1) set up the repo locally 2) run commands 3) pull images from directories. I guess one could run the tests until they like a certain image, but pulling all the images from the same test batch would keep the avatars and bot names consistent.

As someone who doesn't currently have to the repo setup locally, I'd value a Loom video & Notion doc with instructions :)

Is there any way to ensure more consistency in the images, so we're not needing to pull multiple times? Happy to go with this route and then see if the tension builds.

I don’t mind a mix of the sidebar being open or closed for some phases.

Is it possible to have both options always created for all screenshots? We'd potentially use those types of images differently.

[ackernaut]

@NikAiyer Re: separate PR, I like it! Break the work down into future pieces. Re: data-cy props, thanks for sharing the extra context from your conversation with Matt, sounds good!

@avivapinchas Thanks for the extra input here. Re: setting up the repo, we could probably break this down into 2 parts and make it as friendly as possible for folks who are going wider over deeper role-wise. 1) set up the repo 2) [do the thing locally you need—in this case running tests for screenshots] +1 for Loom and Notion. I can help with this once we get if farther.

Re: sidebar open and closed, I think that @NikAiyer did include both. There are more images than what I shared as output from the tests, and they are named as different actions representing different movements in the linear progression of the meeting. Here’s a zip of the last batch I ran, shared from my drive.

[NikAiyer]

@avivapinchas Thanks for the feedback! I believe I was able to get everything you wanted into the testing. For the consistency of the screenshots, I was able to get most of them to a consistent level. The only one I'm sort of iffy about is the Group Phase, which might need some tweaks to get better images.
@ackernaut Would you be able to take another look at it and see if it still looks good?

[ackernaut]

Looking now! On Friday, it would hang up when trying to pull latest (after the fact, I now wonder if this was due to using a VPN). Today it’s working, but my VPN is off.

[ackernaut]

@NikAiyer this is looking good!!

• ✅   I see the active chits of other participants in the Reflect phase
• ✅   I see some interesting in-flight drag states during Group
• ✅   I see that the Discuss phase shots have 3 reflections from 2 prompts
• ✅   I see the new file naming convention
• 👻   I like the 1280x720 crop of the Summary but it doesn’t have reflections, topics, or tasks. Is this due to timing?

Here’s the batch that was generated (shared zip via Google Drive)

[NikAiyer]

Hmm yeah I think that might be due to a timing thing. Maybe it doesn't get voted on in time so topics don't show up. I'll add a longer wait there to ensure it gets done.

Edit: Looks like I forgot to reload the page before I started taking pictures. Sometimes it'll render the page before the cache is restored.

[NikAiyer]

This pull request also fixes issue #3799, specifically the issue with some tests failing in CI.

[ackernaut]

@NikAiyer looks like that worked as the demo summary now has data. The shots are looking good. There’s one annoying detail with the vote shots. Most of them have a visible scrollbar. Do you know what triggers the scrollbar? Is it the timing after the vote is made?

[NikAiyer]

I think that happens when there are a lot of reflections in one category. I'll let them get grouped a little bit so that doesn't show up.

[ackernaut]

Good to know, timing sure is nuanced. Thanks for the attention to these details. I’m looking again locally.

[ackernaut]

👏 thanks @NikAiyer for the many iterations, merging now!