Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make embed tests work with single playwright config #7781

Merged
merged 2 commits into from
Mar 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions packages/embeds/embed-core/playwright/lib/testUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ async function selectFirstAvailableTimeSlotNextMonth(frame: Frame, page: Page) {

// Waiting for full month increment
await frame.waitForTimeout(1000);
expect(await page.screenshot()).toMatchSnapshot("availability-page-2.png");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented these snapshots, they were never compared. But I plan to work on logic to compare them.

// expect(await page.screenshot()).toMatchSnapshot("availability-page-2.png");
// TODO: Find out why the first day is always booked on tests
await frame.locator('[data-testid="day"][data-disabled="false"]').nth(1).click();
await frame.click('[data-testid="time"]');
Expand All @@ -104,15 +104,15 @@ export async function bookFirstEvent(username: string, frame: Frame, page: Page)
// This doesn't seem to be replicable with the speed of a person, only during automation.
// It would also allow correct snapshot to be taken for current month.
await frame.waitForTimeout(1000);
expect(await page.screenshot()).toMatchSnapshot("availability-page-1.png");
// expect(await page.screenshot()).toMatchSnapshot("availability-page-1.png");
const eventSlug = new URL(frame.url()).pathname;
await selectFirstAvailableTimeSlotNextMonth(frame, page);
await frame.waitForNavigation({
url(url) {
return url.pathname.includes(`/${username}/book`);
},
});
expect(await page.screenshot()).toMatchSnapshot("booking-page.png");
// expect(await page.screenshot()).toMatchSnapshot("booking-page.png");
// --- fill form
await frame.fill('[name="name"]', "Embed User");
await frame.fill('[name="email"]', "embed-user@example.com");
Expand All @@ -123,7 +123,7 @@ export async function bookFirstEvent(username: string, frame: Frame, page: Page)

// Make sure we're navigated to the success page
await expect(frame.locator("[data-testid=success-page]")).toBeVisible();
expect(await page.screenshot()).toMatchSnapshot("success-page.png");
// expect(await page.screenshot()).toMatchSnapshot("success-page.png");

//NOTE: frame.click('body') won't work here. Because the way it works, it clicks on the center of the body tag which is an element inside the popup view and that won't close the popup
await frame.evaluate(() => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ test.describe("Popup Tests", () => {
await expect(embedIframe).toBeEmbedCalLink(calNamespace, getActionFiredDetails, {
pathname: "/free",
});
expect(await page.screenshot()).toMatchSnapshot("event-types-list.png");
// expect(await page.screenshot()).toMatchSnapshot("event-types-list.png");
if (!embedIframe) {
throw new Error("Embed iframe not found");
}
Expand Down
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed snapshots till we fix snapshot comparison

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
2 changes: 1 addition & 1 deletion packages/embeds/embed-core/playwright/tests/inline.e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ test("Inline Iframe - Configured with Dark Theme", async ({
theme: "dark",
},
});
expect(await page.screenshot()).toMatchSnapshot("event-types-list.png");
// expect(await page.screenshot()).toMatchSnapshot("event-types-list.png");
if (!embedIframe) {
throw new Error("Embed iframe not found");
}
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
2 changes: 1 addition & 1 deletion packages/embeds/embed-react/playwright/tests/basic.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@ test("Inline Usage Snapshot", async ({ page, getActionFiredDetails, addEmbedList
theme: "dark",
},
});
expect(await page.screenshot()).toMatchSnapshot("react-component-inline.png");
// expect(await page.screenshot()).toMatchSnapshot("react-component-inline.png");
});
Diff not rendered.
Diff not rendered.
Diff not rendered.
Diff not rendered.
6 changes: 3 additions & 3 deletions playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ if (IS_EMBED_REACT_TEST) {

const config: PlaywrightTestConfig = {
forbidOnly: !!process.env.CI,
retries: 2,
retries: process.env.CI ? 2 : 0,
Copy link
Member Author

@hariombalhara hariombalhara Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like tests to not retry locally so that the failure reason can be seen quickly. Also, I believe it also allows a flake to be detected easily.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not disable retires altogether so tests fail fast on CI?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because on CI it's too flaky to run without retries due to parallel runners. On dev it's more likely you are running only a single test or a few.

workers: os.cpus().length,
timeout: DEFAULT_TEST_TIMEOUT,
maxFailures: headless ? 10 : undefined,
Expand Down Expand Up @@ -102,13 +102,13 @@ const config: PlaywrightTestConfig = {
name: "@calcom/embed-core",
testDir: "./packages/embeds/embed-core/",
testMatch: /.*\.(e2e|test)\.tsx?/,
use: { ...devices["Desktop Chrome"] },
use: { ...devices["Desktop Chrome"], baseURL: "http://localhost:3100/" },
},
{
name: "@calcom/embed-react",
testDir: "./packages/embeds/embed-react/",
testMatch: /.*\.(e2e|test)\.tsx?/,
use: { ...devices["Desktop Chrome"] },
use: { ...devices["Desktop Chrome"], baseURL: "http://localhost:3101/" },
},
{
name: "@calcom/embed-core--firefox",
Expand Down