Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add E2E test to test checkout performance #4344

Merged
merged 29 commits into from
Jul 13, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
dac8544
Add a new performance test suite inside e2e folder
harriswong Jun 9, 2022
51936c0
Use page.evaulate() via puppeteer to output performance metrics
harriswong Jun 10, 2022
60767d1
Run the test 3 times and then calculate average.
harriswong Jun 13, 2022
d474d08
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 13, 2022
9a85b29
Add performance test when UPE is enabled
harriswong Jun 13, 2022
3198b96
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 16, 2022
d9b113c
Add performance test for WooPay without UPE
harriswong Jun 16, 2022
338ce18
Save performance metrics to a local file
harriswong Jun 16, 2022
7bc32db
Move logic out from the test spec
harriswong Jun 16, 2022
746274d
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 22, 2022
56cda30
Update woopay selector name from the dev tool
harriswong Jun 22, 2022
6d2c2ae
Add changelog
harriswong Jun 22, 2022
e8ae1ee
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 22, 2022
a326f98
Make performance its own npm command
harriswong Jun 22, 2022
a79bc45
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 23, 2022
51a10df
Update the selector used for enable/disable woopay
harriswong Jun 23, 2022
332c72f
Update element selector
harriswong Jun 27, 2022
6357f5a
Fix typo on "without"
harriswong Jun 28, 2022
a6474f0
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 28, 2022
4e5e2d0
Revert "Make performance its own npm command"
harriswong Jun 29, 2022
b4ca77c
Add a local env variable allowing us to skip performance tests
harriswong Jun 29, 2022
ac4b2ad
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 29, 2022
f71615a
Merge branch 'develop' into add/1021-performance-measure
harriswong Jun 30, 2022
5ef16e4
Revert "Add a local env variable allowing us to skip performance tests"
harriswong Jun 30, 2022
6240c57
Revert "Revert "Make performance its own npm command""
harriswong Jun 30, 2022
504274b
Merge branch 'develop' into add/1021-performance-measure
harriswong Jul 5, 2022
e93354d
Merge branch 'develop' into add/1021-performance-measure
harriswong Jul 11, 2022
4dbd8ed
Ignore performance tests via .gitignore
harriswong Jul 11, 2022
88e0e9a
Merge branch 'develop' into add/1021-performance-measure
harriswong Jul 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/e2e/config/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ const e2ePaths = {
wcpay: path.resolve( __dirname, '../specs/wcpay' ),
subscriptions: path.resolve( __dirname, '../specs/subscriptions' ),
blocks: path.resolve( __dirname, '../specs/blocks' ),
performance: path.resolve( __dirname, '../specs/performance' ),
};

// Allow E2E tests to run specific tests - wcpay, subscriptions, blocks, all (default).
Expand Down
47 changes: 47 additions & 0 deletions tests/e2e/specs/performance/payment-methods.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/**
* External dependencies
*/
import config from 'config';

/**
* Internal dependencies
*/
import { setupProductCheckout } from '../../utils/payments';
import { shopperWCP } from '../../utils';
import { getLoadingDurations } from '../../utils/performance';

describe( 'Checkout page performance', () => {
beforeEach( async () => {
await setupProductCheckout(
config.get( 'addresses.customer.billing' )
);
} );

afterAll( async () => {
// Clear the cart at the end so it's ready for another test
await shopperWCP.emptyCart();
} );

it( 'measures on page load', async () => {
await page.waitForSelector( '#payment_method_woocommerce_payments' );
const {
serverResponse,
firstPaint,
domContentLoaded,
loaded,
firstContentfulPaint,
firstBlock,
} = await getLoadingDurations();

await expect( page ).toMatch( 'Checkout' );
console.log(
'result',
serverResponse,
firstPaint,
domContentLoaded,
loaded,
firstContentfulPaint,
firstBlock
);
} );
} );
37 changes: 37 additions & 0 deletions tests/e2e/utils/performance.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
export async function getLoadingDurations() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the function from woocommerce-blocks https://github.com/woocommerce/woocommerce-blocks/blob/trunk/tests/utils/performance.ts. The differences are that I can't destruct the performance.getEntriesByType(..) assignment or use array prototype.find. I get the following errors:
ReferenceError: _slicedToArray2
ReferenceError: _find

It could be a puppeteer problem since these are used within page.evaulate(). Our puppeteer version is 2 and it relies on puppeteer-utils. Blocks uses jest-environment-puppeteer where its puppeteer is at 14. It doesn't seem to have these issues. I think it would be good to upgrade it to the newer puppeteer version.

In any case, I worked around the ReferenceError and this function works in this branch.

return await page.evaluate( () => {
const {
requestStart,
responseStart,
responseEnd,
domContentLoadedEventEnd,
loadEventEnd,
} = performance.getEntriesByType( 'navigation' )[ 0 ];
const paintTimings = performance.getEntriesByType( 'paint' );

let firstPaintTimings, firstContentfulPaintTimings;

paintTimings.forEach( ( item ) => {
if ( 'first-paint' === item.name ) {
firstPaintTimings = item;
}
if ( 'first-contentful-paint' === item.name ) {
firstContentfulPaintTimings = item;
}
} );

return {
// Server side metric.
serverResponse: responseStart - requestStart,
// For client side metrics, consider the end of the response (the
// browser receives the HTML) as the start time (0).
firstPaint: firstPaintTimings.startTime - responseEnd,
domContentLoaded: domContentLoadedEventEnd - responseEnd,
loaded: loadEventEnd - responseEnd,
firstContentfulPaint:
firstContentfulPaintTimings.startTime - responseEnd,
// This is evaluated right after Puppeteer found the block selector.
firstBlock: performance.now() - responseEnd,
};
} );
}