Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: support artifact assertions in smoke tests #8044

Merged
merged 15 commits into from
Apr 11, 2019
Merged

tests: support artifact assertions in smoke tests #8044

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
7c1ed2d
Artifact assertions
mattzeunert Apr 7, 2019
1c16558
Revert "Artifact assertions"
mattzeunert Apr 7, 2019
97fe391
Wrap expectations in LHR
mattzeunert Apr 7, 2019
88d5a83
Update smoke test logic to support lhr/audit and artifact assertions
mattzeunert Apr 7, 2019
b499b52
Move artifacts to .tmp
mattzeunert Apr 8, 2019
e91d5cd
Make errorCode top level
mattzeunert Apr 8, 2019
810b37e
Remove empty line
mattzeunert Apr 8, 2019
ecc7091
Fix types and actually assert stuff
mattzeunert Apr 8, 2019
bc81d78
Style tweaks
mattzeunert Apr 8, 2019
fc85de0
Merge comparisons/assertions
mattzeunert Apr 8, 2019
d668e24
Read artifacts file as string instead of buffer
brendankenny Apr 10, 2019
9970b6a
PR feedback
mattzeunert Apr 10, 2019
6d4f206
ExpectedRunNERResult
mattzeunert Apr 10, 2019
6714136
category -> name
mattzeunert Apr 10, 2019
e5d34ea
expectedArtifacts var
mattzeunert Apr 11, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
414 changes: 208 additions & 206 deletions lighthouse-cli/test/smokehouse/a11y/expectations.js
View file
Open in desktop

Large diffs are not rendered by default.

220 changes: 112 additions & 108 deletions lighthouse-cli/test/smokehouse/byte-efficiency/expectations.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,134 +10,138 @@
*/
module.exports = [
{
requestedUrl: 'http://localhost:10200/byte-efficiency/tester.html',
finalUrl: 'http://localhost:10200/byte-efficiency/tester.html',
audits: {
'unminified-css': {
details: {
overallSavingsBytes: '>17000',
items: {
length: 2,
lhr: {
requestedUrl: 'http://localhost:10200/byte-efficiency/tester.html',
finalUrl: 'http://localhost:10200/byte-efficiency/tester.html',
audits: {
'unminified-css': {
details: {
overallSavingsBytes: '>17000',
items: {
length: 2,
},
},
},
},
'unminified-javascript': {
score: '<1',
details: {
overallSavingsBytes: '>45000',
overallSavingsMs: '>500',
items: [
{
url: 'http://localhost:10200/byte-efficiency/script.js',
wastedBytes: '46481 +/- 100',
wastedPercent: '87 +/- 5',
},
{
url: 'inline: \n function unusedFunction() {\n // Un...',
wastedBytes: '6581 +/- 100',
wastedPercent: '99.6 +/- 0.1',
},
{
url: 'inline: \n // Used block #1\n // FILLER DATA JUS...',
wastedBytes: '6559 +/- 100',
wastedPercent: 100,
'unminified-javascript': {
score: '<1',
details: {
overallSavingsBytes: '>45000',
overallSavingsMs: '>500',
items: [
{
url: 'http://localhost:10200/byte-efficiency/script.js',
wastedBytes: '46481 +/- 100',
wastedPercent: '87 +/- 5',
},
{
url: 'inline: \n function unusedFunction() {\n // Un...',
wastedBytes: '6581 +/- 100',
wastedPercent: '99.6 +/- 0.1',
},
{
url: 'inline: \n // Used block #1\n // FILLER DATA JUS...',
wastedBytes: '6559 +/- 100',
wastedPercent: 100,
},
],
},
},
'unused-css-rules': {
details: {
overallSavingsBytes: '>35000',
items: {
length: 2,
},
],
},
},
},
'unused-css-rules': {
details: {
overallSavingsBytes: '>35000',
items: {
length: 2,
'unused-javascript': {
score: '<1',
details: {
overallSavingsBytes: '>=25000',
overallSavingsMs: '>300',
items: {
length: 2,
},
},
},
},
'unused-javascript': {
score: '<1',
details: {
overallSavingsBytes: '>=25000',
overallSavingsMs: '>300',
items: {
length: 2,
'offscreen-images': {
details: {
items: [
{
url: /lighthouse-unoptimized.jpg$/,
}, {
url: /lighthouse-480x320.webp$/,
}, {
url: /lighthouse-480x320.webp\?invisible$/,
}, {
url: /large.svg$/,
},
],
},
},
},
'offscreen-images': {
details: {
items: [
{
url: /lighthouse-unoptimized.jpg$/,
}, {
url: /lighthouse-480x320.webp$/,
}, {
url: /lighthouse-480x320.webp\?invisible$/,
}, {
url: /large.svg$/,
'uses-webp-images': {
details: {
overallSavingsBytes: '>60000',
items: {
length: 4,
},
],
},
},
'uses-webp-images': {
details: {
overallSavingsBytes: '>60000',
items: {
length: 4,
},
},
},
'uses-text-compression': {
score: '<1',
details: {
overallSavingsMs: '>700',
overallSavingsBytes: '>50000',
items: {
length: 2,
'uses-text-compression': {
score: '<1',
details: {
overallSavingsMs: '>700',
overallSavingsBytes: '>50000',
items: {
length: 2,
},
},
},
},
'uses-optimized-images': {
details: {
overallSavingsBytes: '>10000',
items: {
length: 1,
'uses-optimized-images': {
details: {
overallSavingsBytes: '>10000',
items: {
length: 1,
},
},
},
},
'uses-responsive-images': {
displayValue: 'Potential savings of 75\xa0KB',
details: {
overallSavingsBytes: '>75000',
items: [
{wastedPercent: '<60'},
{wastedPercent: '<60'},
{wastedPercent: '<60'},
],
'uses-responsive-images': {
displayValue: 'Potential savings of 75\xa0KB',
details: {
overallSavingsBytes: '>75000',
items: [
{wastedPercent: '<60'},
{wastedPercent: '<60'},
{wastedPercent: '<60'},
],
},
},
},
},
},
{
requestedUrl: 'http://localhost:10200/byte-efficiency/gzip.html',
finalUrl: 'http://localhost:10200/byte-efficiency/gzip.html',
audits: {
'network-requests': {
details: {
items: [
{
url: 'http://localhost:10200/byte-efficiency/gzip.html',
},
{
url: 'http://localhost:10200/byte-efficiency/script.js?gzip=1',
transferSize: 1136,
resourceSize: 52997,
},
{
url: 'http://localhost:10200/byte-efficiency/script.js',
transferSize: 53181,
resourceSize: 52997,
},
],
lhr: {
requestedUrl: 'http://localhost:10200/byte-efficiency/gzip.html',
finalUrl: 'http://localhost:10200/byte-efficiency/gzip.html',
audits: {
'network-requests': {
details: {
items: [
{
url: 'http://localhost:10200/byte-efficiency/gzip.html',
},
{
url: 'http://localhost:10200/byte-efficiency/script.js?gzip=1',
transferSize: 1136,
resourceSize: 52997,
},
{
url: 'http://localhost:10200/byte-efficiency/script.js',
transferSize: 53181,
resourceSize: 52997,
},
],
},
},
},
},
Expand Down
Loading