Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map reports to rows in aggregate csv report [WIP] #133

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 39 additions & 25 deletions lighthouse_runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ const path = require('path');
const { URL } = require('url');
const open = require('open');
const os = require('os');
const csvStringify = require('csv-stringify/lib/sync');
const { reportToRow, reportToRowHeaders } = require('./reportToRow');

let autoOpen = false;
let port;
let outputMode;
Expand Down Expand Up @@ -100,17 +103,28 @@ const processResults = (processObj) => {
} else {
filePath = path.join(tempFilePath, replacedUrl + '.mobile.report.' + opts.output);
}
// // @TODO ASYNC IS BETTER BUT WE ARENT CURRENTLY AWAITING THE PROMISE PROPERLY
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the original version, I don't think the promise was being properly awaited so I was getting errors when my new code tried to interact with the last individual report. I haven't had time to study it closely, but I think there was a race condition that just didn't matter until I added more moving peaces down-stream. I could be wrong, but switching to the synchronous version of writeFile fixed the problem immediately.

// https://stackoverflow.com/questions/34811222/writefile-no-such-file-or-directory
fs.writeFile(filePath, report, {
// fs.writeFile(filePath, report, {
// encoding: 'utf-8'
// }, (err) => {
// if (err) throw err;
// if (opts.emulatedFormFactor && opts.emulatedFormFactor === 'desktop') {
// console.log('Wrote desktop report: ', currentUrl, 'at: ', tempFilePath);
// } else {
// console.log('Wrote mobile report: ', currentUrl, 'at: ', tempFilePath);
// }
// });

fs.writeFileSync(filePath, report, {
encoding: 'utf-8'
}, (err) => {
if (err) throw err;
if (opts.emulatedFormFactor && opts.emulatedFormFactor === 'desktop') {
console.log('Wrote desktop report: ', currentUrl, 'at: ', tempFilePath);
} else {
console.log('Wrote mobile report: ', currentUrl, 'at: ', tempFilePath);
}
});
if (opts.emulatedFormFactor && opts.emulatedFormFactor === 'desktop') {
console.log('Wrote desktop report: ', currentUrl, 'at: ', tempFilePath);
} else {
console.log('Wrote mobile report: ', currentUrl, 'at: ', tempFilePath);
}

};
/**
* Helper function to queue up async promises.
Expand Down Expand Up @@ -248,32 +262,33 @@ const aggregateCSVReports = (directoryPath) => {
let mobileAggregatePath = path.join(directoryPath, mobileAggregateReportName);
let desktopWriteStream = fs.createWriteStream(desktopAggregatePath, { flags: 'a' });
let mobileWriteStream = fs.createWriteStream(mobileAggregatePath, { flags: 'a' });
let desktopCounter = 0;
let mobileCounter = 0;

let desktopRows = [
reportToRowHeaders
];
let mobileRows = [
reportToRowHeaders
];

try {
files.forEach(fileName => {
if (fileName !== desktopAggregateReportName && fileName !== mobileAggregateReportName) {
let filePath = path.join(directoryPath, fileName);
let fileContents = fs.readFileSync(filePath, { encoding: 'utf-8' });
console.log(`Bundling ${fileName} into aggregated report`);
const newRow = reportToRow(fileContents);
if (fileName.includes('.desktop')) {
if (desktopCounter === 0) {
desktopWriteStream.write(fileContents + '\n');
desktopCounter++;
} else {
let newContents = fileContents.split('\n').slice(1).join('\n');
desktopWriteStream.write(newContents + '\n');
}
desktopRows.push(newRow);
} else if (fileName.includes('.mobile')) {
if (mobileCounter === 0) {
mobileWriteStream.write(fileContents + '\n');
mobileCounter++;
} else {
let newContents = fileContents.split('\n').slice(1).join('\n');
mobileWriteStream.write(newContents + '\n');
}
mobileRows.push(newRow);
}
}
});
desktopWriteStream.write(csvStringify(desktopRows));
console.log('Wrote desktop aggregate report');
mobileWriteStream.write(csvStringify(mobileRows));
console.log('Wrote mobile aggregate report');

}
catch (e) {
console.error(e);
Expand All @@ -282,7 +297,6 @@ const aggregateCSVReports = (directoryPath) => {
return true;
}


/**
* Opens generated reports in your preferred browser as an explorable list
* @param {Number} port Port used by Express
Expand Down
39 changes: 39 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"dependencies": {
"chrome-launcher": "^0.13.0",
"commander": "^5.0.0",
"csv": "^5.3.2",
"express": "^4.17.1",
"lighthouse": "^6.0.0",
"open": "7.0.4",
Expand Down
174 changes: 174 additions & 0 deletions reportToRow.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
const csvParse = require('csv-parse/lib/sync');

const reportToRowHeaders = [
"Requested URL",
"Final URL",
"Performance: First Contentful Paint (numeric)",
"Performance: Speed Index (numeric)",
"Performance: Largest Contentful Paint (numeric)",
"Performance: Time to Interactive (numeric)",
"Performance: Total Blocking Time (numeric)",
"Performance: Cumulative Layout Shift (numeric)",
"Performance: First CPU Idle (numeric)",
"Performance: Max Potential First Input Delay (numeric)",
"Performance: First Meaningful Paint (numeric)",
"Performance: Estimated Input Latency (numeric)",
"Performance: Eliminate render-blocking resources (numeric)",
"Performance: Properly size images (numeric)",
"Performance: Defer offscreen images (numeric)",
"Performance: Minify CSS (numeric)",
"Performance: Minify JavaScript (numeric)",
"Performance: Remove unused CSS (numeric)",
"Performance: Remove unused JavaScript (numeric)",
"Performance: Efficiently encode images (numeric)",
"Performance: Serve images in next-gen formats (numeric)",
"Performance: Enable text compression (numeric)",
"Performance: Preconnect to required origins (numeric)",
"Performance: Initial server response time was short (binary)",
"Performance: Avoid multiple page redirects (numeric)",
"Performance: Preload key requests (numeric)",
"Performance: Use video formats for animated content (numeric)",
"Performance: Avoids enormous network payloads (numeric)",
"Performance: Serve static assets with an efficient cache policy (numeric)",
"Performance: Avoids an excessive DOM size (numeric)",
"Performance: Avoid chaining critical requests (informative)",
"Performance: User Timing marks and measures (notApplicable)",
"Performance: JavaScript execution time (numeric)",
"Performance: Minimizes main-thread work (numeric)",
"Performance: Ensure text remains visible during webfont load (binary)",
"Performance: Performance budget (notApplicable)",
"Performance: Timing budget (notApplicable)",
"Performance: Keep request counts low and transfer sizes small (informative)",
"Performance: Minimize third-party usage (binary)",
"Performance: Largest Contentful Paint element (informative)",
"Performance: Avoid large layout shifts (informative)",
"Performance: Uses HTTP/2 for its own resources (binary)",
"Performance: Uses passive listeners to improve scrolling performance (binary)",
"Performance: Avoids `document.write()` (binary)",
"Performance: Avoid long main-thread tasks (informative)",
"Performance: Network Requests (informative)",
"Performance: Network Round Trip Times (informative)",
"Performance: Server Backend Latencies (informative)",
"Performance: Tasks (informative)",
"Performance: Diagnostics (informative)",
"Performance: Metrics (informative)",
"Performance: Screenshot Thumbnails (informative)",
"Performance: Final Screenshot (informative)",
"Accessibility: `[accesskey]` values are unique (notApplicable)",
"Accessibility: `[aria-*]` attributes match their roles (notApplicable)",
"Accessibility: `[aria-hidden='true']` is not present on the document `<body>` (binary)",
"Accessibility: `[aria-hidden='true]` elements do not contain focusable descendents (notApplicable)",
"Accessibility: ARIA input fields have accessible names (notApplicable)",
"Accessibility: `[role]`s have all required `[aria-*]` attributes (notApplicable)",
"Accessibility: Elements with an ARIA `[role]` that require children to contain a specific `[role]` have all required children. (notApplicable)",
"Accessibility: `[role]`s are contained by their required parent element (notApplicable)",
"Accessibility: `[role]` values are valid (notApplicable)",
"Accessibility: ARIA toggle fields have accessible names (notApplicable)",
"Accessibility: `[aria-*]` attributes have valid values (notApplicable)",
"Accessibility: `[aria-*]` attributes are valid and not misspelled (notApplicable)",
"Accessibility: Buttons have an accessible name (notApplicable)",
"Accessibility: The page contains a heading, skip link, or landmark region (binary)",
"Accessibility: Background and foreground colors have a sufficient contrast ratio (binary)",
"Accessibility: `<dl>`'s contain only properly-ordered `<dt>` and `<dd>` groups, `<script>`, `<template>` or `<div>` elements. (notApplicable)",
"Accessibility: Definition list items are wrapped in `<dl>` elements (notApplicable)",
"Accessibility: Document has a `<title>` element (binary)",
"Accessibility: `[id]` attributes on active, focusable elements are unique (notApplicable)",
"Accessibility: ARIA IDs are unique (notApplicable)",
"Accessibility: No form fields have multiple labels (notApplicable)",
"Accessibility: `<frame>` or `<iframe>` elements have a title (notApplicable)",
"Accessibility: Heading elements appear in a sequentially-descending order (binary)",
"Accessibility: `<html>` element has a `[lang]` attribute (binary)",
"Accessibility: `<html>` element has a valid value for its `[lang]` attribute (binary)",
"Accessibility: Image elements have `[alt]` attributes (notApplicable)",
"Accessibility: `<input type='image'>` elements have `[alt]` text (notApplicable)",
"Accessibility: Form elements have associated labels (notApplicable)",
"Accessibility: Presentational `<table>` elements avoid using `<th>`, `<caption>` or the `[summary]` attribute. (notApplicable)",
"Accessibility: Links have a discernible name (binary)",
"Accessibility: Lists contain only `<li>` elements and script supporting elements (`<script>` and `<template>`). (binary)",
"Accessibility: List items (`<li>`) are contained within `<ul>` or `<ol>` parent elements (binary)",
"Accessibility: The document does not use `<meta http-equiv='refresh'>` (notApplicable)",
"Accessibility: `[user-scalable='no']` is not used in the `<meta name='viewport'>` element and the `[maximum-scale]` attribute is not less than 5. (binary)",
"Accessibility: `<object>` elements have `[alt]` text (notApplicable)",
"Accessibility: No element has a `[tabindex]` value greater than 0 (notApplicable)",
"Accessibility: Cells in a `<table>` element that use the `[headers]` attribute refer to table cells within the same table. (notApplicable)",
"Accessibility: `<th>` elements and elements with `[role='columnheader'/'rowheader']` have data cells they describe. (notApplicable)",
"Accessibility: `[lang]` attributes have a valid value (notApplicable)",
"Accessibility: `<video>` elements contain a `<track>` element with `[kind='captions']` (notApplicable)",
"Accessibility: `<video>` elements contain a `<track>` element with `[kind='description']` (notApplicable)",
"Accessibility: The page has a logical tab order (manual)",
"Accessibility: Interactive controls are keyboard focusable (manual)",
"Accessibility: Interactive elements indicate their purpose and state (manual)",
"Accessibility: The user's focus is directed to new content added to the page (manual)",
"Accessibility: User focus is not accidentally trapped in a region (manual)",
"Accessibility: Custom controls have associated labels (manual)",
"Accessibility: Custom controls have ARIA roles (manual)",
"Accessibility: Visual order on the page follows DOM order (manual)",
"Accessibility: Offscreen content is hidden from assistive technology (manual)",
"Accessibility: HTML5 landmark elements are used to improve navigation (manual)",
"Best Practices: Uses HTTPS (binary)",
"Best Practices: Links to cross-origin destinations are safe (binary)",
"Best Practices: Avoids requesting the geolocation permission on page load (binary)",
"Best Practices: Avoids requesting the notification permission on page load (binary)",
"Best Practices: Avoids front-end JavaScript libraries with known security vulnerabilities (binary)",
"Best Practices: Allows users to paste into password fields (binary)",
"Best Practices: Displays images with correct aspect ratio (binary)",
"Best Practices: Serves images with appropriate resolution (binary)",
"Best Practices: Page has the HTML doctype (binary)",
"Best Practices: Properly defines charset (binary)",
"Best Practices: Avoids Application Cache (binary)",
"Best Practices: Detected JavaScript libraries (binary)",
"Best Practices: Avoids deprecated APIs (binary)",
"Best Practices: No browser errors logged to the console (binary)",
"SEO: Has a `<meta name='viewport'>` tag with `width` or `initial-scale` (binary)",
"SEO: Document has a `<title>` element (binary)",
"SEO: Document has a meta description (binary)",
"SEO: Page has successful HTTP status code (binary)",
"SEO: Links have descriptive text (binary)",
"SEO: Links are crawlable (binary)",
"SEO: Page isn’t blocked from indexing (binary)",
"SEO: robots.txt is valid (notApplicable)",
"SEO: Image elements have `[alt]` attributes (notApplicable)",
"SEO: Document has a valid `hreflang` (binary)",
"SEO: Document has a valid `rel=canonical` (notApplicable)",
"SEO: Document uses legible font sizes (notApplicable)",
"SEO: Document avoids plugins (binary)",
"SEO: Tap targets are sized appropriately (notApplicable)",
"SEO: Structured data is valid (manual)",
"Progressive Web App: Page load is fast enough on mobile networks (binary)",
"Progressive Web App: Current page does not respond with a 200 when offline (binary)",
"Progressive Web App: `start_url` does not respond with a 200 when offline (binary)",
"Progressive Web App: Uses HTTPS (binary)",
"Progressive Web App: Does not register a service worker that controls page and `start_url` (binary)",
"Progressive Web App: Web app manifest does not meet the installability requirements (binary)",
"Progressive Web App: Redirects HTTP traffic to HTTPS (binary)",
"Progressive Web App: Is not configured for a custom splash screen (binary)",
"Progressive Web App: Does not set a theme color for the address bar. (binary)",
"Progressive Web App: Content is sized correctly for the viewport (notApplicable)",
"Progressive Web App: Has a `<meta name='viewport'>` tag with `width` or `initial-scale` (binary)",
"Progressive Web App: Contains some content when JavaScript is not available (binary)",
"Progressive Web App: Does not provide a valid `apple-touch-icon` (binary)",
"Progressive Web App: Manifest doesn't have a maskable icon (binary)",
"Progressive Web App: Site works cross-browser (manual)",
"Progressive Web App: Page transitions don't feel like they block on the network (manual)",
"Progressive Web App: Each page has a URL (manual)"
];

const reportToRow = (csvFileContents) => {
const reportRows = csvParse(csvFileContents, {
columns: true,
skip_empty_lines: true,
ltrim: true
});
let columns = [
reportRows[0].requestedUrl,
reportRows[0].finalUrl
];
reportRows.forEach( (reportRow,index) => { columns.push(reportRow.score) });
const csvRow = columns;
return csvRow;
}

module.exports = {
reportToRow,
reportToRowHeaders
}
Loading