Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add warning and collect details of URL errors in sitemap #830

Merged
merged 33 commits into from
Sep 6, 2024
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f81409c
adding server error check
fellyph Aug 21, 2024
99addcf
Throw error properly without printing the whole stack trace.
amovar18 Aug 21, 2024
3abdb31
Add failure cross when analysis fails.
amovar18 Aug 26, 2024
5ccd0c1
Merge branch 'develop' of github.com:GoogleChromeLabs/ps-analysis-too…
amovar18 Sep 2, 2024
cab220b
Add support to collect all erroredOutUrls.
amovar18 Sep 2, 2024
6d5c71a
Move getSiteReport to utils package.
amovar18 Sep 2, 2024
c7a1c99
Add Urls with issues to cli dashboard.
amovar18 Sep 2, 2024
0e1285d
Add support for stackTrace in error collection.
amovar18 Sep 2, 2024
cc887ca
Add URL with issue icon.
amovar18 Sep 2, 2024
25d0613
Add error stack trace properly.
amovar18 Sep 3, 2024
f530a2f
Fix failing tests
amovar18 Sep 3, 2024
93e7e4a
Fix cli e2e
amovar18 Sep 3, 2024
2290f7b
Fix cli e2e and update the icon size.
amovar18 Sep 3, 2024
2af6403
Test cli e2e test passing.
amovar18 Sep 3, 2024
afe8dea
Remove rmsync.
amovar18 Sep 3, 2024
2760b2c
Change the url for testing.
amovar18 Sep 3, 2024
c627b82
Remove debug.
amovar18 Sep 3, 2024
54cfbaf
Skip CLI E2E tests.
amovar18 Sep 3, 2024
b97b20b
Change description to error description.
amovar18 Sep 4, 2024
5d9f5bc
Add a console.log message to indicate failing urls.
amovar18 Sep 4, 2024
30a86aa
Add method to generate error logs.
amovar18 Sep 4, 2024
8b275a1
Prettify stack trace.
amovar18 Sep 4, 2024
22383a4
Add feedback for the urls.
amovar18 Sep 4, 2024
ebb2ca1
Fix bugs.
amovar18 Sep 5, 2024
6dd01f3
Fix name or urls.
amovar18 Sep 5, 2024
b17e9d4
Fix condition.
amovar18 Sep 5, 2024
e87eb1d
Fix QA feedbacks.
amovar18 Sep 5, 2024
3769a45
Fix url in sidebar.
amovar18 Sep 6, 2024
68b01f6
use url instead of _url
amovar18 Sep 6, 2024
1945c57
Use shorthand and use index.
amovar18 Sep 6, 2024
32ceef5
Refactor code.
amovar18 Sep 6, 2024
f8eb501
Move error logs in common and download errorlog when using -o.
amovar18 Sep 6, 2024
116df07
Address minor code feedbacks.
amovar18 Sep 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use shorthand and use index.
  • Loading branch information
amovar18 committed Sep 6, 2024
commit 1945c5746f7f0ca28527f111eaf4c24cefb50004
28 changes: 15 additions & 13 deletions packages/cli/src/utils/getSiteReport.ts
Original file line number Diff line number Diff line change
@@ -41,37 +41,37 @@ function getSiteReport(
processedData: any,
technologyAnalysisData: any
) {
return urls.map((url, ind) => {
return urls.map((url, index) => {
const hasTimeOutError = (
processedData[ind].erroredOutUrls[url] as SingleURLError[]
processedData[index].erroredOutUrls[url] as SingleURLError[]
)?.some(
({ errorName }) => errorName === 'TimeoutError' || errorName === 'i'
);

const detectedMatchingSignatures: LibraryData = {
...detectMatchingSignatures(
processedData[ind].resources ?? [],
processedData[index].resources ?? [],
Object.fromEntries(
LIBRARIES.map((library) => [library.name, library.detectionFunction])
) as DetectionFunctions
),
...(processedData[ind]?.domQueryMatches ?? {}),
...(processedData[index]?.domQueryMatches ?? {}),
};

if (
processedData[ind].erroredOutUrls[url] &&
processedData[ind].erroredOutUrls[url].length > 0
processedData[index].erroredOutUrls[url] &&
processedData[index].erroredOutUrls[url].length > 0
) {
if (hasTimeOutError) {
return {
pageUrl: parseUrl(url) ? new URL(url).href : encodeURI(url),
technologyData: technologyAnalysisData
? technologyAnalysisData[ind]
? technologyAnalysisData[index]
: [],
cookieData: processedData[ind].cookieData,
cookieData: processedData[index].cookieData,
libraryMatches: detectedMatchingSignatures ?? [],
erroredOutUrls: [
...processedData[ind].erroredOutUrls[url].map(
...processedData[index].erroredOutUrls[url].map(
(errors: SingleURLError) => {
return {
url: parseUrl(url) ? new URL(url).href : encodeURI(url),
@@ -89,10 +89,10 @@ function getSiteReport(
cookieData: {},
libraryMatches: [],
erroredOutUrls: [
...processedData[ind].erroredOutUrls[url].map(
...processedData[index].erroredOutUrls[url].map(
(errors: SingleURLError) => {
return {
url: url,
url,
...errors,
};
}
@@ -103,8 +103,10 @@ function getSiteReport(

return {
pageUrl: encodeURI(url),
technologyData: technologyAnalysisData ? technologyAnalysisData[ind] : [],
cookieData: processedData[ind].cookieData,
technologyData: technologyAnalysisData
? technologyAnalysisData[index]
: [],
cookieData: processedData[index].cookieData,
libraryMatches: detectedMatchingSignatures ?? [],
} as unknown as CompleteJson;
});