Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Miscellaneous QA issues #795

Merged
merged 34 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0734e4d
Fix tabFrames calculation in the report.
amovar18 Jul 31, 2024
be66259
Remove unwanted `.
amovar18 Jul 31, 2024
2db3c97
Add unknown to the analytics.
amovar18 Jul 31, 2024
b87672a
Change httpOnly to httponly.
amovar18 Jul 31, 2024
a34eab7
Fix http only filter sorting.
amovar18 Jul 31, 2024
d17d583
Fix types that break the build.
amovar18 Jul 31, 2024
abbf1c6
Remove animation which used to delay thecolor updation.
amovar18 Jul 31, 2024
3334481
Add darkmode for report.
amovar18 Jul 31, 2024
688c29b
Add darkmode for extension report.
amovar18 Jul 31, 2024
ce5ac22
Fix white color.
amovar18 Aug 1, 2024
1caf654
Fix empty circle color.
amovar18 Aug 1, 2024
cb730a7
Center align section, add text for zero blocked cookies, update heade…
mayan-000 Aug 1, 2024
cbf986d
Fix failing test.
amovar18 Aug 1, 2024
79522ae
Merge branch 'fix/build-issues' of github.com:GoogleChromeLabs/ps-ana…
amovar18 Aug 1, 2024
52978a6
Fix failing test.
amovar18 Aug 1, 2024
2c4c509
Add httonly to js cookies in cli.
amovar18 Aug 1, 2024
61387a9
Move generateTableCSV to common folder and reuse it.
amovar18 Aug 1, 2024
f120c4c
Fix file naming and import.
amovar18 Aug 1, 2024
649f42e
Fix build breaking.
amovar18 Aug 1, 2024
98ab7a0
Make samesite lowercase default.
amovar18 Aug 2, 2024
4053495
Use lax bydefault.
amovar18 Aug 2, 2024
78bcbe7
Fix failing tests.
amovar18 Aug 2, 2024
5c74437
Fix failing text.
amovar18 Aug 2, 2024
42ad683
Fix command in the workflow doc.
amovar18 Aug 2, 2024
bf75543
Fix command in the workflow doc.
amovar18 Aug 2, 2024
97447a6
Fix sameSite attribute.
amovar18 Aug 2, 2024
37c80ad
Use i18n for samesite in CSV and update key in hook
mayan-000 Aug 2, 2024
b0cd3a5
Update comparator
mayan-000 Aug 2, 2024
b19169d
Update comparator
mayan-000 Aug 2, 2024
3ba6a08
Fix the csv sanitisation.
amovar18 Aug 2, 2024
b987813
Merge branch 'fix/build-issues' of github.com:GoogleChromeLabs/ps-ana…
amovar18 Aug 2, 2024
378720c
Fix failing tests.
amovar18 Aug 2, 2024
a86d93f
Fix sanitisation of records and fix failing tests
amovar18 Aug 2, 2024
8700363
Add edgecase data to cli-dashboard.
amovar18 Aug 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/development-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ npm run dev:cli

After making changes in the `cli`, `cli-dashboard`, or any CLI-related packages, use:
```bash
npm run build:cli
npm run build:cli:dashboard
```

## Build Workflows
Expand Down
13 changes: 9 additions & 4 deletions packages/analysis-utils/src/browserManagement/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
this.indent = indent;
}

debugLog(msg: any) {

Check warning on line 81 in packages/analysis-utils/src/browserManagement/index.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type
if (this.shouldLogDebug && this.spinnies) {
this.spinnies.add(msg, {
text: msg,
Expand Down Expand Up @@ -334,7 +334,7 @@
domain: parsedCookie.domain,
path: parsedCookie.path || '/',
value: parsedCookie.value,
samesite: parsedCookie.samesite || 'Lax',
samesite: parsedCookie.samesite?.toLowerCase() || 'lax',
expires: parsedCookie.expires || 'Session',
httponly: parsedCookie.httponly || false,
secure: parsedCookie.secure || false,
Expand Down Expand Up @@ -407,9 +407,9 @@
domain: associatedCookie.cookie.domain,
path: associatedCookie.cookie.path || '/',
value: associatedCookie.cookie.value,
sameSite: associatedCookie.cookie.sameSite || 'Lax',
samesite: associatedCookie.cookie.sameSite?.toLowerCase() || 'lax',
expires: associatedCookie.cookie.expires || 'Session',
httpOnly: associatedCookie.cookie.httpOnly || false,
httponly: associatedCookie.cookie.httpOnly || false,
secure: associatedCookie.cookie.secure || false,
partitionKey: '',
},
Expand Down Expand Up @@ -562,7 +562,12 @@
}
const key = cookie.name + ':' + cookie.domain + ':' + cookie.path;
frameCookies[key] = {
parsedCookie: { ...cookie, partitionKey: '' },
parsedCookie: {
...cookie,
partitionKey: '',
httponly: false,
samesite: cookie.sameSite?.toLowerCase() || 'lax',
},
};
});

Expand All @@ -578,7 +583,7 @@
}

getResources(urls: string[]) {
const allFetchedResources: { [key: string]: any } = {};

Check warning on line 586 in packages/analysis-utils/src/browserManagement/index.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type

urls.forEach((url) => {
const page = this.pages[url];
Expand Down
2 changes: 1 addition & 1 deletion packages/analysis-utils/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
category: string;
description: string;
isFirstParty: 'Yes' | 'No';
sameSite: string;
samesite: string;
partitionKey?: Protocol.Network.CookiePartitionKey;
pageUrl: string;
requestUrls: { [key: string]: string };
Expand Down Expand Up @@ -107,8 +107,8 @@
};

export interface Job {
data: any;

Check warning on line 110 in packages/analysis-utils/src/types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type
status: string;
response: any;

Check warning on line 112 in packages/analysis-utils/src/types.ts

View workflow job for this annotation

GitHub Actions / Lint

Unexpected any. Specify a different type
message: string;
}
6 changes: 5 additions & 1 deletion packages/cli-dashboard/src/app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ const App = () => {
return;
}

if (window.matchMedia('(prefers-color-scheme: dark)').matches) {
bodyTag.classList.add('dark');
}

bodyTag.style.fontSize = '75%';
}, []);

Expand Down Expand Up @@ -130,7 +134,7 @@ const App = () => {
}

return (
<div className="w-full h-screen flex">
<div className="w-full h-screen flex dark:bg-raisin-black text-raisin-black dark:text-bright-gray">
<SiteReport
completeJson={completeJsonReport}
cookies={cookies}
Expand Down
145 changes: 145 additions & 0 deletions packages/cli-dashboard/src/dummyData/PSAT_DATA.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,151 @@ export default {
cookieData: {
'https://open.spotify.com': {
frameCookies: {
'_cc_cccrwdcntrl.net/': {
parsedCookie: {
domain: '.crwdcntrl.net',
name: '_cc_cc',
path: '/',
samesite: 'none',
httponly: false,
priority: 'Medium',
value:
'"ACZ4nGNQMDewNDA1N080NjA0SzVINbQwSzExMbcwNLMwNDQzNjBnAIK0Wb7zGRAAACRYCVA%3D"',
partitionKey: '',
expires: '2025-04-15T11:27:27.639Z',
httpOnly: false,
sameParty: false,
sameSite: 'None',
secure: true,
session: false,
size: 82,
sourcePort: 443,
sourceScheme: 'Secure',
},
warningReasons: [],
blockedReasons: ['ThirdPartyPhaseout'],
networkEvents: {
requestEvents: [],
responseEvents: [],
},
headerType: 'http',
isFirstParty: false,
url: '',
frameIdList: ['6C93EFD2145CB7798F6C74BC08576FAE'],
analytics: {
platform: 'Lotame',
category: 'Marketing',
name: '_cc_cc',
domain: 'crwdcntrl.net',
description: 'OCD__cc_cc_description',
retention: 'OCD_retention_session',
dataController: 'Lotame',
gdprUrl:
'https://www.lotame.com/about-lotame/privacy/lotames-products-services-privacy-policy/',
wildcard: '0',
},
blockingStatus: {
inboundBlock: 'NOT_BLOCKED',
outboundBlock: 'BLOCKED_IN_ALL_EVENTS',
},
isBlocked: true,
},
'_cc_audcrwdcntrl.net/': {
parsedCookie: {
domain: '.crwdcntrl.net',
name: '_cc_aud',
path: '/',
samesite: 'none',
httponly: false,
priority: 'Medium',
value: '"ABR4nGNgYGBIm%2BU7nwEOABfpAe0%3D"',
partitionKey: '',
expires: '2025-04-15T11:27:27.639Z',
httpOnly: false,
sameParty: false,
sameSite: 'None',
secure: true,
session: false,
size: 41,
sourcePort: 443,
sourceScheme: 'Secure',
},
warningReasons: [],
blockedReasons: ['ThirdPartyPhaseout'],
networkEvents: {
requestEvents: [],
responseEvents: [],
},
headerType: 'http',
isFirstParty: false,
url: '',
frameIdList: ['6C93EFD2145CB7798F6C74BC08576FAE'],
analytics: {
platform: 'Lotame',
category: 'Marketing',
name: '_cc_aud',
domain: 'crwdcntrl.net',
description: 'OCD__cc_aud_description',
retention: 'OCD_retention_269_days',
dataController: 'Lotame',
gdprUrl:
'https://www.lotame.com/about-lotame/privacy/lotames-products-services-privacy-policy/',
wildcard: '0',
},
blockingStatus: {
inboundBlock: 'NOT_BLOCKED',
outboundBlock: 'BLOCKED_IN_ALL_EVENTS',
},
isBlocked: true,
},
'plsGeoObjspectator.org/': {
isBlocked: false,
parsedCookie: {
domain: 'spectator.org',
expires: '2024-08-03T07:24:10.000Z',
httpOnly: false,
name: 'plsGeoObj',
path: '/',
priority: 'Medium',
sameParty: false,
sameSite: 'Lax',
secure: false,
session: false,
size: 117,
sourcePort: 443,
sourceScheme: 'Secure',
value:
'{"ip":"59.88.27.175","country":"IN","region":"MH","city":"Pune","zip":"411004","location":"18.6161,73.7286"}',
httponly: false,
samesite: 'lax',
partitionKey: '',
},
networkEvents: {
requestEvents: [],
responseEvents: [],
},
blockedReasons: [],
analytics: {
platform: 'Unknown',
category: 'Uncategorized',
name: '',
domain: '',
description: '',
retention: '',
dataController: '',
gdprUrl: '',
wildcard: '',
},
url: '',
headerType: 'http',
isFirstParty: true,
frameIdList: ['6C93EFD2145CB7798F6C74BC08576FAE'],
blockingStatus: {
inboundBlock: 'NOT_BLOCKED',
outboundBlock: 'NOT_BLOCKED',
},
warningReasons: [],
},
'sp_t:.spotify.com:/': {
parsedCookie: {
name: 'sp_t',
Expand Down
4 changes: 2 additions & 2 deletions packages/common/src/cookies.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -203,9 +203,9 @@ export type CookieJsonDataType = {
partitionKey: string;
path: string;
expires: string;
httpOnly: boolean;
httponly: boolean;
secure: boolean;
sameSite: string;
samesite: string;
priority?: 'Low' | 'Medium' | 'High';
size?: number;
};
Expand Down
2 changes: 1 addition & 1 deletion packages/common/src/utils/findAnalyticsMatch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ const wildTest = (wildcard: string, str: string): boolean => {
};

export const emptyAnalytics = {
platform: '',
platform: 'Unknown',
category: 'Uncategorized',
name: '',
domain: '',
Expand Down
65 changes: 12 additions & 53 deletions packages/common/src/utils/generateReports/generateAllCookiesCSV.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@
/**
* External dependencies
*/
import sanitizeCsvRecord from '../sanitizeCsvRecord';
import { I18n } from '@google-psat/i18n';
/**
* Internal dependencies
*/
import {
CookieTableData,
type CompleteJson,
type CookieJsonDataType,
} from '../../cookies.types';
import calculateEffectiveExpiryDate from '../calculateEffectiveExpiryDate';
import generateExtensionCookieTableCSV from './generateExtensionCookietableCSV';
import generateCLICookieTableCSV from './generateCLICookieTableCSV';

export const COOKIES_DATA_HEADER = [
() => I18n.getMessage('name'),
Expand Down Expand Up @@ -59,61 +60,19 @@ const generateAllCookiesCSV = (siteAnalysisData: CompleteJson): string => {
});
});

if (isExtension) {
COOKIES_DATA_HEADER.push(
() => I18n.getMessage('priority'),
() => I18n.getMessage('size')
);
}

let cookieRecords = '';
const cookieRecords: CookieTableData[] = [];

for (const cookie of cookieMap.values()) {
//This should be in the same order as cookieDataHeader
const expires = calculateEffectiveExpiryDate(cookie.parsedCookie.expires);

const recordsArray = [
cookie.parsedCookie.name,
cookie.isFirstParty
? I18n.getMessage('firstParty')
: I18n.getMessage('thirdParty'),
cookie.parsedCookie.domain || ' ',
cookie.parsedCookie.partitionKey || ' ',
cookie.parsedCookie.sameSite,
I18n.getMessage(
cookie.analytics?.category?.toLowerCase() || 'uncategorized'
),
cookie.analytics.platform,
cookie.parsedCookie.httpOnly
? I18n.getMessage('yes')
: I18n.getMessage('no'),
cookie.parsedCookie.secure
? I18n.getMessage('yes')
: I18n.getMessage('no'),
cookie.parsedCookie.value,
cookie.parsedCookie.path,
expires === 'Session' ? I18n.getMessage('session') : expires,
cookie.isBlocked ? I18n.getMessage('yes') : I18n.getMessage('no'),
cookie.analytics.GDPR || 'NA',
];

if (isExtension) {
recordsArray.push(
I18n.getMessage((cookie.parsedCookie?.priority || ' ').toLowerCase()),
cookie.parsedCookie?.size?.toString() ?? ' '
);
}

recordsArray.map(sanitizeCsvRecord);

cookieRecords += recordsArray.join(',') + '\r\n';
//@ts-ignore
cookieRecords.push(cookie);
}
if (isExtension) {
return generateExtensionCookieTableCSV(cookieRecords, true) as string;
} else {
return generateCLICookieTableCSV(cookieRecords, true) as string;
}

return (
COOKIES_DATA_HEADER.map((header) => header()).join(',') +
'\r\n' +
cookieRecords
);
return '';
};

export default generateAllCookiesCSV;
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,12 @@
/**
* External dependencies.
*/
import { type CookieTableData, sanitizeCsvRecord } from '@google-psat/common';
import { I18n } from '@google-psat/i18n';
/**
* Internal dependencies.
*/
import sanitizeCsvRecord from '../sanitizeCsvRecord';
import { type CookieTableData } from '../../cookies.types';

const COOKIES_TABLE_DATA_HEADER_CLI = [
() => I18n.getMessage('name'),
Expand All @@ -36,7 +40,10 @@ const COOKIES_TABLE_DATA_HEADER_CLI = [
() => I18n.getMessage('gdpr'),
];

const generateCLICookieTableCSV = (cookies: CookieTableData[]): Blob => {
const generateCLICookieTableCSV = (
cookies: CookieTableData[],
returnString = false
): Blob | string => {
let cookieRecords = '';

for (const cookie of cookies) {
Expand All @@ -48,8 +55,7 @@ const generateCLICookieTableCSV = (cookies: CookieTableData[]): Blob => {
: I18n.getMessage('thirdParty'),
cookie.parsedCookie.domain || ' ',
cookie.parsedCookie.partitionKey || ' ',
//@ts-ignore
cookie.parsedCookie?.sameSite ?? cookie.parsedCookie?.samesite,
I18n.getMessage((cookie.parsedCookie?.samesite ?? 'lax').toLowerCase()),
I18n.getMessage(
cookie.analytics?.category?.toLowerCase() || 'uncategorized'
),
Expand All @@ -70,6 +76,14 @@ const generateCLICookieTableCSV = (cookies: CookieTableData[]): Blob => {
cookieRecords += recordsArray.join(',') + '\r\n';
}

if (returnString) {
return (
COOKIES_TABLE_DATA_HEADER_CLI.map((header) => header()).join(',') +
'\r\n' +
cookieRecords
);
}

return new Blob([
COOKIES_TABLE_DATA_HEADER_CLI.map((header) => header()).join(',') +
'\r\n' +
Expand Down
Loading
Loading