Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Audit Log Catalog #158

Merged
merged 6 commits into from
Mar 11, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4,340 changes: 4,340 additions & 0 deletions AUDIT_CATALOG.md

Large diffs are not rendered by default.

9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,14 @@ This repository contains definitions for the following CloudEvents:
|Firebase Remote Config|[Proto](proto/google/events/firebase/remoteconfig/v1/data.proto) / [JSON](https://googleapis.github.io/google-cloudevents/jsonschema/google/events/firebase/remoteconfig/v1/RemoteConfigEventData.json)|<br>Data Type:<br>`google.events.firebase.remoteconfig.v1.RemoteConfigEventData`<br>CloudEvent Type(s):<br>`google.firebase.remoteconfig.remoteConfig.v1.updated`|
<!-- GENERATED END -->

A [registry of the JSON schema catalog](https://googleapis.github.io/google-cloudevents/jsonschema/catalog.json) is also available.
## Google CloudEvent Catalogs

This repository contains a couple catalogs supporting Google CloudEvent event discovery:

- [JSON Schema catalog](https://googleapis.github.io/google-cloudevents/jsonschema/catalog.json)
- Metadata containing a list of JSON schemas for Google CloudEvents.
- [Cloud Audit Log catalog](AUDIT_CATALOG.md)
- For the type `google.cloud.audit.log.v1.written`, a list of possible values for `methodName` and `serviceName`.

## Google CloudEvent Type Libraries

Expand Down
7 changes: 6 additions & 1 deletion tools/readme-catalog/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# readme-catalog

Edits the top-level [`README.md`](../../README.md) event catalog.
Edits README catalog markdown files using a generator.

The following files are generated:

- The top-level [`README.md`](../../README.md) – CloudEvent catalog
- The [`AUDIT_CATALOG.md`](../../AUDIT_CATALOG.md) – Cloud Audit Log catalog

## Requirements

Expand Down
17 changes: 17 additions & 0 deletions tools/readme-catalog/common.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/**
* Replaces the contents of a string's GENERATED comments with a replacement.
* @param {string} s The string to replace.
* @param {string} replacement The replacement string.
*/
export const getGeneratedStringWithReplacement = (s, replacement) => {
const README_GEN_START = '<!-- GENERATED START -->';
const README_GEN_END = '<!-- GENERATED END -->';

const README_BEFORE_TABLE = s.substring(0, s.indexOf(README_GEN_START) + README_GEN_START.length);
const README_AFTER_TABLE = s.substring(s.indexOf(README_GEN_END));

// Return result (with newlines)
return `${README_BEFORE_TABLE}
${replacement}
${README_AFTER_TABLE}`;
};
62 changes: 62 additions & 0 deletions tools/readme-catalog/gen-audit-catalog.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import * as fs from 'fs';
import { dirname } from 'path';
import { fileURLToPath } from 'url';
import { getGeneratedStringWithReplacement } from './common.js';

/**
* Generates a Cloud Audit Log catalog markdown for the README.
* @example Example input:
* [{
* serviceName: 'workflows.googleapis.com',
* displayName: 'Workflows',
* methods: [
* {
* methodName: 'google.cloud.workflows.v1.Workflows.CreateWorkflow',
* lastAdded: '1607367890'
* },
* ]
* },
* ...
* ]
* {@link ../../json/audit/service_catalog.json}
*/
const __dirname = dirname(fileURLToPath(import.meta.url));
const CATALOG = JSON.parse(fs.readFileSync(`${__dirname}/../../json/audit/service_catalog.json`));

/**
* Generates the Cloud Audit Log discovery documentation at AUDIT_CATALOG.md.
*/
export const genAuditCatalog = () => {
// Update README
const AUDIT_CATALOG_PATH = `${__dirname}/../../AUDIT_CATALOG.md`;
const auditCatalogContents = fs.readFileSync(AUDIT_CATALOG_PATH).toString();
const replacementString = CATALOG.services.map(getServiceCatalogEntryMarkdown).join('');
const updatedReadmeContents = getGeneratedStringWithReplacement(auditCatalogContents, replacementString);

console.log(updatedReadmeContents);

// Save updated README
fs.writeFileSync(AUDIT_CATALOG_PATH, updatedReadmeContents);
console.log('- Updated README.');
};

/**
* Gets markdown for a CAL service catalog entry.
* @param {object} calEntry A single
* @returns {string} Markdown with info about this CAL entry
*/
const getServiceCatalogEntryMarkdown = (calEntry) => {
return `### ${calEntry.displayName}

#### \`serviceName\`

- \`${calEntry.serviceName}\`

#### \`methodName\`

${calEntry.methods.map((method => {
return `- \`${method.methodName}\``;
})).join('\n')}

`;
};
66 changes: 66 additions & 0 deletions tools/readme-catalog/gen-cloudevent-catalog.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import * as fs from 'fs';
import { dirname } from 'path';
import { fileURLToPath } from 'url';
import { getGeneratedStringWithReplacement } from './common.js';

/**
* Generates a CloudEvent catalog for the README.
*/

/**
* Gets markdown for the proto / JSON schema table cell.
* @param {object} schemaEntry The JSON schema catalog entry.
* @returns {string} Markdown for the schema entry.
*/
const getSchemaLinkMarkdown = (schemaEntry) => {
// Get the path the to the data from the datatype.
// in: "google.events.cloud.audit.v1.LogEntryData"
// out: "cloud/audit/v1"
const prefix = 'google.';
const schemaPath = schemaEntry.datatype.substring(
schemaEntry.datatype.indexOf(prefix) + prefix.length,
schemaEntry.datatype.indexOf(`.${schemaEntry.name}`),
).replace(/\./g, '/');

// Proto link is the path + data.proto
const protoLink = `proto/google/${schemaPath}/data.proto`;

// JSON schema path is the URL + jsonschema path + {name}.json
const jsonschemaPath = `${schemaPath}/${schemaEntry.name}`;
const jsonLink = `https://googleapis.github.io/google-cloudevents/jsonschema/google/${jsonschemaPath}.json`;
return `[Proto](${protoLink}) / [JSON](${jsonLink})`;
};

/**
* Updates the main README with a CloudEvent catalog.
*/
export const genCloudEventCatalog = () => {
console.log('- Updating README...');

// Generate table
const replacementTableRow = (schemaEntry) => {
// Add escaped ticks (\`) to code.
const PRODUCT = schemaEntry.product;
const SCHEMAS = getSchemaLinkMarkdown(schemaEntry);
const DATA_TYPE = `\`${schemaEntry.datatype}\``;
const CLOUDEVENT_TYPE = schemaEntry.cloudeventTypes.map(t => `\`${t}\``).join('<br/>');
return `|${PRODUCT}|${SCHEMAS}|<br>Data Type:<br>${DATA_TYPE}<br>CloudEvent Type(s):<br>${CLOUDEVENT_TYPE}|`;
};

// Get the catalog file, and replace the table.
const __dirname = dirname(fileURLToPath(import.meta.url));
const CATALOG = JSON.parse(fs.readFileSync(`${__dirname}/../../jsonschema/catalog.json`));
const replacementTable =
`|Product|Schemas|Types|
|-|-|-|
${CATALOG.schemas.map(replacementTableRow).join('\n')}`;

// Update README
const README_PATH = `${__dirname}/../../README.md`;
const readmeContents = fs.readFileSync(README_PATH).toString();
const updatedReadmeContents = getGeneratedStringWithReplacement(readmeContents, replacementTable);

// Save updated README
fs.writeFileSync(README_PATH, updatedReadmeContents);
console.log('- Updated README.');
};
77 changes: 5 additions & 72 deletions tools/readme-catalog/index.js
Original file line number Diff line number Diff line change
@@ -1,75 +1,8 @@
const fs = require('fs');
import {genAuditCatalog} from './gen-audit-catalog.js';
import {genCloudEventCatalog} from './gen-cloudevent-catalog.js';

/**
* Generates a catalog for the README.
* Generate and update the READMEs.
*/
const CATALOG = JSON.parse(fs.readFileSync(`${__dirname}/../../jsonschema/catalog.json`));

/**
* Replaces the contents of a string's GENERATED comments with a replacement.
* @param {string} s The string to replace.
* @param {string} replacement The replacement string.
*/
const getGeneratedStringWithReplacement = (s, replacement) => {
const README_GEN_START = '<!-- GENERATED START -->';
const README_GEN_END = '<!-- GENERATED END -->';

const README_BEFORE_TABLE = s.substring(0, s.indexOf(README_GEN_START) + README_GEN_START.length);
const README_AFTER_TABLE = s.substring(s.indexOf(README_GEN_END));

// Return result (with newlines)
return `${README_BEFORE_TABLE}
${replacement}
${README_AFTER_TABLE}`;
};

/**
* Gets markdown for the proto / JSON schema table cell.
* @param {object} schemaEntry The JSON schema catalog entry.
* @returns {string} Markdown for the schema entry.
*/
const getSchemaLinkMarkdown = (schemaEntry) => {
// Get the path the to the data from the datatype.
// in: "google.events.cloud.audit.v1.LogEntryData"
// out: "cloud/audit/v1"
const prefix = 'google.';
const schemaPath = schemaEntry.datatype.substring(
schemaEntry.datatype.indexOf(prefix) + prefix.length,
schemaEntry.datatype.indexOf(`.${schemaEntry.name}`),
).replace(/\./g, '/');

// Proto link is the path + data.proto
const protoLink = `proto/google/${schemaPath}/data.proto`;

// JSON schema path is the URL + jsonschema path + {name}.json
const jsonschemaPath = `${schemaPath}/${schemaEntry.name}`;
const jsonLink = `https://googleapis.github.io/google-cloudevents/jsonschema/google/${jsonschemaPath}.json`;
return `[Proto](${protoLink}) / [JSON](${jsonLink})`;
}

(async () => {
console.log('- Updating README...');

// Generate table
const replacementTableRow = (schemaEntry) => {
// Add escaped ticks (\`) to code.
const PRODUCT = schemaEntry.product;
const SCHEMAS = getSchemaLinkMarkdown(schemaEntry);
const DATA_TYPE = `\`${schemaEntry.datatype}\``;
const CLOUDEVENT_TYPE = schemaEntry.cloudeventTypes.map(t => `\`${t}\``).join('<br/>');
return `|${PRODUCT}|${SCHEMAS}|<br>Data Type:<br>${DATA_TYPE}<br>CloudEvent Type(s):<br>${CLOUDEVENT_TYPE}|`;
};
const replacementTable =
`|Product|Schemas|Types|
|-|-|-|
${CATALOG.schemas.map(replacementTableRow).join('\n')}`;

// Update README
const README_PATH = `${__dirname}/../../README.md`;
const readmeContents = fs.readFileSync(README_PATH).toString();
const updatedReadmeContents = getGeneratedStringWithReplacement(readmeContents, replacementTable);

// Save updated README
fs.writeFileSync(README_PATH, updatedReadmeContents);
console.log('- Updated README.');
})();
genAuditCatalog();
genCloudEventCatalog();
1 change: 1 addition & 0 deletions tools/readme-catalog/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"scripts": {
"start": "node ."
},
"type": "module",
"dependencies": {
}
}