Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add list-files: csv format #68

Merged
merged 3 commits into from
Feb 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: "Build"
on:
push:
paths-ignore: [ '*.md' ]
branches:
- master

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/pull-request-verification.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: "Pull Request Verification"
on:
pull_request:
paths-ignore: [ '*.md' ]
branches:
- master
- develop
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Changelog

## v2.9.0
- [Add list-files: csv format](https://github.com/dorny/paths-filter/pull/68)

## v2.8.0
- [Add count output variable](https://github.com/dorny/paths-filter/pull/65)
- [Fix log grouping of changes](https://github.com/dorny/paths-filter/pull/61)
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,13 @@ For more scenarios see [examples](#examples) section.


# What's New
- Add `list-files: csv` format
- Configure matrix job to run for each folder with changes using `changes` output
- Improved listing of matching files with `list-files: shell` and `list-files: escape` options
- Support local changes
- Fixed retrieval of all changes via Github API when there are 100+ changes
- Paths expressions are now evaluated using [picomatch](https://github.com/micromatch/picomatch) library
- Support workflows triggered by any event
- Fixed compatibility with older (<2.23) versions of git

For more information see [CHANGELOG](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md)

Expand Down Expand Up @@ -122,6 +122,8 @@ For more information see [CHANGELOG](https://github.com/dorny/paths-filter/blob/

# Enables listing of files matching the filter:
# 'none' - Disables listing of matching files (default).
# 'csv' - Coma separated list of filenames.
# If needed it uses double quotes to wrap filename with unsafe characters.
# 'json' - Matching files paths are formatted as JSON array.
# 'shell' - Space delimited list usable as command line argument list in Linux shell.
# If needed it uses single or double quotes to wrap filename with unsafe characters.
Expand Down
23 changes: 23 additions & 0 deletions __tests__/csv-escape.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import {csvEscape} from '../src/list-format/csv-escape'

describe('csvEscape() backslash escapes every character except subset of definitely safe characters', () => {
test('simple filename should not be modified', () => {
expect(csvEscape('file.txt')).toBe('file.txt')
})

test('directory separator should be preserved and not escaped', () => {
expect(csvEscape('path/to/file.txt')).toBe('path/to/file.txt')
})

test('filename with spaces should be quoted', () => {
expect(csvEscape('file with space')).toBe('"file with space"')
})

test('filename with "," should be quoted', () => {
expect(csvEscape('file, with coma')).toBe('"file, with coma"')
})

test('Double quote should be escaped by another double quote', () => {
expect(csvEscape('file " with double quote')).toBe('"file "" with double quote"')
})
})
12 changes: 6 additions & 6 deletions __tests__/shell-escape.test.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
import {escape, shellEscape} from '../src/shell-escape'
import {backslashEscape, shellEscape} from '../src/list-format/shell-escape'

describe('escape() backslash escapes every character except subset of definitely safe characters', () => {
test('simple filename should not be modified', () => {
expect(escape('file.txt')).toBe('file.txt')
expect(backslashEscape('file.txt')).toBe('file.txt')
})

test('directory separator should be preserved and not escaped', () => {
expect(escape('path/to/file.txt')).toBe('path/to/file.txt')
expect(backslashEscape('path/to/file.txt')).toBe('path/to/file.txt')
})

test('spaces should be escaped with backslash', () => {
expect(escape('file with space')).toBe('file\\ with\\ space')
expect(backslashEscape('file with space')).toBe('file\\ with\\ space')
})

test('quotes should be escaped with backslash', () => {
expect(escape('file\'with quote"')).toBe('file\\\'with\\ quote\\"')
expect(backslashEscape('file\'with quote"')).toBe('file\\\'with\\ quote\\"')
})

test('$variables should be escaped', () => {
expect(escape('$var')).toBe('\\$var')
expect(backslashEscape('$var')).toBe('\\$var')
})
})

Expand Down
2 changes: 2 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ inputs:
description: |
Enables listing of files matching the filter:
'none' - Disables listing of matching files (default).
'csv' - Coma separated list of filenames.
If needed it uses double quotes to wrap filename with unsafe characters.
'json' - Serialized as JSON array.
'shell' - Space delimited list usable as command line argument list in linux shell.
If needed it uses single or double quotes to wrap filename with unsafe characters.
Expand Down
110 changes: 70 additions & 40 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4648,7 +4648,8 @@ const github = __importStar(__webpack_require__(469));
const filter_1 = __webpack_require__(235);
const file_1 = __webpack_require__(258);
const git = __importStar(__webpack_require__(136));
const shell_escape_1 = __webpack_require__(751);
const shell_escape_1 = __webpack_require__(206);
const csv_escape_1 = __webpack_require__(410);
async function run() {
try {
const workingDirectory = core.getInput('working-directory', { required: false });
Expand Down Expand Up @@ -4825,18 +4826,20 @@ function exportResults(results, format) {
function serializeExport(files, format) {
const fileNames = files.map(file => file.filename);
switch (format) {
case 'csv':
return fileNames.map(csv_escape_1.csvEscape).join(',');
case 'json':
return JSON.stringify(fileNames);
case 'escape':
return fileNames.map(shell_escape_1.escape).join(' ');
return fileNames.map(shell_escape_1.backslashEscape).join(' ');
case 'shell':
return fileNames.map(shell_escape_1.shellEscape).join(' ');
default:
return '';
}
}
function isExportFormat(value) {
return value === 'none' || value === 'shell' || value === 'json' || value === 'escape';
return ['none', 'csv', 'shell', 'json', 'escape'].includes(value);
}
run();

Expand Down Expand Up @@ -5028,6 +5031,43 @@ module.exports = {
};


/***/ }),

/***/ 206:
/***/ (function(__unusedmodule, exports) {

"use strict";

Object.defineProperty(exports, "__esModule", { value: true });
exports.shellEscape = exports.backslashEscape = void 0;
// Backslash escape every character except small subset of definitely safe characters
function backslashEscape(value) {
return value.replace(/([^a-zA-Z0-9,._+:@%/-])/gm, '\\$1');
}
exports.backslashEscape = backslashEscape;
// Returns filename escaped for usage as shell argument.
// Applies "human readable" approach with as few escaping applied as possible
function shellEscape(value) {
if (value === '')
return value;
// Only safe characters
if (/^[a-zA-Z0-9,._+:@%/-]+$/m.test(value)) {
return value;
}
if (value.includes("'")) {
// Only safe characters, single quotes and white-spaces
if (/^[a-zA-Z0-9,._+:@%/'\s-]+$/m.test(value)) {
return `"${value}"`;
}
// Split by single quote and apply escaping recursively
return value.split("'").map(shellEscape).join("\\'");
}
// Contains some unsafe characters but no single quote
return `'${value}'`;
}
exports.shellEscape = shellEscape;


/***/ }),

/***/ 211:
Expand Down Expand Up @@ -8813,6 +8853,33 @@ function Octokit(plugins, options) {
}


/***/ }),

/***/ 410:
/***/ (function(__unusedmodule, exports) {

"use strict";

Object.defineProperty(exports, "__esModule", { value: true });
exports.csvEscape = void 0;
// Returns filename escaped for CSV
// Wraps file name into "..." only when it contains some potentially unsafe character
function csvEscape(value) {
if (value === '')
return value;
// Only safe characters
if (/^[a-zA-Z0-9._+:@%/-]+$/m.test(value)) {
return value;
}
// https://tools.ietf.org/html/rfc4180
// If double-quotes are used to enclose fields, then a double-quote
// appearing inside a field must be escaped by preceding it with
// another double quote
return `"${value.replace(/"/g, '""')}"`;
}
exports.csvEscape = csvEscape;


/***/ }),

/***/ 413:
Expand Down Expand Up @@ -15225,43 +15292,6 @@ function sync (path, options) {

module.exports = require("fs");

/***/ }),

/***/ 751:
/***/ (function(__unusedmodule, exports) {

"use strict";

Object.defineProperty(exports, "__esModule", { value: true });
exports.shellEscape = exports.escape = void 0;
// Backslash escape every character except small subset of definitely safe characters
function escape(value) {
return value.replace(/([^a-zA-Z0-9,._+:@%/-])/gm, '\\$1');
}
exports.escape = escape;
// Returns filename escaped for usage as shell argument.
// Applies "human readable" approach with as few escaping applied as possible
function shellEscape(value) {
if (value === '')
return value;
// Only safe characters
if (/^[a-zA-Z0-9,._+:@%/-]+$/m.test(value)) {
return value;
}
if (value.includes("'")) {
// Only safe characters, single quotes and white-spaces
if (/^[a-zA-Z0-9,._+:@%/'\s-]+$/m.test(value)) {
return `"${value}"`;
}
// Split by single quote and apply escaping recursively
return value.split("'").map(shellEscape).join("\\'");
}
// Contains some unsafe characters but no single quote
return `'${value}'`;
}
exports.shellEscape = shellEscape;


/***/ }),

/***/ 753:
Expand Down
16 changes: 16 additions & 0 deletions src/list-format/csv-escape.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// Returns filename escaped for CSV
// Wraps file name into "..." only when it contains some potentially unsafe character
export function csvEscape(value: string): string {
if (value === '') return value

// Only safe characters
if (/^[a-zA-Z0-9._+:@%/-]+$/m.test(value)) {
return value
}

// https://tools.ietf.org/html/rfc4180
// If double-quotes are used to enclose fields, then a double-quote
// appearing inside a field must be escaped by preceding it with
// another double quote
return `"${value.replace(/"/g, '""')}"`
}
2 changes: 1 addition & 1 deletion src/shell-escape.ts → src/list-format/shell-escape.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// Backslash escape every character except small subset of definitely safe characters
export function escape(value: string): string {
export function backslashEscape(value: string): string {
return value.replace(/([^a-zA-Z0-9,._+:@%/-])/gm, '\\$1')
}

Expand Down
11 changes: 7 additions & 4 deletions src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ import {Webhooks} from '@octokit/webhooks'
import {Filter, FilterResults} from './filter'
import {File, ChangeStatus} from './file'
import * as git from './git'
import {escape, shellEscape} from './shell-escape'
import {backslashEscape, shellEscape} from './list-format/shell-escape'
import {csvEscape} from './list-format/csv-escape'

type ExportFormat = 'none' | 'json' | 'shell' | 'escape'
type ExportFormat = 'none' | 'csv' | 'json' | 'shell' | 'escape'

async function run(): Promise<void> {
try {
Expand Down Expand Up @@ -210,10 +211,12 @@ function exportResults(results: FilterResults, format: ExportFormat): void {
function serializeExport(files: File[], format: ExportFormat): string {
const fileNames = files.map(file => file.filename)
switch (format) {
case 'csv':
return fileNames.map(csvEscape).join(',')
case 'json':
return JSON.stringify(fileNames)
case 'escape':
return fileNames.map(escape).join(' ')
return fileNames.map(backslashEscape).join(' ')
case 'shell':
return fileNames.map(shellEscape).join(' ')
default:
Expand All @@ -222,7 +225,7 @@ function serializeExport(files: File[], format: ExportFormat): string {
}

function isExportFormat(value: string): value is ExportFormat {
return value === 'none' || value === 'shell' || value === 'json' || value === 'escape'
return ['none', 'csv', 'shell', 'json', 'escape'].includes(value)
}

run()