Skip to content

Commit

Permalink
feat: add filter CLI program
Browse files Browse the repository at this point in the history
  • Loading branch information
gajus committed Aug 3, 2018
1 parent c8edd7a commit 35c3d8e
Show file tree
Hide file tree
Showing 3 changed files with 111 additions and 15 deletions.
30 changes: 29 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ JSON logger for Node.js and browser.
* [`fatal`](#fatal)
* [Middlewares](#middlewares)
* [CLI program](#cli-program)
* [`pretty-print` program](#pretty-print-program)
* [`filter` program](#filter-program)
* [Transports](#transports)
* [Environment variables](#environment-variables)
* [Conventions](#conventions)
Expand Down Expand Up @@ -103,7 +105,7 @@ Produces output:

Roarr is designed to print all or none logs (refer to the [`ROARR_LOG` environment variable](#environment-variables) documentation).

To filter logs you need to use a JSON processor, e.g. [jq](https://stedolan.github.io/jq/).
To filter logs you need to use [`roarr filter` CLI program](#filter-program) or a JSON processor such as [jq](https://stedolan.github.io/jq/).

### jq primer

Expand Down Expand Up @@ -140,6 +142,8 @@ ROARR_LOG=true node ./index.js | jq -cRM 'fromjson? | select(.context.logLevel >

```

For a simplified way of filtering Roarr logs, refer to [`roarr filter` CLI program](#filter-program).

## Log message format

|Property name|Contents|
Expand Down Expand Up @@ -340,6 +344,8 @@ Raise an issue to add your middleware of your own creation.

## CLI program

### `pretty-print` program

Roarr comes with a CLI program used to pretty-print logs for development purposes.

To format the logs, pipe the program output to `roarr pretty-print` program, e.g.
Expand Down Expand Up @@ -373,6 +379,28 @@ The `roarr pretty-print` CLI program is using the context property names suggest

Explore other CLI commands and options using `roarr --help`.

### `filter` program

[Log filtering](#filtering-logs) can be done using a JSON processor such as `jq`. However, `jq` [does make it easy to ignore invalid JSON](https://github.com/stedolan/jq/issues/1547).

Roarr `filter` CLI program allows to filter only Roarr JSON messages ignoring all the other content, e.g.

```bash
$ echo '
{"context":{"package":"raygun","namespace":"createHttpProxyServer","logLevel":40},"message":"internal SSL Server running on 0.0.0.0:59222","sequence":0,"time":1533310067405,"version":"1.0.0"}
{"context":{"package":"raygun","namespace":"createHttpProxyServer","logLevel":40},"message":"gracefully shutting down the proxy server","sequence":1,"time":1533310067438,"version":"1.0.0"}
{"context":{"package":"raygun","namespace":"createOnCloseEventHandler","logLevel":30},"message":"raygun server closed","sequence":2,"time":1533310067439,"version":"1.0.0"}
foo bar
{"foo": "bar"}
{"context":{"package":"raygun","namespace":"createOnCloseEventHandler","logLevel":30},"message":"internal SSL close","sequence":3,"time":1533310067439,"version":"1.0.0"}
' | roarr filter 'select(.context.logLevel > 30)' | roarr pretty-print
[2018-08-03T15:27:47.405Z] WARN (40) (@raygun) (#createHttpProxyServer): internal SSL Server running on 0.0.0.0:59222
[2018-08-03T15:27:47.438Z] WARN (40) (@raygun) (#createHttpProxyServer): gracefully shutting down the proxy server
foo bar
{"foo": "bar"}

```
## Transports
A transport in most logging libraries is something that runs in-process to perform some operation with the finalised log line. For example, a transport might send the log line to a standard syslog server after processing the log line and reformatting it.
Expand Down
30 changes: 16 additions & 14 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,33 +13,35 @@
"dependencies": {
"boolean": "^0.1.3",
"chalk": "^2.4.1",
"node-jq": "^1.3.1",
"prettyjson": "^1.2.1",
"semver-compare": "^1.0.0",
"split2": "^2.2.0",
"sprintf-js": "^1.1.1",
"through2": "^2.0.3",
"ulid": "^2.3.0",
"yargs": "^12.0.1"
},
"description": "JSON logger for Node.js and browser.",
"devDependencies": {
"@babel/cli": "^7.0.0-beta.54",
"@babel/core": "^7.0.0-beta.54",
"@babel/node": "^7.0.0-beta.54",
"@babel/plugin-transform-flow-strip-types": "^7.0.0-beta.54",
"@babel/preset-env": "^7.0.0-beta.54",
"@babel/register": "^7.0.0-beta.54",
"@babel/cli": "^7.0.0-beta.55",
"@babel/core": "^7.0.0-beta.55",
"@babel/node": "^7.0.0-beta.55",
"@babel/plugin-transform-flow-strip-types": "^7.0.0-beta.55",
"@babel/preset-env": "^7.0.0-beta.55",
"@babel/register": "^7.0.0-beta.55",
"ava": "^1.0.0-beta.6",
"babel-plugin-istanbul": "^5.0.0",
"babel-plugin-istanbul": "^5.0.1",
"coveralls": "^3.0.2",
"eslint": "^5.1.0",
"eslint": "^5.2.0",
"eslint-config-canonical": "^11.0.0",
"flow-bin": "^0.76.0",
"flow-copy-source": "^2.0.1",
"flow-bin": "^0.77.0",
"flow-copy-source": "^2.0.2",
"husky": "^1.0.0-rc.13",
"nock": "^9.4.2",
"nyc": "^13.0.0",
"semantic-release": "^15.8.0",
"sinon": "^6.1.3"
"nock": "^9.4.4",
"nyc": "^13.0.1",
"semantic-release": "^15.9.3",
"sinon": "^6.1.4"
},
"engines": {
"node": ">=8.0"
Expand Down
66 changes: 66 additions & 0 deletions src/bin/commands/filter.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// @flow

import split from 'split2';
import through from 'through2';
import {
run
} from 'node-jq';
import {
isRoarrLine
} from './utilities';

type ArgvType = {|
+excludeOrphans: boolean,
+jqExpression: string
|};

type LogFilterConfigurationType = {|
+excludeOrphans: boolean,
+jqExpression: string
|};

const filterLog = (configuration: LogFilterConfigurationType, line: string, callback: (error?: Error, line?: string) => {}) => {
if (!isRoarrLine(line)) {
callback(undefined, configuration.excludeOrphans ? '' : line + '\n');

return;
}

run(configuration.jqExpression, line, {
input: 'string',
output: 'string'
})
.then((data) => {
return callback(undefined, data ? data + '\n' : '');
})
.catch((error) => {
return callback(error);
});
};

export const command = 'filter <jq-expression>';
export const desc = 'Filter Roarr messages using jq.';

// eslint-disable-next-line flowtype/no-weak-types
export const builder = (yargs: Object) => {
return yargs
.options({
'exclude-orphans': {
default: false,
describe: 'Excludes messages that cannot be recognized as Roarr log message.',
type: 'boolean'
}
});
};

export const handler = (argv: ArgvType) => {
// argv
process.stdin
.pipe(split())
.pipe(through((chunk, encoding, callback) => {
const line = chunk.toString();

filterLog(argv, line, callback);
}))
.pipe(process.stdout);
};

0 comments on commit 35c3d8e

Please sign in to comment.