Filtering examples

[PenelopeFudd]

Hi;
Just started trying former2, and I like it.

However, I have (way too) many log streams, and former2 got stuck for hours trying to enumerate them:

Too many requests for CloudWatchLogs.describeLogStreams, sleeping for 500ms

Is there a way to restrict which services it probes? I tried --search-filter, but it didn't seem to fix the problem.

Thanks!

[iann0036]

Hey @PenelopeFudd,

The latest version (0.2.47) should have the ability to add --exclude-services CloudWatch to your run to ignore CloudWatch services. I'll be working towards filtering this down beyond the service level soon.

[PenelopeFudd]

Maybe it could be done like this: --exclude-services "*" --include-services Lambda,Events,IAM ?

I wasn't aware that CloudFormation could (or even should) create log streams. :-)

[PenelopeFudd]

I apologize, but I'm fairly new to npm, and I'm not quite sure how to install the latest version of former2 from Github:

$ npm install iann0036/former2 -g
npm ERR! premature close

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/penelopefudd/.npm/_logs/2020-08-12T18_14_21_017Z-debug.log

$ cat /Users/penelopefudd/.npm/_logs/2020-08-12T18_14_21_017Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '/usr/local/Cellar/node/14.6.0/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'install',
1 verbose cli   'iann0036/former2',
1 verbose cli   '-g'
1 verbose cli ]
2 info using npm@6.14.7
3 info using node@v14.6.0
4 verbose npm-session 209e503c1cbf77df
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 silly fetchPackageMetaData error for github:iann0036/former2 premature close
8 timing stage:rollbackFailedOptional Completed in 0ms
9 timing stage:runTopLevelLifecycles Completed in 19748ms
10 verbose stack Error: premature close
10 verbose stack     at PassThrough.onclose (/usr/local/lib/node_modules/npm/node_modules/end-of-stream/index.js:47:67)
10 verbose stack     at PassThrough.emit (events.js:326:22)
10 verbose stack     at emitCloseNT (internal/streams/destroy.js:81:10)
10 verbose stack     at processTicksAndRejections (internal/process/task_queues.js:79:21)
11 verbose cwd /Users/penelopefudd/work/former2/cli
12 verbose Darwin 19.6.0
13 verbose argv "/usr/local/Cellar/node/14.6.0/bin/node" "/usr/local/bin/npm" "install" "iann0036/former2" "-g"
14 verbose node v14.6.0
15 verbose npm  v6.14.7
16 error premature close
17 verbose exit [ 1, true ]

# Tried all of these:
$ npm install iann0036/former2
$ npm install https://github.com/iann0036/former2
$ npm install https://github.com/iann0036/former2.git
$ npm install https://github.com/iann0036/former2/cli
$ npm install https://github.com/iann0036/former2/tree/master/cli
$ npm install https://github.com/iann0036/former2/tree/master/cli/package.json
$ npm install https://raw.githubusercontent.com/iann0036/former2/master/cli/package.json

I then downloaded the repo and tried locally, with different results:

$ git clone  git@github.com:iann0036/former2.git; cd former2 ; npm install cli
npm WARN saveError ENOENT: no such file or directory, open '/private/tmp/former2/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/private/tmp/former2/package.json'
npm WARN former2 No description
npm WARN former2 No repository field.
npm WARN former2 No README data
npm WARN former2 No license field.

+ former2@0.2.47
added 25 packages from 73 contributors and audited 25 packages in 2.118s
found 0 vulnerabilities

$ former2
internal/fs/utils.js:298
    throw err;
    ^

Error: ENOENT: no such file or directory, open '/Users/penelopefudd/work/former2/cli/deepmerge.js'
    at Object.openSync (fs.js:465:3)
    at Object.readFileSync (fs.js:368:35)
    at Object.<anonymous> (/Users/penelopefudd/work/former2/cli/main.js:46:9)
    at Module._compile (internal/modules/cjs/loader.js:1236:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1257:10)
    at Module.load (internal/modules/cjs/loader.js:1085:32)
    at Function.Module._load (internal/modules/cjs/loader.js:950:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47 {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/Users/penelopefudd/work/former2/cli/deepmerge.js'
}

What am I doing wrong?

[PenelopeFudd]

I found out what I'm doing wrong:

$ git clone  git@github.com:iann0036/former2.git
$ cd former2/cli
$ former2   # command not found
$ npm install -g .
$ former2    # error (deepmerge.js not found)
$ npm install .
$ former2    # Works!

Apparently when you do 'npm install -g .' it doesn't run the "prepare" script; 'npm install .' does. :-)

[jlandowner]

Hi, I've opened a PR to add filtering examples.
It describes how to use the new option --service and the existing options --exclude-service and --search-filter.

Also I have already added the docker image easy to use latest Former2 CLI for users who are not familiar with npm and node.js.
You can see how to use docker in the Former2 CLI docs.

I hope these help you.