Lockfile autodetection for test, wizard, protect and monitor #228

lili2311 · 2018-10-02T10:17:02Z

Ready for review
Follows CONTRIBUTING rules
Reviewed by Snyk internal team

What does this PR do?

Introduce yarn & package-lock.json autodetection to use lockfile only (without node_modules traversal)
Leave node_module traversal as fall back for package.json & shrinkwrap projects

The state of NPM lockfile support in this pr:

projects with package.json & package-lock.json will be tested via the new lockfile parser lib for snyk protect, test, monitor

The state of Yarn lockfile support in this pr:

projects with package.json & yarn.lock will be tested via the new lockfile parser lib for snyk test, monitor
projects with package.json & yarn.lock will be tested via the old node_modules traversal lib (resolve-deps) for snyk protect and wizard
projects with package.json & yarn.lock on node@4 will be tested via the old node_modules traversal lib (resolve-deps) for all snyk commands. Yarn parser lib we use is not compatible with node@4

Where should the reviewer start?

most changes are in the npm & yarn plugin to test if a project is lockfile based before choosing which way to generate the tree

How should this be manually tested?

run test, monitor, protect and wizard locally on a lockfile based project

What are the relevant tickets?

https://snyksec.atlassian.net/browse/SC-6240
https://snyksec.atlassian.net/browse/SC-6277

lili2311 · 2018-10-02T10:55:09Z

WORK IN PROGRESS
Need to add some changes to wizard to better support this

lili2311 · 2018-10-03T13:23:56Z

src/cli/commands/protect/wizard.js

+      if (!nodeModulesExist) {
+        // throw a custom error
+        throw new Error(
+          'Missing node_modules folder: we can\'t patch without having installed packages.' +


test can now be performed without node_modules so change the wording to better reflect why we need node_modules during wizard for all node lockfiles & non lockfiles project types

lili2311 · 2018-10-03T13:24:44Z

src/cli/commands/protect/wizard.js

+            if (isLockFileBased) {
+              // we need to trigger a lockfile update after adding snyk
+              // as a dep
+              return protect.update(['snyk'], live, packageManager);


if a project is lockfile based we need to also update the lockfile after adding in snyk as a dep, without this the files go out of sync and fail a re-test on monitor command

lili2311 · 2018-10-03T13:25:25Z

src/lib/detect.js

@@ -10,6 +10,7 @@ var chalk = require('chalk');

 var DETECTABLE_FILES = [
  'yarn.lock',
+  'package-lock.json',


change order to ensure lockfile based projects get processed as suck

lili2311 · 2018-10-03T14:17:25Z

src/cli/commands/protect/index.js

+  // but not the latest node-lockfile-parser
+  // HACK: if yarn set traverseNodeModules option to
+  // bypass lockfile test for wizard
+  options.traverseNodeModules = true;


Fallback for yarn projects where we currently do not support traversing node_modules with the new lockfile-parser

lili2311 · 2018-10-03T14:17:43Z

src/cli/commands/protect/wizard.js

+  // but not the latest node-lockfile-parser
+  // HACK: if yarn set traverseNodeModules option to
+  // bypass lockfile test for wizard
+  if (targetFile.endsWith('yarn.lock')) {


Fallback for yarn projects where we currently do not support traversing node_modules with the new lockfile-parser

miiila · 2018-10-04T11:48:16Z

Great job, waiting for tests to pass before final approvement.

src/cli/commands/protect/wizard.js

miiila

just a small comment for now :-)

miiila · 2018-10-04T11:27:09Z

src/cli/commands/protect/wizard.js

-          return snyk.test(cwd, options).then(function (res) {
-            if (alerts.hasAlert('tests-reached') && res.isPrivate) {
-              return;
+          var intro = __dirname + '/../../../../help/wizard-intro.txt';


- var + const

Lockfile autodetection functionality wuth fall back for yarn.lock files on node < 6 && yarn//lock wizard flow.

snyksec · 2018-10-08T14:40:55Z

🎉 This PR is included in version 1.102.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

lili2311 self-assigned this Oct 2, 2018

lili2311 force-pushed the fix/lockfiles-monitor branch 2 times, most recently from 97535e1 to 5a32fd3 Compare October 2, 2018 10:49

lili2311 force-pushed the fix/lockfiles-monitor branch from 5a32fd3 to aca149e Compare October 2, 2018 11:05

michael-go added the 🐣 in progress label Oct 3, 2018

lili2311 force-pushed the fix/lockfiles-monitor branch 2 times, most recently from ddc11cc to e2e8484 Compare October 3, 2018 12:32

lili2311 commented Oct 3, 2018

View reviewed changes

lili2311 force-pushed the fix/lockfiles-monitor branch from e2e8484 to f398253 Compare October 3, 2018 13:27

lili2311 commented Oct 3, 2018

View reviewed changes

adrukh reviewed Oct 4, 2018

View reviewed changes

src/cli/commands/protect/wizard.js Outdated Show resolved Hide resolved

miiila reviewed Oct 4, 2018

View reviewed changes

lili2311 force-pushed the fix/lockfiles-monitor branch 7 times, most recently from d41a593 to 81f337e Compare October 5, 2018 15:19

lili2311 added 6 commits October 5, 2018 17:47

feat: autodetection for test, monitor, wizard & protect

20b8ccc

Lockfile autodetection functionality wuth fall back for yarn.lock files on node < 6 && yarn//lock wizard flow.

fix: Fall back to config org if no option passed

99021be

chore: var => const

5f4399b

fix: Update runtime helper path

39976bc

test: Update tests

6faa02b

tests: Fixes for outdated tests

5307275

lili2311 deleted the fix/lockfiles-monitor branch October 8, 2018 13:52

This was referenced Aug 30, 2022

[Snyk] Upgrade micromatch from 4.0.2 to 4.0.5 #3663

Closed

[Snyk] Upgrade micromatch from 4.0.2 to 4.0.5 #3797

Closed

[Snyk] Upgrade micromatch from 4.0.2 to 4.0.5 #3808

Closed

[Snyk] Upgrade micromatch from 4.0.2 to 4.0.5 #3815

Closed

leongold mentioned this pull request Nov 30, 2022

[Snyk] Fix for 5 vulnerabilities leongold/snyk#25

Open

This was referenced Nov 30, 2022

[Snyk] Fix for 7 vulnerabilities mikolajroszak/mikolaj.com-snyk#38

Open

[Snyk] Fix for 6 vulnerabilities allRounderWiser/snyk#153

Open

david-j-bailey mentioned this pull request Nov 30, 2022

[Snyk] Fix for 7 vulnerabilities david-j-bailey/snyk#35

Open

This was referenced Nov 30, 2022

[Snyk] Fix for 7 vulnerabilities carrot1213/cli#10

Open

[Snyk] Fix for 6 vulnerabilities thexdesk/snyk#14

Open

froscheimer mentioned this pull request Dec 1, 2022

[Snyk] Fix for 6 vulnerabilities froscheimer/snyk#186

Open

snyk-bot mentioned this pull request Dec 1, 2022

[Snyk] Fix for 4 vulnerabilities jeremy001181/snyk#15

Open

sapans30 mentioned this pull request Dec 1, 2022

[Snyk] Fix for 7 vulnerabilities sapans30/snyk#36

Open

snyk-bot mentioned this pull request Dec 1, 2022

[Snyk] Fix for 7 vulnerabilities dawit-snyk/snyk#15

Open

hiep98 mentioned this pull request Dec 1, 2022

[Snyk] Fix for 7 vulnerabilities hiep98/snyk#28

Open

snyk-bot mentioned this pull request Dec 1, 2022

[Snyk] Fix for 7 vulnerabilities toharzand/snyk#41

Open

romathur mentioned this pull request Dec 1, 2022

[Snyk] Fix for 5 vulnerabilities romathur/snyk#147

Open

This was referenced Dec 1, 2022

[Snyk] Fix for 6 vulnerabilities mmmd03/snyk#19

Open

[Snyk] Fix for 7 vulnerabilities laozhudetui/snyk#41

Open

RotemMiara mentioned this pull request Dec 1, 2022

[Snyk] Fix for 7 vulnerabilities RotemMiara/cli#13

Open

AliADDOU mentioned this pull request Jun 30, 2023

[Snyk] Security upgrade tough-cookie from 2.3.4 to 4.1.3 AliADDOU/cli#50

Open

mikolajroszak mentioned this pull request Oct 16, 2023

[Snyk] Upgrade ora from 5.4.0 to 7.0.1 mikolajroszak/mikolaj.com-snyk#78

Open

carrot1213 mentioned this pull request Nov 30, 2023

[Snyk] Fix for 1 vulnerabilities carrot1213/cli#37

Open

shawna-alpdemir mentioned this pull request Nov 30, 2023

[Snyk] Fix for 1 vulnerabilities shawna-alpdemir/cli#26

Open

laozhudetui mentioned this pull request Nov 30, 2023

[Snyk] Fix for 1 vulnerabilities laozhudetui/snyk#66

Open

mikolajroszak mentioned this pull request Jan 13, 2024

[Snyk] Upgrade ora from 5.4.0 to 8.0.1 mikolajroszak/mikolaj.com-snyk#151

Open

gitworkflows mentioned this pull request Jan 19, 2024

[Snyk] Upgrade ora from 5.4.0 to 8.0.1 gsoc2/snyk#3

Open

claireksnyk mentioned this pull request Jan 25, 2024

[Snyk] Upgrade ora from 5.4.0 to 8.0.1 claireksnyk/cli#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lockfile autodetection for test, wizard, protect and monitor #228

Lockfile autodetection for test, wizard, protect and monitor #228

lili2311 commented Oct 2, 2018 •

edited

Loading

lili2311 commented Oct 2, 2018

lili2311 Oct 3, 2018

lili2311 Oct 3, 2018

lili2311 Oct 3, 2018

lili2311 Oct 3, 2018

lili2311 Oct 3, 2018

miiila commented Oct 4, 2018

miiila left a comment

miiila Oct 4, 2018

snyksec commented Oct 8, 2018

Lockfile autodetection for test, wizard, protect and monitor #228

Lockfile autodetection for test, wizard, protect and monitor #228

Conversation

lili2311 commented Oct 2, 2018 • edited Loading

What does this PR do?

Where should the reviewer start?

How should this be manually tested?

What are the relevant tickets?

lili2311 commented Oct 2, 2018

lili2311 Oct 3, 2018

Choose a reason for hiding this comment

lili2311 Oct 3, 2018

Choose a reason for hiding this comment

lili2311 Oct 3, 2018

Choose a reason for hiding this comment

lili2311 Oct 3, 2018

Choose a reason for hiding this comment

lili2311 Oct 3, 2018

Choose a reason for hiding this comment

miiila commented Oct 4, 2018

miiila left a comment

Choose a reason for hiding this comment

miiila Oct 4, 2018

Choose a reason for hiding this comment

snyksec commented Oct 8, 2018

lili2311 commented Oct 2, 2018 •

edited

Loading