Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support setup.py & Pipfile dependencies in the python docker images #925

Merged
merged 5 commits into from
Jan 28, 2020
Merged

feat: Support setup.py & Pipfile dependencies in the python docker images #925

merged 5 commits into from
Jan 28, 2020

Conversation

lili2311
Copy link
Contributor

@lili2311 lili2311 commented Dec 24, 2019 •
edited
Loading

  • Ready for review
  • Follows CONTRIBUTING rules
  • Reviewed by Snyk internal team

What does this PR do?

  • Setup.py support was added in the python plugin but the image doesn't install the dependencies
  • Custom requirements.txt support
  • Improve docs / drop repetitive examples

How should this be manually tested?

Check the updated README for more examples:

Build the local image from the docker dir:
docker build -f Dockerfile.python-3 . -t snyk/snyk-cli:python-3-local

Run on a setup.py project:

docker run -it -e "SNYK_TOKEN=*******" -e "TARGET_FILE=setup.py" -v "$PWD:/project" snyk/snyk-cli:python-3-local

Tickets

https://github.com/snyk/snyk/issues/786
https://github.com/snyk/snyk/issues/332

@lili2311 lili2311 requested a review from a team as a code owner December 24, 2019 17:24
@ghost ghost requested review from gitphill and orsagie December 24, 2019 17:24
@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch from 21b91a8 to c210602 Compare December 27, 2019 10:09
lili2311
lili2311 commented Dec 27, 2019
View reviewed changes
docker/docker-python-entrypoint.sh
echo "Target file = ${TARGET_FILE}"

case $MANIFEST_NAME in
*req*.txt)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support custom named requirement files

lili2311
lili2311 commented Dec 27, 2019
View reviewed changes
docker/docker-python-entrypoint.sh
*setup.py)
echo "Installing dependencies from setup.py"
pip install -U -e "${PROJECT_PATH}"
;;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support setup.py

@lili2311 lili2311 self-assigned this Dec 27, 2019
@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch from 40671f7 to fb442df Compare December 27, 2019 11:19
@lili2311 lili2311 mentioned this pull request Dec 27, 2019
Closed
3 tasks
@lili2311 lili2311 changed the title feat: Support setup.py in the python docker images feat: Support setup.py & Pipfile dependencies in the python docker images Dec 27, 2019
@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch from 1572d67 to 67c926b Compare December 27, 2019 15:12
adrukh
adrukh reviewed Dec 29, 2019
View reviewed changes
README.md Outdated Show resolved Hide resolved
README.md Outdated
@@ -81,37 +85,52 @@ The general format of tags is [snyk-version]-[package-manager]-[package-manager-
[package-manager] - One of the available package managers (e.g: npm, mvn, gradle, etc...).
[package-manager-version] - The version of the package manager that is installed inside the image.

Please see the following examples on how to run Snyk inside docker:

**Note** We will need to mount the project root folder when running the image so that Snyk can access the code within the container. The host project folder will be mounted to `/project` on the container and will be used to read the dependencies file and write results for CI builds.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'Tone of voice' comment - using we is kind of an exception in this text. I suggest to avoid it as it is confusing. The documentation should be read as a set of instructions, less of a story?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed we

README.md Outdated Show resolved Hide resolved
README.md
@@ -70,9 +72,11 @@ The following environment variables can be used when running the container on do

- `SNYK_TOKEN` - Snyk API token, obtained from [https://app.snyk.io/account](https://app.snyk.io/account).
- `USER_ID` - [OPTIONAL] Current user ID on the host machine. If not provided will take the user ID of the currently running user inside the container. This is used for CI builds such as Jenkins where we are running with a non-privileged user and want to allow the user to access the mounted project folder.
- `MONITOR` - [OPTIONAL] If set, tells the image that we want to run `snyk monitor` after running `snyk test`.
- `MONITOR` - [OPTIONAL] If set, will generate an html report via `snyk-to-html` and runs `snyk monitor` after running `snyk test`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, much better! More consistent tone of voice, I find the use of we confusing.

README.md Outdated Show resolved Hide resolved
README.md Outdated
-v "<PROJECT_DIRECTORY>:/project"
snyk/snyk-cli:npm test --org=my-org-name
```
`snyk/snyk-cli:npm` - [see all available `npm` tagged images](https://hub.docker.com/r/snyk/snyk-cli/tags?page=1&name=npm)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonderful reference to the relevant tags!

README.md Outdated Show resolved Hide resolved
docker/docker-entrypoint.sh
@@ -101,6 +101,7 @@ fi
runCmdAsDockerUser "touch \"${PROJECT_PATH}/${PROJECT_FOLDER}/${HTML_FILE}\""

if [ -n "$MONITOR" ]; then
echo "Monitoring & generating report ..."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may mess up with our CI plugins logic, which parses the output of this script. Please consult with @snyk/comet

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#925 (comment)

docker/docker-python-entrypoint.sh
elif [ -f "${PROJECT_PATH}/Pipfile" ]; then
installPipfileDeps
fi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The echo lines along this script are great for debugging, but may interfere with our own (or our users') parsing logic that processes the output of this script. Maybe @julienduchesne can shed some light, along with @snyk/comet ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed here this will have no effect https://snyk.slack.com/archives/CHAB91L0N/p1578258775055200?thread_ts=1578073050.054200&cid=CHAB91L0N

@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch 2 times, most recently from 3595bc1 to b64312f Compare January 6, 2020 14:37
gitphill
gitphill reviewed Jan 8, 2020
View reviewed changes
README.md Outdated Show resolved Hide resolved
gitphill
gitphill reviewed Jan 8, 2020
View reviewed changes
README.md Outdated Show resolved Hide resolved
@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch from b64312f to 7623659 Compare January 27, 2020 17:14
@lili2311 lili2311 force-pushed the feat/support-setup-py-docker branch from 7623659 to a0293d0 Compare January 27, 2020 17:16
@lili2311
feat: enable --json for docker cli by default
75f9a7d 
`snyk test` and `snyk monitor` via Docker cli
expect to have json vulnerabilities data to
generate the report. Make `--json` hardcoded
to avoid parsing errors and match the default
expecation.
@lili2311 lili2311 merged commit f02cc8a into master Jan 28, 2020
@lili2311 lili2311 deleted the feat/support-setup-py-docker branch January 28, 2020 09:15
@snyksec
Copy link

snyksec commented Jan 28, 2020

🎉 This PR is included in version 1.284.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adrukh adrukh adrukh left review comments

@gitphill gitphill gitphill approved these changes

@orsagie orsagie Awaiting requested review from orsagie

Assignees

@lili2311 lili2311

Labels
released
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lili2311 @snyksec @adrukh @gitphill