Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: remove parallel walk #5180

Merged
merged 30 commits into from
Apr 17, 2024
Merged

Conversation

knqyf263
Copy link
Collaborator

@knqyf263 knqyf263 commented Sep 13, 2023

Description

Since filepath.WalkDir is fast enough now, this PR removes github.com/saracen/walker. Also, it defines the Walker interface so that other tools can override the behavior.

Related issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263 knqyf263 requested a review from tonaim September 13, 2023 14:06
@knqyf263 knqyf263 self-assigned this Sep 13, 2023
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263
Copy link
Collaborator Author

knqyf263 commented Jan 6, 2024

@santhosh1729 I think we'll merge this PR now, and you can give us feedback anytime.

Signed-off-by: knqyf263 <knqyf263@gmail.com>
Copy link

github-actions bot commented Apr 2, 2024

This PR is stale because it has been labeled with inactivity.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Apr 2, 2024
@github-actions github-actions bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Apr 16, 2024
@knqyf263 knqyf263 requested a review from DmitriyLewen as a code owner April 16, 2024 04:09
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263
Copy link
Collaborator Author

@DmitriyLewen Could you quickly take a look? Aqua confirmed it, and it should be fine, but I want to double-check it.

Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pkg/fanal/artifact/vm/vm.go Outdated Show resolved Hide resolved
Copy link
Contributor

@nikpivkin nikpivkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263 knqyf263 enabled auto-merge April 17, 2024 18:02
@knqyf263 knqyf263 added this pull request to the merge queue Apr 17, 2024
Merged via the queue into aquasecurity:main with commit 13e72ec Apr 17, 2024
12 checks passed
@knqyf263 knqyf263 deleted the unify_walk branch April 17, 2024 18:45
fl0pp5 pushed a commit to altlinux/trivy that referenced this pull request May 6, 2024
Signed-off-by: knqyf263 <knqyf263@gmail.com>
skipFiles []string
skipDirs []string
var defaultSkipDirs = []string{
"**/.git",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question. Why you just ignore the .git folder?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The git dir shouldn't have files Trivy wants to find, and it is time-consuming to walk the dir.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The git dir shouldn't have files Trivy wants to find, and it is time-consuming to walk the dir.

thank u for answering. But I want to say something. just like what github official does.

https://github.com/actions/checkout/blob/44c2b7a8a4ea60a981eaca3cf939b5f4305c123b/src/git-auth-helper.ts#L286

They allows save PAT tokens or request OAuth tokens in this actions. And backup this So some careless people do like

FROM node:16
RUN mkdir /app
COPY . /app
RUN yarn install
EXPOSE 80
CMD ["node", "/app/app.js"]

And Python/PHP/Javascript programmers always do like this. : )

So at least whitelist the .git/config .git/modules/[any]/config and blacklist something like .git/objects

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FROM node:16
RUN mkdir /app
COPY . /app
RUN yarn install
EXPOSE 80
CMD ["node", "/app/app.js"]

What's wrong in this Dockerfile? PAT will be deleted in the post-job step.

Copy link

@Esonhugh Esonhugh May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FROM node:16
RUN mkdir /app
COPY . /app
RUN yarn install
EXPOSE 80
CMD ["node", "/app/app.js"]

What's wrong in this Dockerfile? PAT will be deleted in the post-job step.

Yes post job. But docker image building is before it :)
It's

graph TD;
START ---> github.token.start(register a Temp github.token) ---> checkout --->  checkout-1(PAT sets so use PAT, do submodules or sth, save it in .git/config) ---> docker-image-building ---> docker-image-uploading(docker image tag login push) ---> post-docker-login(post docker login if login action) ---> post-checkout(post checkout action, clean up if not set persisence, delete local config) ---> End(Action Ends and revoke the Temp github.token)
Loading

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It's interesting. Did you ensure PAT can be exposed to a container image?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It's interesting. Did you ensure PAT can be exposed to a container image?

pretty sure. Play with cross repos ref submodules they must do like that. I can create an environment. : )

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created an issue. Thanks!
#6699

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created an issue. Thanks! #6699

nice work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

errorCallback is not handled in fast scanning
7 participants