Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

683 windows filepath #735

Merged
merged 11 commits into from
Jan 6, 2022
Merged

683 windows filepath #735

merged 11 commits into from
Jan 6, 2022

Conversation

spiffcs
Copy link
Contributor

@spiffcs spiffcs commented Jan 6, 2022

Support Windows Directory Resolver

  • Add function that converts windows to posix functionality
  • Add function that converts posix to windows
  • Add build tags to remove windows developer environment errors
  • redact carriage return specific windows issues

Summary

When scanning on windows using the dir directive the current directory resolver implementation cannot lookup file nodes that match glob patterns in the filetree. By converting Windows filepaths on the way in so that their locations match the posix standard, we allow for syft to match on directory resolver matching criteria.

Paths are then translated back into windows paths on the way out so the presentation layer shows users the correct on disk details for the given SBOM items.

Considertations

  • Exclude functionality needs to be updated in a separate PR
  • Location data regarding Realpath and VirtualPath is sometimes incorrect. I'm currently trying to track down where this bug exists. Should VirtualPath ever just be the Posix representation as found in the file tree or should we be converting these back?

Screenshot demo showing functionality on Windows 11:

image

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: spiffcs <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: spiffcs <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: spiffcs <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
@spiffcs spiffcs force-pushed the 683-windows-filepath branch from 6d63d8b to ce9e958 Compare January 6, 2022 01:05
@github-actions
Copy link

github-actions bot commented Jan 6, 2022

Benchmark Test Results

Benchmark results from the latest changes vs base branch
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.77ms ± 8%    1.34ms ± 1%  -24.26%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.57ms ± 2%    3.05ms ± 0%  -14.75%  (p=0.016 n=5+4)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.13ms ± 4%    0.94ms ± 0%  -17.05%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2        1.03ms ± 4%    0.83ms ± 1%  -19.48%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                    1.22ms ± 9%    0.96ms ± 1%  -21.89%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                     1.13ms ± 3%    0.86ms ± 1%  -23.75%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      16.3ms ± 7%    12.5ms ± 1%  -23.20%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.66ms ± 4%    1.27ms ± 1%  -23.58%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          2.24µs ± 7%    1.96µs ± 2%  -12.84%  (p=0.008 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               252kB ± 0%     252kB ± 0%   -0.20%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            1.05MB ± 0%    1.06MB ± 0%   +1.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     227kB ± 0%     230kB ± 0%   +1.25%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         207kB ± 0%     207kB ± 0%   -0.20%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     253kB ± 0%     252kB ± 0%   -0.35%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      233kB ± 0%     234kB ± 0%   +0.72%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.50MB ± 0%    3.50MB ± 0%     ~     (p=0.222 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.30MB ± 0%    1.30MB ± 0%   +0.40%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            560B ± 0%      560B ± 0%     ~     (all equal)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               6.28k ± 0%     6.31k ± 0%   +0.48%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             21.0k ± 0%     21.3k ± 0%   +1.66%  (p=0.008 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     5.98k ± 0%     6.04k ± 0%   +0.96%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         5.32k ± 0%     5.32k ± 0%   +0.04%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     7.04k ± 0%     7.05k ± 0%   +0.07%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      6.78k ± 0%     6.78k ± 0%   +0.03%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                       64.9k ± 0%     65.3k ± 0%   +0.55%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      7.28k ± 0%     7.35k ± 0%   +0.97%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            13.0 ± 0%      13.0 ± 0%     ~     (all equal)

Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
@spiffcs spiffcs force-pushed the 683-windows-filepath branch 2 times, most recently from c46876b to 93a212d Compare January 6, 2022 01:38
Signed-off-by: spiffcs <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
@spiffcs spiffcs force-pushed the 683-windows-filepath branch from 93a212d to e0a8efd Compare January 6, 2022 02:06
@spiffcs spiffcs marked this pull request as ready for review January 6, 2022 16:13
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
if runtime.GOOS == WindowsOS {
filePath = posixToWindows(filePath)
}
return file.NewLazyReadCloser(filePath), nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice find

Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice work --I don't think we need to add window-specific tests for this work quite yet. We can take that on in the future.

@spiffcs spiffcs merged commit 01dc78c into main Jan 6, 2022
@spiffcs spiffcs deleted the 683-windows-filepath branch January 6, 2022 16:39
spiffcs added a commit that referenced this pull request Jan 19, 2022
…hub.com/hectorj2f/syft into hectorj2f/add_dependencies_to_cyclonedx

* 'hectorj2f/add_dependencies_to_cyclonedx' of https://github.com/hectorj2f/syft: (29 commits)
  Improve CycloneDX format output (#710)
  Add additional PHP metadata (#753)
  Update Syft formats for SyftJson (#752)
  Add support for "file" source type in syftjson unmarshaling (#750)
  remove contains file from spdx dependency generation
  support .sar for java ecosystem (#748)
  Start developer documentation (#746)
  Align SPDX export more with SPDX 2.2 specification (#743)
  Replace distro type (#742)
  update goreleaser with windows checksums (#740)
  bump stereoscope version to remove old containerd (#741)
  Add support for multiple output files in different formats (#732)
  Add support for searching for jars within archives (#734)
  683 windows filepath (#735)
  Fix CPE encode/decode when it contains special chars (#714)
  support .par for java ecosystems (#727)
  Add arm64 support to install script (#729)
  Revert "bump goreleaser to v1.2 (#720)" (#731)
  Add a version flag (#722)
  Add lpkg as java package format (#694)
  ...
fengshunli pushed a commit to fengshunli/syft that referenced this pull request Jan 24, 2022
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Signed-off-by: fsl <1171313930@qq.com>
spiffcs added a commit that referenced this pull request Jan 24, 2022
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
spiffcs added a commit that referenced this pull request Jan 25, 2022
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
jonasagx pushed a commit to jonasagx/syft that referenced this pull request Jan 28, 2022
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants