Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for Symlinks in all Tar decompressors #192

Closed
wants to merge 1 commit into from
Closed

Adds support for Symlinks in all Tar decompressors #192

wants to merge 1 commit into from

Conversation

prologic
Copy link
Contributor

@prologic prologic commented Jul 8, 2019

This is useful when using Nomad's Artefact mechanisms which internally
uses hashicorp/go-getter to unpack Tar archives that are built from
container root file systems (which usually contain symlinks for runtime
libraries
).

Fixes #60 and supercedes the very old PR #37

(We use a variant of this patch in our production systems and we need to
get this into upstream as soon as conveniently possible)

NB: This uses the
securejoin library to
prevent path escaping and other possible security vulnerabilities that
come with concatenating paths.

@prologic
Copy link
Contributor Author

prologic commented Jul 8, 2019

This is a redo of #171 that was reverted due to security issues.

@prologic
Copy link
Contributor Author

prologic commented Jul 8, 2019

Fixed merge conflicts

This is useful when using Nomad's Artefact mechanisms which internally
uses `hashicorp/go-getter` to unpack Tar archives that are built from
container root file systems (*which usually contain symlinks for runtime
libraries*).

Fixes #60 and supercedes the very old PR #37

(We use a variant of this patch in our production systems and we need to
get this into upstream as soon as conveniently possible)

**NB:** This uses the
[securejoin](https://github.com/cyphar/filepath-securejoin) library to
prevent path escaping and other possible security vulnerabilities that
come with concatenating paths.
@prologic
Copy link
Contributor Author

prologic commented Jul 8, 2019

Please note: There are some failing tests in master that don't appear to have anything to do with my change so CI is failing on a couple of tests.

prologic@Jamess-MacBook
Mon Jul 08 17:05:40
~/Contributions/go-getter
 (fix_symlink_support) 0
$ go test .
--- FAIL: TestGCSGetter (0.00s)
    get_gcs_test.go:40: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestGCSGetter_subdir (0.00s)
    get_gcs_test.go:60: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestGCSGetter_GetFile (0.00s)
    get_gcs_test.go:81: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestGCSGetter_ClientMode_dir (0.00s)
    get_gcs_test.go:113: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestGCSGetter_ClientMode_file (0.00s)
    get_gcs_test.go:129: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestGCSGetter_ClientMode_notfound (0.00s)
    get_gcs_test.go:146: err: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
--- FAIL: TestS3Getter (1.73s)
    get_s3_test.go:36: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: 11DDA457B56600C3, host id: sA7AQAoVBnEgNG2bf1zpLA/58Mgsv37rbhXssJp60ENTlgvrVY5mV4l++/B8dUoShECJwAyOQC0=
--- FAIL: TestS3Getter_subdir (0.31s)
    get_s3_test.go:54: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: 69B5231C0C23B5C6, host id: RO7mQylIrYGEc1ce8QUUQIMUQk+4rxTOpLPiAmPLYs/CzewexSuTqlI1r/iwnNPuGBn7XUc+icQ=
--- FAIL: TestS3Getter_GetFile (0.31s)
    get_s3_test.go:73: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: FBD43BFAA71F4590, host id: p7w2W/ncQ8JK9jVNh3Lg0JTixpGQsDXqzMhtxujON8OAqidkhax0XCfDNe0uwzC8APT5EZ5tfUU=
--- FAIL: TestS3Getter_ClientMode_dir (0.28s)
    get_s3_test.go:121: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: 080AC40FE6783DD3, host id: EhDdfO8u7kPVD0fCGpxyNC2dSDGeK3anAW84axd0lOLnZkD3IStbXr7yDK3Do+RgR4NAC9A+9Ro=
--- FAIL: TestS3Getter_ClientMode_file (0.32s)
    get_s3_test.go:135: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: 6F264D4D121ACDD0, host id: BPmZUCpIfLZ1KTBnsWOUv8jhEfleTYWIRVpu/+RncCXKaOUe0KWED2ff8WkyDWX93Xgsg4L/2gU=
--- FAIL: TestS3Getter_ClientMode_notfound (0.29s)
    get_s3_test.go:153: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: A00E63C4C3393C66, host id: BOIwyoe400jgx9TikssmBTJA9b7eRK/DYyI4RIGfQS91klBRXGrxNnsOUGJrfztYr2rdmdZgQkk=
--- FAIL: TestS3Getter_ClientMode_collision (0.34s)
    get_s3_test.go:168: err: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
        	status code: 403, request id: 252BD05F6E948683, host id: a+Ai//NKt8yP/+PI1ZzJu61xdE+7WS+ODM/oLtDIYoKa7VEqSFmzctFZDmIPjGt+VCooCMzxAF8=
FAIL
FAIL	github.com/hashicorp/go-getter	21.062s

@prologic
Copy link
Contributor Author

prologic commented Jul 8, 2019

Oh on 2nd thoughts the above failures are likely caused by my not having said Google Cloud and AWS credentials. However I'm not going to go setup accounts just to make those pass :) (they should be mocking out externals calls to those APIs)

@prologic
Copy link
Contributor Author

🏓

@mdeggies mdeggies changed the base branch from master to main October 23, 2020 00:54
@prologic prologic closed this Oct 25, 2020
@prologic prologic deleted the fix_symlink_support branch October 25, 2020 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for symlink when untarring an archive
1 participant