Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly normalize Windows package paths. #392

Merged
merged 6 commits into from
Apr 26, 2016

Conversation

jrick
Copy link
Contributor

@jrick jrick commented Apr 22, 2016

When the name passed to util.NormalizeName contains backslashes from a
Windows filepath, the backslashes must be first converted to forward
slashes (used by all package names, including on Windows). This fixes
the strings.TrimPrefix call used to strip the repo root from the
subpackage.

As a result of this, subpackages in lockfiles are written identically
on Windows as other Unix platforms and no duplicate subpackages are
introduced.

Fixes #389.

When the name passed to util.NormalizeName contains backslashes from a
Windows filepath, the backslashes must be first converted to forward
slashes (used by all package names, including on Windows).  This fixes
the strings.TrimPrefix call used to strip the repo root from the
subpackage.

As a result of this, subpackages in lockfiles are written identically
on Windows as other Unix platforms and no duplicate subpackages are
introduced.

Fixes Masterminds#389.
@@ -311,6 +311,7 @@ func NormalizeName(name string) (string, string) {
}
}

name = filepath.ToSlash(name)
root := GetRootFromPackage(name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we get from this? The first thing GetRootFromPackage does is run filepath.ToSlash. GetRootFromPackage is sometimes called outside NormalizeName.

Is there a problem in the latest code that needs to be addressed?

Copy link
Contributor Author

@jrick jrick Apr 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that name contains backslashes. The root is normalized, but the full name is not, so the prefix trim doesn't work as intended.

With github.com\foo\bar\baz as input:

// name == "github.com\foo\bar\baz"
root := GetRootFromPackage(name) // root == "github.com/foo/bar"
extra := strings.TrimPrefix(name, root) // extra == "github.com\foo\bar\baz"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the bug I introduced now.

@jrick
Copy link
Contributor Author

jrick commented Apr 22, 2016

Oops, intended to add a testcase for this and it fails because it just concats the root + extra, which no longer matches the initial input. Fixing shortly.

@mattfarina
Copy link
Member

So, this is entirely a micro-optimization but...

Since the root name is already being normalized then the extra is the only part that needs to have it done. Can that happen here instead?

Since both return values are now checked explicitly, remove check that
the concatinated package path matches the original input, since it
won't always (the point of this function is to normalize input).
@jrick
Copy link
Contributor Author

jrick commented Apr 22, 2016

No. That will convert the backslashes to forward slashes, but the returned subpackage will still contain the full root since the TrimPrefix just above it did not remove the root.

@jrick
Copy link
Contributor Author

jrick commented Apr 22, 2016

I'm not sure why it's failing on Travis now. Passes locally.

@jrick
Copy link
Contributor Author

jrick commented Apr 22, 2016

Oh.. I see.

From filepath/path.go:

func ToSlash(path string) string {
    if Separator == '/' {
        return path
    }
    return strings.Replace(path, string(Separator), "/", -1)
}

Separator is os.PathSeparator so this has different behavior on Windows vs Unix. Specifically, on Unix, it won't replace backslashes with forward slashes since backslashes are not a separator character on those platforms, causing the new test to fail there.

What do you want to do in this case? Should glide replace uses of filepath.ToSlash with its own version that works consistently on all platforms?

@mattfarina
Copy link
Member

Maybe we need to tweak the use of normalizeSlash.

@jrick
Copy link
Contributor Author

jrick commented Apr 22, 2016

Looks like we could just use that instead of filepath.ToSlash.

Suggestions on how to use it though? it's unexported in another package. Copy it into util?

This one will replace backslashes with forward slashes on all
platforms.  If there exists a platform where os.PathSeparator is not a
forward or backwards slash, also replace those.
v = strings.Replace(v, "\\", "/", -1)
if os.PathSeparator != '\\' && os.PathSeparator != '/' {
v = strings.Replace(v, string(os.PathSeparator), "/", -1)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an operating system with the path separator other than / or \ that Go supports? Or that's in wide use? If so that's a separate issue because other things will fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not to my knowledge. Should this only replace backslashes then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using v = strings.Replace(v, "\\", "/", -1) should be enough.

@mattfarina mattfarina merged commit e31a433 into Masterminds:master Apr 26, 2016
@mattfarina
Copy link
Member

@jrick thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

glide update writes subpackages in lockfile differently on Windows than Unix
2 participants