Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix os.path.relpath throwing #164

Merged
merged 2 commits into from
Nov 12, 2018
Merged

Conversation

JornVernee
Copy link
Contributor

@JornVernee JornVernee commented Nov 8, 2018

Replaced calls to os.path.relpath with a safe alternative, since not every path can be made relative on Windows, due to different drives.

This is based on another PR [1], where the author asked someone else to take over since they don't have time to fix the merge conflicts [2].

I tried running mx gate but that throws an unrelated error, so it doesn't look like that works on Windows yet. So be aware that this has not passed gate yet.

[1] : #134
[2] : #134 (comment)

@graalvmbot
Copy link
Collaborator

Hello JornVernee, thanks for contributing a PR to our project!

We use the Oracle Contributor Agreement to make the copyright of contributions clear. We don't have a record of you having signed this yet, based on your email address jbvernee -(at)- xs4all -(dot)- nl. You can sign it at that link.

If you think you've already signed it, please comment below and we'll check.

@JornVernee
Copy link
Contributor Author

I have already signed the OCA: https://openjdk.java.net/census#jvernee

@graalvmbot
Copy link
Collaborator

JornVernee has signed the Oracle Contributor Agreement (based on email address jbvernee -(at)- xs4all -(dot)- nl) so can contribute to this repository.

@JornVernee
Copy link
Contributor Author

JornVernee commented Nov 8, 2018

Hey Doug,

Just to make sure I'd like to wait for the old author to give the OK on me making a separate PR before merging this.

It seems like they have abandoned their own PR, and don't really care, but they did come up with the fix after all.

@dougxc
Copy link
Member

dougxc commented Nov 8, 2018

I think we're fine to proceed based on @okutane 's comment.

@JornVernee
Copy link
Contributor Author

JornVernee commented Nov 8, 2018

@dougxc Ok then, go ahead.

@gilles-duboscq
Copy link
Member

Hi @JornVernee, thanks for taking a stab at this.
Unfortunately we can not simply replace os.path.relpath with maybe_relpath. I would actually say that we can only do it in rare cases.

In particular, there are cases where the logic simply doesn't work if a relative path can not be created, in those it needs to remain an error and the root case has to be fixed.

In many cases it's OK anyway because relapth is used in conditions where a relative path can always be found (in particular when using os.walk).

In other cases, a more complex logic is needed. For example in intellij project generation, when a relative path can be found, the $PROJECT_DIR$ variable should be used and if it can not then an absolute path should be used.

In other places a unix path is used as a short-cut to something else (a relative URL, a logical path in an archive etc.) and an other platform-independent approach should be used.

In order to make progress, I propose that we don't fix everything at once but just fix the ones where maybe_relpath can be used and the intellijinit ones since that was the purpose of the original PR.

@JornVernee
Copy link
Contributor Author

@gilles-duboscq Yeah ok, maybe that was a bit optimistic of me (it was working after all, and together with some other changes I have mx gate passing on the mx suite now).

I will revert the replacement and then try running a few commands to see which lines give the error.

@JornVernee
Copy link
Contributor Author

JornVernee commented Nov 8, 2018

I see travis is running mx gate in strict mode.

When doing that myself I get:

require pylint version = 1.1.x (got 1.8.4)
Pylint not configured correctly. Cannot execute Pylint task.

Should that requirement be bumped up to 1.x.x ?


P.S. Never mind, I think it's just using my python 3 Scripts folder. It doesn't look like my pylint version shown there actually works with python 2.

P.P.S. Okay I installed 1.9.3 version of pylint for python 2, but running that with mx gate is throwing a lot of linter warnings, so I guess that is the reason the version hasn't been bumped up. Any idea where I can get the right version of pylint?

…every path can be made relative on Windows, due to different drives.

rename safe_relpath to maybe_relpath per suggestion

Also rename callsites to maybe_relpath (oops)

Revert replacements to do it again step-by-step to better examine use cases of os.path.relpath

Fix regex to also work with windows path separators.

Use maybe_relpath as a replacement for os.path.relpath a point where `mx ideinit` and `mx gate` are passing.

Added special handling for PROJECTDIR relative paths. Added maybe_relpath to the other dependency branches as well

linter fixes

Removed duplicate '$PROJECT_DIR$/'

replace other uses of PROJECT_DIR and relpath with safe equivalent.

Consolidate code paths

pash relpath_or_absolute calls into make_library since that is where they are joined with PROJECT_DIR (or not)

using os.path.join instead of a hardcoded path separator.

Revert "using os.path.join instead of a hardcoded path separator." since these are URIs and not actual paths

Revert "Revert "using os.path.join instead of a hardcoded path separator." since these are URIs and not actual paths"

Simplified things

remove accidental diffs

undiff more

undiff more

remove spurious withespace
@JornVernee
Copy link
Contributor Author

JornVernee commented Nov 9, 2018

I looked into this and decided to keep the changes simple, @gilles-duboscq or @dougxc can you take another look? I pushed the relpath lookup into make_library so it can either use $PROJECT_DIR$ or an absolute path. (I squashed all my commits, so the commit message looks a bit strange)

I was also able to pass mx build on the graal/compiler suite and run it with mx vm for the first time so that's 👍 . I also ended up fixing #165 in the process (a regex that wasn't matching on Windows path separators).

@gilles-duboscq
Copy link
Member

Any idea where I can get the right version of pylint?

$ pip install 'astroid==1.1.0'
$ pip install 'pylint==1.1.0'

(You have to install strict versions and install astroid first with a strict version although it's an explicit dependency of pylint)

Copy link
Member

@gilles-duboscq gilles-duboscq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. This looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants