Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bazel] Fix mobile-install for python2 #12540

Closed
wants to merge 4 commits into from

Conversation

ThomasCJY
Copy link
Contributor

@ThomasCJY ThomasCJY commented Nov 23, 2020

Background
Fix issue introduced by 1049fe8
It turns out the queue module name is inconsistent across different python versions. We found that:

  • On some macos system:
    • python2 contains both queue and Queue module. All python2 contains Queue module
    • python3 only contains queue module
  • On some Linux system
    • python2 contains only Queue module.
    • python3 only contains queue module

Therefore, some developers are seeing ImportError: No module named queue errors locally on linux machine after using mobile-install.

Change
Import correct Queue module instead

Test
Local test pass

@google-cla
Copy link

google-cla bot commented Nov 23, 2020

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@@ -32,7 +32,7 @@

import hashlib
import os
from queue import Queue
from Queue import Queue
Copy link
Contributor Author

@ThomasCJY ThomasCJY Nov 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use

try:
    import Queue from Queue
except ImportError:
    import queue as Queue 

here to support both python 2 and 3?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try:
    from Queue import Queue
except ImportError:
    import queue as Queue 

This is the syntax that worked when python 2 is used.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ThomasCJY can you change the code to the alternative suggested by @arunkumar9t2 ? That'll make this work in both Pyton2 and 3.

Copy link
Contributor Author

@ThomasCJY ThomasCJY Feb 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks reasonable to me. Unfortunately I lack the necessary permissions to stamp the diff. I think @ahumesky is the right person to loop in here?

@jin jin added the team-Android Issues for Android team label Dec 4, 2020
@nkoroste
Copy link
Contributor

Any updates on this? pretty trivial change...

nkoroste pushed a commit to nkoroste/bazel that referenced this pull request Feb 23, 2021
**Background**
It turns out the queue module name is inconsistent across different python versions. We found that:
* On macos system
  - python2 contains both queue and Queue module. 
  - python3 only contains queue module
* On Linux system
  - python2 contains only Queue module. 
  - python3 only contains queue module 

Therefore, some developers are seeing `ImportError: No module named queue` errors locally on linux machine. 

**Change**
Import correct Queue module instead

**Test**
Local test pass

Upstream PR: bazelbuild#12540
---
Automatic squash commit from https://github.sc-corp.net/Snapchat/bazel/pull/83
Cooled by jchen
@ahumesky
Copy link
Contributor

ahumesky commented Mar 5, 2021

I've imported this change internally, but our linters are pretty unhappy about imports "not at top of file". We seem to be doing this in other places so I think I can work around it.

@ahumesky
Copy link
Contributor

ahumesky commented Mar 9, 2021

looks like this is causing the //tools/android:build_incremental_dexmanifest_test test to fail:

tools/android/build_incremental_dexmanifest.py", line 55, in __init__
    self.queue = Queue()
TypeError: 'module' object is not callable

(and also causing 2 internal tests to fail with the same error)

Looks like this is because the original code from queue import Queue imports the Queue class from the module queue, whereas in the except block, the code import queue as Queue imports the module with the name "Queue".

Are these the 3 scenarios?

  1. from queue import Queue (original. Queue class in queue module)
  2. from Queue import Queue (Queue class is is Queue module)
  3. import Queue (import just module, module is callable to produce Queue object)

At least on my Linux machine, scenario 3 doesn't seem to work. There appears to be both queue and Queue in python2, just queue in python3, and the module isn't callable in any of the cases. I think queue in python2 is from a compatibility package:

>>> print(queue.__file__)
/usr/lib/python2.7/dist-packages/queue/__init__.pyc

So unless something is different on macos, seems this change should be:

try:
  # python2 without compatibility package
  from Queue import Queue
except ImportError:
  # python3
  from queue import Queue

@ThomasCJY
Copy link
Contributor Author

@ahumesky yeah you are right. It's a bit confusing but according to the python doc:

python 2 uses Queue module name and the creation of object is Queue.Queue()
https://docs.python.org/2.7/library/queue.html

python 3 uses queue module name and the creation of object is queue.Queue()
https://docs.python.org/3.8/library/queue.html

so the import syntax should be

try:
  # python2 without compatibility package
  from Queue import Queue
except ImportError:
  # python3
  from queue import Queue

@ThomasCJY
Copy link
Contributor Author

@ahumesky is this good to go?

@bazel-io bazel-io closed this in 48eee8b Apr 22, 2021
Bencodes pushed a commit to lyft/bazel that referenced this pull request May 6, 2021
**Background**
Fix issue introduced by bazelbuild@1049fe8
It turns out the queue module name is inconsistent across different python versions. We found that:
* On some macos system:
  - python2 contains both queue and Queue module. All python2 contains Queue module
  - python3 only contains queue module
* On some Linux system
  - python2 contains only Queue module.
  - python3 only contains queue module

Therefore, some developers are seeing `ImportError: No module named queue` errors locally on linux machine after using mobile-install.

**Change**
Import correct Queue module instead

**Test**
Local test pass

Closes bazelbuild#12540.

PiperOrigin-RevId: 369773133
katre pushed a commit that referenced this pull request Jul 12, 2021
**Background**
Fix issue introduced by 1049fe8
It turns out the queue module name is inconsistent across different python versions. We found that:
* On some macos system:
  - python2 contains both queue and Queue module. All python2 contains Queue module
  - python3 only contains queue module
* On some Linux system
  - python2 contains only Queue module.
  - python3 only contains queue module

Therefore, some developers are seeing `ImportError: No module named queue` errors locally on linux machine after using mobile-install.

**Change**
Import correct Queue module instead

**Test**
Local test pass

Closes #12540.

PiperOrigin-RevId: 369773133
katre pushed a commit to katre/bazel that referenced this pull request Jul 13, 2021
**Background**
Fix issue introduced by bazelbuild@1049fe8
It turns out the queue module name is inconsistent across different python versions. We found that:
* On some macos system:
  - python2 contains both queue and Queue module. All python2 contains Queue module
  - python3 only contains queue module
* On some Linux system
  - python2 contains only Queue module.
  - python3 only contains queue module

Therefore, some developers are seeing `ImportError: No module named queue` errors locally on linux machine after using mobile-install.

**Change**
Import correct Queue module instead

**Test**
Local test pass

Closes bazelbuild#12540.

PiperOrigin-RevId: 369773133
katre pushed a commit to katre/bazel that referenced this pull request Jul 13, 2021
**Background**
Fix issue introduced by bazelbuild@1049fe8
It turns out the queue module name is inconsistent across different python versions. We found that:
* On some macos system:
  - python2 contains both queue and Queue module. All python2 contains Queue module
  - python3 only contains queue module
* On some Linux system
  - python2 contains only Queue module.
  - python3 only contains queue module

Therefore, some developers are seeing `ImportError: No module named queue` errors locally on linux machine after using mobile-install.

**Change**
Import correct Queue module instead

**Test**
Local test pass

Closes bazelbuild#12540.

PiperOrigin-RevId: 369773133
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes team-Android Issues for Android team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants