Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Fix collect_env on Windows #1789

Merged
merged 12 commits into from
Apr 5, 2022
Merged

[Enhancement] Fix collect_env on Windows #1789

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
65975d7
[fix] fix collect_env() on Windows
SuTanTank Mar 10, 2022
746fecb
fix(utils): add back 'Cuda compilation tools' to nvcc info
SuTanTank Mar 10, 2022
9d57970
feat(utils): find default ccompiler
SuTanTank Mar 12, 2022
29692f9
fix(utils, test): parse cl.exe version instead of path
SuTanTank Mar 12, 2022
72666bb
fix: code style
SuTanTank Mar 12, 2022
f29bb33
fix(tools): use distutils
SuTanTank Mar 13, 2022
6363b09
[fix] collect_env() returns GCC or MSVC
SuTanTank Mar 15, 2022
3ebdefb
[fix] unitest for collect_env()
SuTanTank Mar 15, 2022
ecd4e72
add comments
SuTanTank Mar 18, 2022
433cbe7
test: add GCC to expected_keys
SuTanTank Mar 21, 2022
f59c2b3
fix: code style
SuTanTank Apr 4, 2022
3fe1bfd
Update mmcv/utils/env.py
SuTanTank Apr 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 28 additions & 6 deletions mmcv/utils/env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def collect_env():
- CUDA_HOME (optional): The env var ``CUDA_HOME``.
- NVCC (optional): NVCC version.
- GCC: GCC version, "n/a" if GCC is not installed.
- MSVC: Microsoft Virtual C++ Compiler version, Windows only.
- PyTorch: PyTorch version.
- PyTorch compiling details: The output of \
``torch.__config__.show()``.
Expand Down Expand Up @@ -56,18 +57,39 @@ def collect_env():
if CUDA_HOME is not None and osp.isdir(CUDA_HOME):
try:
nvcc = osp.join(CUDA_HOME, 'bin/nvcc')
nvcc = subprocess.check_output(
f'"{nvcc}" -V | tail -n1', shell=True)
nvcc = subprocess.check_output(f'"{nvcc}" -V', shell=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be better if we can get the last line of nvcc -V which is consistent with the former version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Linux

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

On Windows (11.5)

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:52:33_Pacific_Standard_Time_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Not sure what it looks like on other Windows machines. but the last line on Linux match with the second last line on Windows.

I will add 'Cuda compilation tools' to the output.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK~ thanks for your explanation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May leave comments in the code to explain.

nvcc = nvcc.decode('utf-8').strip()
release = nvcc.rfind('Cuda compilation tools')
build = nvcc.rfind('Build ')
nvcc = nvcc[release:build].strip()
except subprocess.SubprocessError:
nvcc = 'Not Available'
env_info['NVCC'] = nvcc

try:
gcc = subprocess.check_output('gcc --version | head -n1', shell=True)
gcc = gcc.decode('utf-8').strip()
env_info['GCC'] = gcc
except subprocess.CalledProcessError: # gcc is unavailable
# Check C++ Compiler.
# For Unix-like, sysconfig has 'CC' variable like 'gcc -pthread ...',
# indicating the compiler used, we use this to get the compiler name
import sysconfig
HAOCHENYE marked this conversation as resolved.
Show resolved Hide resolved
cc = sysconfig.get_config_var('CC')
if cc:
cc = osp.basename(cc.split()[0])
cc_info = subprocess.check_output(f'{cc} --version', shell=True)
env_info['GCC'] = cc_info.decode('utf-8').partition(
'\n')[0].strip()
else:
# on Windows, cl.exe is not in PATH. We need to find the path.
# distutils.ccompiler.new_compiler() returns a msvccompiler
# object and after initialization, path to cl.exe is found.
from distutils.ccompiler import new_compiler
ccompiler = new_compiler()
ccompiler.initialize()
cc = subprocess.check_output(
f'{ccompiler.cc}', stderr=subprocess.STDOUT, shell=True)
env_info['MSVC'] = cc.decode('utf-8').partition('\n')[0].strip()
SuTanTank marked this conversation as resolved.
Show resolved Hide resolved
env_info['GCC'] = 'n/a'
del ccompiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to manually delete ccompiler?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not. I did this because that initialize() call took a few seconds, so I thought this object may take some unnecessary memory.

Copy link
Collaborator

@zhouzaida zhouzaida Apr 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be released by gc in later so we can remove this line.

SuTanTank marked this conversation as resolved.
Show resolved Hide resolved
except subprocess.CalledProcessError:
env_info['GCC'] = 'n/a'

env_info['PyTorch'] = torch.__version__
Expand Down
6 changes: 3 additions & 3 deletions tests/test_utils/test_env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def test_collect_env():
env_info = collect_env()
expected_keys = [
'sys.platform', 'Python', 'CUDA available', 'PyTorch',
'PyTorch compiling details', 'OpenCV', 'MMCV', 'MMCV Compiler',
'PyTorch compiling details', 'OpenCV', 'MMCV', 'MMCV Compiler', 'GCC',
'MMCV CUDA Compiler'
]
for key in expected_keys:
Expand All @@ -26,8 +26,8 @@ def test_collect_env():
for key in ['CUDA_HOME', 'NVCC']:
assert key in env_info

if sys.platform != 'win32':
assert 'GCC' in env_info
if sys.platform == 'win32':
assert 'MSVC' in env_info

assert env_info['sys.platform'] == sys.platform
assert env_info['Python'] == sys.version.replace('\n', '')
Expand Down