Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'utf-8' codec can't decode byte 0xd5 in position 1142: invalid continuation byte #638

Open
Liuwuyang1026 opened this issue Apr 1, 2024 · 24 comments

Comments

@Liuwuyang1026
Copy link

Describe the bug
Setting up PyTorch plugin "bias_act_plugin"... C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for
cl: Command 'cl' returned non-zero exit status 3221225781.
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Failed!
Traceback (most recent call last):
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1666, in _run_ninja_build
subprocess.run(
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 287, in
main() # pylint: disable=no-value-for-parameter
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 282, in main
launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 97, in launch_training
subprocess_fn(rank=0, c=c, temp_dir=temp_dir)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 48, in subprocess_fn
training_loop.training_loop(rank=rank, **c)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\training_loop.py", line 168, in training_loop
img = misc.print_module_summary(G, [z, c])
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\misc.py", line 216, in print_module_summary
outputs = module(*inputs)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 511, in forward
ws = self.mapping(z, c, truncation_psi=truncation_psi, truncation_cutoff=truncation_cutoff, update_emas=update_emas)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 151, in forward
x = getattr(self, f'fc{idx}')(x)
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 100, in forward
x = bias_act.bias_act(x, b, act=self.activation)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\ops\bias_act.py", line 84, in bias_act
if impl == 'cuda' and x.device.type == 'cuda' and _init():
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\ops\bias_act.py", line 41, in _init
_plugin = custom_ops.get_plugin(
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\custom_ops.py", line 136, in get_plugin
torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1080, in load
return _jit_compile(
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1293, in _jit_compile
_write_ninja_file_and_build_library(
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1405, in _write_ninja_file_and_build_library
_run_ninja_build(
File "C:\Users\29125\anaconda3\envs\stylegan3\lib\site-packages\torch\utils\cpp_extension.py", line 1681, in _run_ninja_build
message += f": {error.output.decode()}" # type: ignore[union-attr]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 1142: invalid continuation byte

  • OS: Windows 11
  • PyTorch version :pytorch 1.9.1
  • CUDA toolkit version :CUDA 11.6
  • NVIDIA driver version
  • GPU: RTX 3080
@Liuwuyang1026
Copy link
Author

help,please!help!

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Hi,

I faced the same issue. This is because on windows there is no C++ compiler.

To solve this you need:

  1. Install msvc compiler (this can be done by installing Microsoft Visual Studio and selecting in the installation C and C++ compilers).
  2. Run the Visual Studio 2022 Developer Command Prompt with the following command: "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\VsDevCmd.bat" -arch=x64 -host_arch=x64 (please note that the path may change a bit, so please verify it is the one for you too).
  3. Then you may run StyleGAN from this terminal

Hope it will help.

@Liuwuyang1026
Copy link
Author

Hi,i'm really thank you for your help.But there is some trouble new ,please!

Traceback (most recent call last):
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\utils\cpp_extension.py", line 2096, in _run_ninja_build
subprocess.run(
File "C:\Users\29125\anaconda3\envs\stylegan\lib\subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 287, in
main() # pylint: disable=no-value-for-parameter
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 282, in main
launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 97, in launch_training
subprocess_fn(rank=0, c=c, temp_dir=temp_dir)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\train.py", line 48, in subprocess_fn
training_loop.training_loop(rank=rank, **c)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\training_loop.py", line 168, in training_loop
img = misc.print_module_summary(G, [z, c])
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\misc.py", line 216, in print_module_summary
outputs = module(*inputs)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 511, in forward
ws = self.mapping(z, c, truncation_psi=truncation_psi, truncation_cutoff=truncation_cutoff, update_emas=update_emas)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 151, in forward
x = getattr(self, f'fc{idx}')(x)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\nn\modules\module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\training\networks_stylegan3.py", line 100, in forward
x = bias_act.bias_act(x, b, act=self.activation)
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\ops\bias_act.py", line 84, in bias_act
if impl == 'cuda' and x.device.type == 'cuda' and _init():
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\ops\bias_act.py", line 41, in _init
_plugin = custom_ops.get_plugin(
File "D:\29125\PYcharm\keyanmodule\stylegan3\stylegan3-main\torch_utils\custom_ops.py", line 136, in get_plugin
torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\utils\cpp_extension.py", line 1306, in load
return _jit_compile(
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\utils\cpp_extension.py", line 1710, in _jit_compile
_write_ninja_file_and_build_library(
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\utils\cpp_extension.py", line 1823, in _write_ninja_file_and_build_library
_run_ninja_build(
File "C:\Users\29125\anaconda3\envs\stylegan\lib\site-packages\torch\utils\cpp_extension.py", line 2111, in _run_ninja_build
message += f": {error.output.decode(*SUBPROCESS_DECODE_ARGS)}" # type: ignore[union-attr]
File "C:\Users\29125\anaconda3\envs\stylegan\lib\encodings\oem.py", line 15, in decode
return oem_decode(input, errors, True)
UnicodeDecodeError: 'cp1' codec can't decode bytes in position 0--1: No mapping for the Unicode character exists in the target code page.

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Okay this seems to be the same error or really close one. Let's go step by step:

  1. Did you install Visual Studio with C++ clang-cl compiler ?
  2. If yes, is the Visual Studio 2022 Developer Command Prompt (with command "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\VsDevCmd.bat" -arch=x64 -host_arch=x64) properly launched ? If so you should get a message like this:

** Visual Studio 2022 Developer Command Prompt v17.3.3
** Copyright (c) 2022 Microsoft Corporation


  1. If you have this message, do you clang activated ? To know that run the command cl. If it is activated you should get cl [ option... ] nom de fichier... [ /link linkoption... ] (or something like my os in french ;) ) otherwise if it is not activated you will get 'cl' is not recognized as internal or external command.
  2. If this works too, this is no more a C++ installation problem, but as it is prompted a UnicodeDecodeError. This is a bit more complex since I did not face this kind of error. But this could be since you may you have a UTF-8 cmd (install for windows williamyang1991/VToonify#50 (comment)) ?

@Liuwuyang1026
Copy link
Author

really nice to see your reply,please forgive me for my foolish,I think maybe I have finished step 1 .But step 2 I don't know whether I'm right or not
微信截图_20240402231636

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Yeah thats good. So step 3: should be ok too but run cl. But I think you should read: williamyang1991/VToonify#50. A chinese user had an issue really close problem to the one you got. He changed is CMD to GBK and that fixed the issue. But I don't really know how to do it :/

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

@Liuwuyang1026
Copy link
Author

I'm installing c++ clang tools. I'm a Chinese,so I don't know how to express my total appreciation to you!!!Really thanks!
I don't really understand the second step 2. Do I need to clock in and promote it?
Do you want to run -arch=x64 -host_arch=x64 or ‘’D:\VS2022\VS\Common7\Tools\VsDevCmd.bat‘’ -arch=x64 -host_arch=x64

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

The second step was launching the Visual Dev Command Prompt. The idea is to go fetch all the tools install with Visual Studio, including clang. Clang is the Windows C++ compiler. The ninja fails is likely due to a compilation fails since by default there is no C++ compiler on windows. But the encoding can also be an issue, but I'm a bit further since I never met this kind of issue.

@Liuwuyang1026
Copy link
Author

Hello Neilstid!Lucky to meet you and have your instructions!But I still can't work it.I think I succeed to install clang(maybe,I'm not sure about it).I don't know how to thank you but I really need you to help me manage the trouble!

This is my current error:

Setting up PyTorch plugin "bias_act_plugin"... C:\Users\29125\anaconda3\envs\ganstyle\lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for
cl: Command 'cl' returned non-zero exit status 3221225781.
warnings.warn(f'Error checking compiler version for {compiler}: {error}')

File "C:\Users\29125\anaconda3\envs\ganstyle\lib\site-packages\torch\utils\cpp_extension.py", line 1681, in _run_ninja_build
message += f": {error.output.decode()}" # type: ignore[union-attr]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 1138: invalid continuation byte

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Okay windows did not found path to clang :/. So normally, for now,, if you run where clang it will give you an error, instead of the path to clang (for exemple it returns for me C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\cl.exe).
To let windows find clang, you will need to add to PATH in your system variable the path to clang (should be somewhere here C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\cl.exe), restart your command prompt and check windows now finds clang (still with where clang)

@Liuwuyang1026
Copy link
Author

I don't know whether I finished this requirement:

The code relies heavily on custom PyTorch extensions that are compiled on the fly using NVCC. On Windows, the compilation requires Microsoft Visual Studio. We recommend installing Visual Studio Community Edition and adding it into using .PATH"C:\Program Files (x86)\Microsoft Visual Studio<VERSION>\Community\VC\Auxiliary\Build\vcvars64.bat"

Does it mean add "D:\VS2022\VS\VC\Auxiliary\Build\vcvars64.bat" into environoment variable “Path”?

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Yes you need to add it too to PATH

@Liuwuyang1026
Copy link
Author

I will try right now! Wait for my good news!

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

I really hope it will :D

@Liuwuyang1026
Copy link
Author

Liuwuyang1026 commented Apr 2, 2024

SAD!!!!I have add it into "Path",but when I RUN "where clang",It shows"Message: The file cannot be found using the provided mode."And I run train.py it still the same error
23
asd

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

You also need to start Dev Command Prompt: ‘’D:\VS2022\VS\Common7\Tools\VsDevCmd.bat‘’ -arch=x64 -host_arch=x64

image

@Liuwuyang1026
Copy link
Author

Liuwuyang1026 commented Apr 2, 2024

I follow your step,but train.py still show the same error:Setting up PyTorch plugin "bias_act_plugin"... C:\Users\29125\anaconda3\envs\ganstyle\lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for
cl: Command 'cl' returned non-zero exit status 3221225781.
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
aaa2

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

I'm sorry I've must missing a thing. Running Stylegan from this dev cmd prompt should have worked or at least not having compilation issue since you have clang :/

@Liuwuyang1026
Copy link
Author

Sorry I don't understand what you said;
The error is still the same:(1).Command 'cl' returned non-zero exit status 3221225781.(2)'utf-8' codec can't decode byte 0xd5 in position 1138: invalid continuation byte

@Neilstid
Copy link

Neilstid commented Apr 2, 2024

Yes and thats what I don't understand:
Error 1) Should have been fixed since you now have clang (ps: you need to run stylegan from the Dev Command Prompt so in the terminal before running StyleGAN, running in the same command prompt ‘’D:\VS2022\VS\Common7\Tools\VsDevCmd.bat‘’ -arch=x64 -host_arch=x64 is mandatory to enable clang).
Error 2) It can be fixed by changing the encoding of your terminal (cf above comments)

@Liuwuyang1026
Copy link
Author

Neilstd,sorry and sad to tell you!I follow your step,but a new issue raise!!!
The error: File "", line 565, in module_from_spec
File "", line 1173, in create_module
File "", line 228, in _call_with_frames_removed
ImportError: DLL load failed while importing bias_act_plugin: 找不到指定的模块。
bbb

@Liuwuyang1026
Copy link
Author

Is there any people can help me,ungerntly

@Neilstid
Copy link

Neilstid commented Apr 3, 2024

Is it the same error, or has it at least change ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants