Encoding issues on Windows build in the fix_staged_scripts step

[tarcisiofischer]

Hello.

I'm trying to build the waf package on windows, but I'm having some trouble with conda_build in the fix_staged_scripts step. I couldn't find any similar issue. I'll follow with some details:

While building, there is a file named waf that is meant to be moved to the %SCRIPTS% folder. The problem seems to be that conda-build tries to read the file to do some checking, but fails to do so because of the file encoding (https://github.com/conda/conda-build/blob/master/conda_build/windows.py#L45)

I believe the waf file should be opened with encoding='iso-8859-1'. Making a local change replacing with the code bellow builds the package successfully, but is not a solution:

with open(join(scripts_dir, fn), encoding='iso-8859-1') as f:

Any suggestions here? I'm unsure how this case should be handled.
I'll leave some suggestions that occurred me:

1- Fix this at package level, by avoiding running into the fix_staged_scripts function, somehow. (I don't like this solution because it'll not really fix anything besides this specific package)

2- Do some try...catch with several encodings near the presented code above

3- Have a metadata somewhere (maybe in the recipe?) to handle the file encoding. Fallback to the current case if none.

[msarahan]

Thanks for isolating this. I have had a very hard time reproducing it, but your case here should work.

[nicoddemus]

Using the latest cond-build I still get an error while trying to build waf using @tarcisiofischer's meta.yaml:

...
   adding waflib\extras\boost.py as waflib/extras/boost.py
   adding waflib\extras\cython.py as waflib/extras/cython.py
   adding waflib\extras\swig.py as waflib/extras/swig.py
   adding waflib\extras\use_config.py as waflib/extras/use_config.py
   adding waflib\extras\__init__.py as waflib/extras/__init__.py
Waf: Leaving directory `D:\Programming\Miniconda3\conda-bld\waf-conda_1484775768299\work\waf-waf-1.8.21\build'
'build' finished successfully (1.023s)

(D:\Programming\Miniconda3\conda-bld\waf-conda_1484775768299\_b_env) %@EXECSTR[python X:\dotfiles\\get_prompt.py]copy waf "D:\Programming\Miniconda3\conda-bld\waf-conda_1484775768299\_b_env\Scripts"
        1 arquivo(s) copiado(s).
Traceback (most recent call last):
  File "D:\Programming\Miniconda3\Scripts\conda-build-script.py", line 5, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 321, in main
    execute(sys.argv[1:])
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 312, in execute
    noverify=args.no_verify)
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\api.py", line 89, in build
    need_source_download=need_source_download, config=config)
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\build.py", line 1460, in build_tree
    config=config)
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\build.py", line 1010, in build
    windows.build(m, build_file, config=config)
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\windows.py", line 233, in build
    fix_staged_scripts(join(config.build_prefix, 'Scripts'))
  File "D:\Programming\Miniconda3\lib\site-packages\conda_build\windows.py", line 47, in fix_staged_scripts
    line = bs4.UnicodeDammit(f.readline()).unicode_markup.lower()
  File "D:\Programming\Miniconda3\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 4240: character maps to <undefined>

I'm preparing a PR which definitely (I hope) fixes the issue. 😁

[msarahan]

Dang. I really hope you have a better handle on it than I do. I'm pretty lost on these unicode issues. Thanks for working on it.

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically locked because it has not had recent activity after being closed.

Please open a new issue if needed.

Thanks!