running yamllint throws unsupported locale error

[ghost]

Just ran into an issue with the 1.24 release of yamllint with the error below

#280 (comment)

Traceback (most recent call last):
   File "/usr/bin/yamllint", line 11, in <module>
     sys.exit(run())
   File "/usr/lib/python2.7/site-packages/yamllint/cli.py", line 179, in run
     locale.setlocale(locale.LC_ALL, conf.locale)
   File "/usr/lib64/python2.7/locale.py", line 547, in setlocale
     return _setlocale(category, locale)
 locale.Error: unsupported locale setting

This happends within a docker container running centos:7 and installing it through pip. Running python version 2.7

[adrienverge]

@GerritJanYourhosting thanks for reporting this.
@wolfgangwalther this is probably due to #280. What do you think?

[madrover]

Hi
I'm getting the same issue, see:

$ uname -a
Darwin miquel-mac.binihome.local 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64
$ python3 --version
Python 3.8.3
$ export | grep LC_
LC_ALL=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
$ pip3 install yamllint==1.23.0
Requirement already satisfied: yamllint==1.23.0 in /usr/local/lib/python3.8/site-packages (1.23.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.8/site-packages (from yamllint==1.23.0) (5.3.1)
Requirement already satisfied: pathspec>=0.5.3 in /usr/local/lib/python3.8/site-packages (from yamllint==1.23.0) (0.8.0)
$ yamllint .
$ pip3 install yamllint==1.24.0
Collecting yamllint==1.24.0
  Using cached yamllint-1.24.0-py2.py3-none-any.whl (59 kB)
Requirement already satisfied: pathspec>=0.5.3 in /usr/local/lib/python3.8/site-packages (from yamllint==1.24.0) (0.8.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.8/site-packages (from yamllint==1.24.0) (5.3.1)
Installing collected packages: yamllint
  Attempting uninstall: yamllint
    Found existing installation: yamllint 1.23.0
    Uninstalling yamllint-1.23.0:
      Successfully uninstalled yamllint-1.23.0
Successfully installed yamllint-1.24.0
$ yamllint .
Traceback (most recent call last):
  File "/usr/local/bin/yamllint", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.8/site-packages/yamllint/cli.py", line 179, in run
    locale.setlocale(locale.LC_ALL, conf.locale)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

I'll revert to 1.23.0 for now...

Thanks!

[adrienverge]

Thanks @GerritJanYourhosting and @madrover for the report. The problematic commit was reverted, you can use 1.24.1 that fixes the crash.

[madrover]

That solved the issue... thanks! :)

[CAM-Gerlach]

@adrienverge I can reproduce this error under yamllint 0.26.0 and 0.26.1, 1.26.0 and 1.26.1 on Windows, while running on identical files and config under yamllint 0.25.0 1.25.0, the error does not occur. Neither export LC_ALL=C.UTF-8 nor export LC_ALL=en_US.UTF-8 makes any difference, though perhaps its because yamllint is running under pre-commit and doesn't pick up the env variables? Should I open a new issue? Is additional testing/diagnostic results helpful? Thanks!

Traceback:

Traceback (most recent call last):
  File "C:\Miniconda3\envs\docs-env\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Miniconda3\envs\docs-env\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\C. A. M. Gerlach\.cache\pre-commit\repo4mn_2vdo\py_env-python3\Scripts\yamllint.EXE\__main__.py", line 7, in <module>
  File "c:\users\c. a. m. gerlach\.cache\pre-commit\repo4mn_2vdo\py_env-python3\lib\site-packages\yamllint\cli.py", line 198, in run
    locale.setlocale(locale.LC_ALL, conf.locale)
  File "C:\Miniconda3\envs\docs-env\lib\locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

[adrienverge]

Versions 0.26.0 and 0.26.1 are very old, you shouldn't use them.

No need to open a new issue, since the error looks the same.

Windows is not really supported (or in "best effort") and I don't have this OS, so I can't help here. If you have time to debug further, it's welcome!

[CAM-Gerlach]

Versions 0.26.0 and 0.26.1 are very old, you shouldn't use them.

I mispoke, sorry, I meant the error can be repro'd on the latest versions, 1.26.0 and 1.26.1, but not on 1.25.0.

Windows is not really supported (or in "best effort") and I don't have this OS, so I can't help here. If you have time to debug further, it's welcome!

My bandwidth at the moment is extremely tight, but once I get time, I can see if I can repro outside of pre-commit, get a dev environment set up and run a git bisect between the 1.26.0 and 1.25.0 tags to figure out what's going on. Nothing has touched that line or anything near it since 1.25.0 and none of the commits look like obvious candidates, so I'm kinda baffled what is triggering it, but I'll see what I can do.

On a broader level, as a popular pure Python package that doesn't do anything particularly low-level, consistent support for Windows should be relatively cheap. Particularly in the context of its use as a pre-commit hook, it not working on one major platform is a blocker to adoption for any platform, as we generally only want to deploy hooks that we're sure developers and contributors can reliably use. However, it is ultimately up to you whether to support it and I'll do my best to pitch in here.

Also, semi-related and you might be already aware, but now that the project is ported to Github Actions with #366 , adding Windows (and macOS) builds to the CI matrix is as easy as adding as little as a single line to the workflow.yml file (I could submit a PR with that); perhaps in combination with dropping at least some intermediate versions, as between a narrow enough range of non-EoL Python versions (e.g. 3.6-3.9), given Python's deprecation policy, a regression that appears only in an intermediate version is very unlikely, certainly much less so than a platform-specific issue. Let me know if that's something you'd like to see. Thanks!

[CAM-Gerlach]

Not sure how I didn't remember this issue from before, but this appears to be the same root problem as #347 , and the solution, so long as it is acceptable, is relatively straightforward—treat yaml files as being UTF-8 by default, per the YAML spec, instead of incorrectly assuming the OS locale. If you agree with the approach proposed there, I can go ahead with a PR. However, I it does now repro on 1.25.0, unsure exactly how/why that is the case, but the proposed change should fix it regardless by avoid the platform and environment-dependent behavior.