invalid syntax on async keyword #33

numberoverzero · 2016-01-13T04:05:40Z

When I flake8 the file from the command line everything passes, but I get invalid syntax from the plugin.

I'm using pyenv, but the shims seem to be loading correctly below, since it pulls the path from the global python version.

$ pyenv global
3.5.0

Output from editor open to file lint (conftest.py)

DPI scale: 1
startup, version: 3083 linux x64 channel: stable
executable: /opt/sublime_text/sublime_text
working dir: /
packages path: /home/numberoverzero/.config/sublime-text-3/Packages
state path: /home/numberoverzero/.config/sublime-text-3/Local
zip path: /opt/sublime_text/Packages
zip path: /home/numberoverzero/.config/sublime-text-3/Installed Packages
ignored_packages: ["Vintage"]
pre session restore time: 0.10723
startup time: 0.205469
first paint time: 0.255404


<SNIP> reloading plugin spam </SNIP>


reloading plugin SublimeLinter-flake8.linter
SublimeLinter: flake8 linter loaded 
reloading plugin SublimeLinter.commands
reloading plugin SublimeLinter.sublimelinter
plugins loaded
SublimeLinter: debug mode: on 
SublimeLinter: temp directory: /tmp/SublimeLinter3-numberoverzero 
SublimeLinter: find_python(version='3', script=None, module=None) 
SublimeLinter: find_python: requested version = {'minor': None, 'major': 3} 
SublimeLinter: user shell: /bin/bash 
SublimeLinter: computed PATH using /bin/bash:
/home/numberoverzero/.pyenv/shims
/home/numberoverzero/.pyenv/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
/usr/local/games
/usr/local/bin
/home/numberoverzero/bin
~/npm/bin
/home/numberoverzero/.rvm/bin

SublimeLinter: find_posix_python: python3 => /home/numberoverzero/.pyenv/shims/python3 
SublimeLinter: find_python: system python = /home/numberoverzero/.pyenv/shims/python3 
SublimeLinter: find_python: available version = {'minor': 5, 'major': 3} 
SublimeLinter: find_python: <= ('/home/numberoverzero/.pyenv/shims/python3', None, 3, 5) 
SublimeLinter: sys.path for /home/numberoverzero/.pyenv/shims/python3:
/home/numberoverzero/ws/bloop
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/arrow-0.7.0-py3.5.egg
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/jmespath-0.9.0-py3.5.egg
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/python_dateutil-2.4.2-py3.5.egg
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/six-1.10.0-py3.5.egg
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python35.zip
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/plat-linux
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/lib-dynload
/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages

SublimeLinter: flake8 imported <module 'flake8.engine' from '/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/flake8/engine.py'> 
Package Control: Skipping automatic upgrade, last run at 2016-01-12 19:34:04, next run at 2016-01-12 20:34:04 or after
SublimeLinter: flake8 version: 2.5.1 
SublimeLinter: flake8: (>= 2.2.2) satisfied by 2.5.1 
SublimeLinter: flake8 activated: <builtin> 
SublimeLinter: chdir not set or invalid, using /home/numberoverzero/ws/bottom/tests 
SublimeLinter: flake8: conftest.py <builtin> 
SublimeLinter: flake8 options: {'reporter': <class 'SublimeLinter-flake8.linter.Flake8.get_report.<locals>.Report'>, 'jobs': '1', 'max_complexity': -1} 
SublimeLinter: flake8 output:
conftest.py:22:14: E901 SyntaxError: invalid syntax

The text was updated successfully, but these errors were encountered:

numberoverzero · 2016-01-28T09:06:29Z

ping.

Any more information I can provide? I removed all versions of flake8 from my virtualenvs and then installed it only in 3.5.0. I can post pip freeze and the (largely unchanged) ST console output again if it would help.

groteworld · 2016-01-28T16:30:39Z

Is conftest.py:22:14 invalid py2.7 or py3.X?
what does the command which flake8 return?

numberoverzero · 2016-01-28T21:05:51Z

Thanks for taking a look!
Email reply failed for some reason.

conftest.py is only valid in py3.5+ since it uses the async keyword. Code has moved around since the initial report but here's a conftest file that exhibits the same failure.

Here's the smallest sample that should reproduce the error (through STL-flake8; this is still passing from the command line):

async def foo():
    pass

I start subl from ~, here's the output of which flake8 and a few others, from that directory.

$ pwd
/home/numberoverzero

$ which flake8
/home/numberoverzero/.pyenv/shims/flake8

$ pyenv which flake8
/home/numberoverzero/.pyenv/versions/3.5.0/bin/flake8

$ ipython
In [1]: import flake8
In [2]: flake8.__file__
Out[2]: '/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/flake8/__init__.py'
In [3]: flake8.__version__
Out[3]: '2.5.1'

From the original console output, it seems to find the correct flake8:

SublimeLinter: flake8 imported <module 'flake8.engine' from '/home/numberoverzero/.pyenv/versions/3.5.0/lib/python3.5/site-packages/flake8/engine.py'> 
Package Control: Skipping automatic upgrade, last run at 2016-01-12 19:34:04, next run at 2016-01-12 20:34:04 or after
SublimeLinter: flake8 version: 2.5.1 
SublimeLinter: flake8: (>= 2.2.2) satisfied by 2.5.1 
SublimeLinter: flake8 activated: <builtin> 
SublimeLinter: chdir not set or invalid, using /home/numberoverzero/ws/bottom/tests

No idea what the chdir is about, but I'd be surprised if it's relevant.

jacobsvante · 2016-03-29T16:19:33Z

I'm having the exact same problem. Here's my bug report:

Looking in the Sublime console SublimeLinter-flake8 loads the latest version of the underlying flake8 library (2.5.4 at the time of writing). Passing the file to flake8 in the console does not produce any error/warning output and exists with code 0. I checked the code from within the python 3 interpreter as well like this, giving me no error:

/usr/local/bin/python3
Python 3.5.0 (default, Oct 25 2015, 10:30:34)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import flake8.run
>>> flake8.run.check_file('/path/to/app.py')
0
>>> flake8.__file__
'/usr/local/lib/python3.5/site-packages/flake8/__init__.py'

Here's the output that the Sublime console gives. As you can see from the file path below it's the same flake8 package that I used programmatically above.

SublimeLinter: flake8 imported <module 'flake8.engine' from '/usr/local/lib/python3.5/site-packages/flake8/engine.py'> 
SublimeLinter: flake8 version: 2.5.4 
SublimeLinter: flake8: (>= 2.2.2) satisfied by 2.5.4 
SublimeLinter: flake8 activated: <builtin> 
SublimeLinter: flake8: __init__.py <builtin> 
SublimeLinter: flake8 options: {'reporter': <class 'SublimeLinter-flake8.linter.Flake8.get_report.<locals>.Report'>, 'max_complexity': -1, 'jobs': '1'} 
SublimeLinter: json activated: <builtin> 
reloading Packages/User/SublimeLinter.sublime-settings
SublimeLinter: flake8: __init__.py <builtin> 
SublimeLinter: flake8 options: {'reporter': <class 'SublimeLinter-flake8.linter.Flake8.get_report.<locals>.Report'>, 'max_complexity': -1, 'jobs': '1'}

When I save the file I get:

SublimeLinter: flake8: app.py <builtin> 
SublimeLinter: flake8 options: {'reporter': <class 'SublimeLinter-flake8.linter.Flake8.get_report.<locals>.Report'>, 'max_complexity': -1, 'jobs': 'auto', 'ignore': ['E501', 'E711', 'E712']} 
SublimeLinter: flake8 output:
app.py:3:10: E901 SyntaxError: invalid syntax

The file looks like this:

import asyncio

async def hello_world():
    print("Hello World!")
...

Gr1N · 2016-03-30T20:59:20Z

I have the same problem, and to my mind problem in outdated version of pyflakes, but when I install latest version of pyflakes==1.1.0 I got version conflict: pkg_resources.VersionConflict: (pyflakes 1.1.0 (/usr/local/lib/python3.5/site-packages), Requirement.parse('pyflakes<1.1,>=0.8.1')).

Any ideas?

UPD:
I was wrong, I found problem... https://gitlab.com/pycqa/flake8/blob/master/setup.py#L52
So, I think that this PR can be used as temporary solution #28.

UPD2:
And again no, I was fixed issue with flake8+pyflakes==1.1.0 and got error:

complexity.py:11:10: E901 SyntaxError: invalid syntax

For this code:

async def get(redis, user_ip):
    await store(redis, user_ip)

So I'm going to debug deeper...

Gr1N · 2016-04-02T13:28:42Z

So, bad news guys, I found why we got this error...

Problem in Sublime builtin Python version and how pep8 linter works.
For example open your terminal with Python 3.5+ and run this code:

➔ python3
Python 3.5.1 (default, Dec  7 2015, 21:59:10)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> compile('async def a(): pass', '', 'exec', 1024)
<_ast.Module object at 0x10136c518>
>>>

Looks good, very good, but let's try to run this code in Sublime console:

>>> compile('async def a(): pass', '', 'exec', 1024)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "", line 1
    async def a(): pass
            ^
SyntaxError: invalid syntax

Ops....

Why? Because Sublime uses Python 3.3.3...

>>> import sys
>>> sys.version
'3.3.3 (default, Dec 19 2013, 14:22:24) \n[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)]'

To my mind we have only one way to fix it — replace builtin Python, but I don't know is it possible or not. I found this issue, in Sublime tracker, but Python 3.5+ drops support for Windows XP and authors doesn't want upgrade Python (yeah that's really crazy)...

Any ideas?

UPD: I created request to upgrade Python version sublimehq/sublime_text#1163

groteworld · 2016-04-04T06:09:29Z

So after seeing the Core#1163 issue play out...

Option 1:
If this is something we MUST have, is to require that all python code be linted on save, load, or manual. This is because none of the linters, that I've seen, allow you to pass in a block of code. AFAIK they all require a file be passed in, and this is probably too much I/O for background. We were able to circumvent, by loading the linters libraries and running the reporters from there, over strings of code, but as stated in that issue, is probably wrong to start.

Option 2:
Which I have done 0 research on, would be to add the functionality of passing code, instead of files, to the linter executables (flake8, pep8, pyflakes). I have a sneaking suspicion, that it's been tried numerous times before.

Option 3:
We figure a way to have the PythonLinter classes recognize and add asyncio functionality. No research done to this either, but importlib could grab the 3.5 module and bring it into 3.3. I have no idea what that would even do, but I assume it breaks. I'm literally kicking the can to sprout some ideas.

Option 4:
We deal with it and don't get SL linting for async/await projects. This idea, which I don't like, only comes from the idea of Option 1 and how much I don't like it either.

jacobsvante · 2016-04-04T15:20:32Z

Please excuse my ignorance but aren't like all other linters already doing the linting through an external program already? If so, can't we use the same approach for Python code?
Say we specify an external path to python and then call flake8 through it like this:

/path/to/python3.5 -m flake8 /path/to/file.py

groteworld · 2016-04-04T15:47:36Z

So that's option 1. It uses SL's tempfile_suffix variable to save the contents to a temporary file and lints that with the executable. Using this variable though causes the linter to only work on loads and saves, to cut down on the amount of I/O happening on the disk.

To be honest I haven't looked into this more than opened the file and saw the issue in question. I'll try to give this some dev time tonight.

Gr1N · 2016-04-04T16:05:12Z

I vote for the option 1, this solution should work and it is not depends on SL Python version.

jacobsvante · 2016-04-05T06:44:30Z

Alright, understood. Sounds like option 1 is the way to go then.

* Never use builtin python 3.3 for linting Fixes SublimeLinter#33 * Make `show-code` actually work Fixes SublimeLinter#35 * Add option `executable` so you can point explicitly to a flake8 installation in case SL's which isn't smart enough (which it isn't on Windows)

kaste · 2016-05-10T20:18:49Z

Should be fixed by #38.

* Never use builtin python 3.3 for linting Fixes SublimeLinter#33 * Make `show-code` actually work Fixes SublimeLinter#35 * Add option `executable` so you can point explicitly to a flake8 installation in case SL's which isn't smart enough (which it isn't on Windows)

jolth · 2018-02-06T16:39:18Z

Hello, this happened to me with spacevim and only install flake8 into me virtuel enviroment for solve.

kaste · 2018-02-07T15:17:22Z

You should set executable: 'path/to/my/flake8' in your settings. Your project settings are a good location for that.

sangmoon · 2018-04-29T08:30:41Z

I had same issue, and solved it add paths into setting.
OS is osx.
flake_env is just virtualenv which flake8 is installed.

// SublimeLinter Settings - User
{
    "paths": {
        "linux": [],
        "osx": ["~/flake8_env/bin"],
        "windows": []
    },
}

jacobsvante mentioned this issue Mar 29, 2016

Recognize MagicPython as Python with default settings SublimeLinter/SublimeLinter#352

Merged

kaste mentioned this issue May 10, 2016

Remove cruft #38

Merged

groteworld closed this as completed in #38 Jun 12, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

invalid syntax on async keyword #33

invalid syntax on async keyword #33

numberoverzero commented Jan 13, 2016

numberoverzero commented Jan 28, 2016

groteworld commented Jan 28, 2016

numberoverzero commented Jan 28, 2016

jacobsvante commented Mar 29, 2016

Gr1N commented Mar 30, 2016

Gr1N commented Apr 2, 2016

groteworld commented Apr 4, 2016

jacobsvante commented Apr 4, 2016

groteworld commented Apr 4, 2016

Gr1N commented Apr 4, 2016

jacobsvante commented Apr 5, 2016

kaste commented May 10, 2016

jolth commented Feb 6, 2018

kaste commented Feb 7, 2018

sangmoon commented Apr 29, 2018

invalid syntax on async keyword #33

invalid syntax on async keyword #33

Comments

numberoverzero commented Jan 13, 2016

numberoverzero commented Jan 28, 2016

groteworld commented Jan 28, 2016

numberoverzero commented Jan 28, 2016

jacobsvante commented Mar 29, 2016

Gr1N commented Mar 30, 2016

Gr1N commented Apr 2, 2016

groteworld commented Apr 4, 2016

jacobsvante commented Apr 4, 2016

groteworld commented Apr 4, 2016

Gr1N commented Apr 4, 2016

jacobsvante commented Apr 5, 2016

kaste commented May 10, 2016

jolth commented Feb 6, 2018

kaste commented Feb 7, 2018

sangmoon commented Apr 29, 2018