IPython (<=7.19) incompatible with jedi 0.18.0

[DaniilAnichin]

Relevant traceback reads as follows:

  File "../venv/lib/python3.8/site-packages/IPython/core/completer.py", line 2029, in _complete
    completions = self._jedi_matches(
  File "../venv/lib/python3.8/site-packages/IPython/core/completer.py", line 1373, in _jedi_matches
    interpreter = jedi.Interpreter(
  File "../venv/lib/python3.8/site-packages/jedi/api/__init__.py", line 725, in __init__
    super().__init__(code, environment=environment,
TypeError: __init__() got an unexpected keyword argument 'column'

[DaniilAnichin]

Seems like I should open same in IPython, as column was marked deprecated

[davidhalter]

I think we should continue the discussion in ipython/ipython#12740.

IMO this is an downstream issue and they should just do a new release.

[buriy]

@davidhalter
Please could you make an urgent fix if it's easier for you than for them! (they didn't do a release in 12 days after this breakage!...)
We have a rare situation when a package is used by hundreds of thousands of people, I believe you have more obligations in this situation than usually a developer has.
No reason to break ipython just because you finally removed a piece of code you deprecated previously.
P.S. Also please use warn.warn() for deprecated functionality so everyone would see the warning months before the removal.

[xiaosiyu0603]

I met the same problem, and iPython crashed at the same place. Downgrade the jedi to 0.17.2 can solve.

[davidhalter]

@buriy There was a release in 2020-04-14 that raised warn(). That's why the thing got fixed in spring in IPython.

I will just wait for a fix in IPython for this. I'm sorry if this is annoying for some people, but this is not my paid work and I don't think that missing autocompletions are an extremely problematic thing (it's also fixable by downgrading in the meantime).

If anything it show a more fundamental problem: Lack of funding for IPython. I mean I myself have a hard time dealing with issues in Jedi and I'm at a ticket counter of ~1700. IPython is at a ticket count of ~12500. This is an insane amount of work. If you really want to help here, start contributing towards IPython and become one of the core developers. Of course you don't have to, but the problem is that it's easy to write stuff like I believe you have more obligations in this situation than usually a developer has, when in fact it's hard to work on such a big project for long time.

To me it's a tradeoff, I like open source and I like working on hard problems. But I don't like the support part. I could now make a release, it would probably take me an hour. But I could also catch up on sleep, play with my little daughter or do something I actually like doing, like reimplementing Jedi in Rust. In the end it comes to the same reason why I don't backport fixes to older releases: It's just not fun. It's actually insanely not fun. It's so not fun that I like my actual job way better :).

Hope you understand.

To anyone else: Just use pip install jedi==0.17.2 for the next few days.

[buriy]

@davidhalter problem is that not autocompletion doesn't work, problem is that ipython crashes while doing that and I lost some work already due to that. That's not cool.
There's also no clear resolution path, so, you see the crash and don't know what to do. First I thought that a library broke that I experimented with. I spend an hour realizing this is not related, and then I spent like 10 minutes finding this downgrade instruction.

If you could help users -- thousands of users! -- to fix this and save hundreds hours of their work -- why are you standing aside and not doing this?

You frightened ipython team, forcing them to schedule a new release, now it's time to fix it back and wait a little bit more until they will finally make releases with this fix in upstream (and ipython 7.16.2 for python 3.6 too, not only master / Ipython 7.19 for python 3.7+!).

P.S. I'm author/maintainer of other packages, though, not so famous: only 2k stars ;)
So I think I can write about obligations.
If I could make a new release of ipython or jedi myself with this thing fixed -- I would do that right now.

[matthewgilbert]

@buriy why don't you use pin your versions of jedi? This was suggested multiple times above as well as in ipython/ipython#12740.

In general, you should use conda and keep track of your environments, that way when you upgrade packages if things break it is much easier to track down. I went through the same process when I upgraded and it took me about 5-10 minutes to track down the issue and another 5 minutes to resolve once I say the jedi==0.17.2 advice.

Shaming a volunteer who provides an incredibly useful resource for free, that's not cool.

[davidhalter]

P.S. I'm author/maintainer of other packages, though, not so famous: only 2k stars ;)
So I think I can write about obligations.

You are the author of a library with an issue counter of 160 a few hundred commits. This is not at all comparable to IPython or Jedi.

[buriy]

@matthewgilbert I use ipython in 50 different environments and didn't know I should be aware of tracking jedi version in each one! Don't victimize users that they're guilty just by using the software. I do pinning with poetry, and I have good habits of avoiding mistakes because of the crazy developers world, but I updated the ipython with other packages a day before the crash, and everything worked fine, should I never update my projects now? Not only ipython was updated that day, but only next day ipython crashed, in a minute after I started an autocomplete, on running next command. How should I realize this is because of the jedi upgrade I did a day before or that it's related to autocomplete at all?
Ah, you say, jedi developer is not to blame, they're cool, because they worked hard to make their part in a way that you can run the final product 100% of the time, and it's ipython who used it wrong. No, this time this specific developer made the bigger project crash by publishing an incompatible update.
I understand how desperate @davidhalter was that ipython doesn't do a compatible update in months, but planting a bomb is never a good solution.

Actually who is to blame is of course ipython authors who don't pin their dependencies.
If they just do "jedi<0.18" in their setup.py, everything would work well.
Python word is crazy, nobody cares about dependencies.
I'll go and make a PR right now for ipython.

@davidhalter

You are the author of a library with an issue counter of 160 a few hundred commits. This is not at all comparable to IPython or Jedi.

Not at all comparable, I agree, but I'm not just a user, and I know that maintaining a product is hard. That's the way they are comparable.
And I believe that the bigger the project is, the more its developers should be serious about the releases and crashes. That's what I say.
Of course, I understand that open-source developers want to develop and don't want to support their products.
You're basically saying that you don't want to maintain your library as much as you do. That's ok. And very typical -- we're all volunteers here.
But this works differently.
Nobody would blame you for not issuing updates for days and months. You can go spend time with your family and play with your kids as much as you want. But if you do something for people and then it fails on the users' side because of your actions it's normal to take responsibility for that.
It was you who broke ipython in some cases with your 0.18 update. And now you say they should go and make a new release to fix it. Why don't you realize this?

P.S.
Someone finally realized the pinning thing:
https://github.com/ipython/ipython/pull/12751/files
But only in a post-mortem.
And still not pinning jedi 0.18 for ipython master, although that might crash unexpectedly next time with jedi 0.19:
https://github.com/ipython/ipython/blob/master/setup.py#L187
They haven't learned anything.
Edit: after one more week passed, they did the pinning finally.

[davidhalter]

You are wasting my time, sorry. I'm not going to reply anymore other than that:

And I believe that the bigger the project is, the more its developers should be serious about the releases and crashes. That's what I say.

I don't have any responsibility. Really. But I still try to do a good job. If I didn't, Jedi wouldn't be where it is. It's actually way way harder than you think.

As I said if you wan to help: Start contributing. From your online profiles I don't see that. I see 78 contributions on your GitHub. This is not what I do: Investing 10 hours a week.

[tim-win]

Please bump to version 1.0 and limit breaking changes to major releases. This affected me and my team.

[tim-win]

Made a separate issue for this- this one is closed closed closed: #1726

[webknjaz]

@davidhalter May I suggest you enable Discussions in this repo? It's a Q&A StackOverflow-like feature that is typically useful for things that aren't bugs and could exist outside the issues tab. Ref: https://docs.github.com/en/discussions/quickstart. An example discussion page looks like this: aio-libs/aiohttp#4979.