bpo-34014: Added support of contextvars for BaseEventLoop.run_in_executor

[hellysmile]

An updated version of #8035

according to latest discussion on https://bugs.python.org/issue34014

If proposed solution looks good I'll start updating related docs changes

https://bugs.python.org/issue34014

[1st1]

[Blocking this to prevent accidental merge.]

As per the bpo discussion: our goal is to provide a new API for working with threadpools and processpools. If we end up having their design ready in a few months we might want to revisit this PR. If not, we'll likely merge this PR as is.

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[hellysmile]

@1st1 thanks for the update!

Is there any PEP/API spec/ideas available?

[hellysmile]

master was just merged

[hellysmile]

I have made the requested changes; please review again.

[bedevere-bot]

Thanks for making the requested changes!

@1st1: please review the changes made to this pull request.

[1st1]

I'm still not sure about this -- we can't make contextvars work with process pools. :( Need to design a new API for 3.9. You can champion that; first we need to start a discussion. We'll need to assess the current situation, what are the alternatives (curio, trio, twisted), and what's the best possible api we can design for asyncio given its constraints.

[hellysmile]

@1st1 seems using something rather than ThreadPoolExecutor will be prohibited in 3.9

https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L811

what about to restrict same way using not ThreadPoolExecutor instances as executor argument in run_in_executor

and silently apply propagation of contextvars

then we can point this changes to Python 3.9

[1st1]

and silently apply propagation of contextvars

Yeah, that's another problem, as we didn't do so in 3.7/3.8.

@asvetlov what do you think?

[asvetlov]

process pool executor is forbidden as default because things like DNS resolving doesn't work with it IIRC.
Passing explicit ProcessPoolExecutor into run_in_executor() is still supported.

Sorry, I don't follow what the proposal is? Please elaborate silently apply propagation of contextvars part

[NargiT]

any news about this ?

[NargiT]

@asvetlov is there an alternative to this ?

[asvetlov]

@asvetlov is there an alternative to this ?

Take a look at asyncio.to_thread()