-
Notifications
You must be signed in to change notification settings - Fork 6
3.6 Alternatives and Prior Art
Eric Snow edited this page May 31, 2023
·
1 revision
- Python Features / Modules
- see "Concurrency-Models"
- threading
- multiprocessing
- asyncio
- concurrent.futures
- ...
- see "Concurrency-Models"
- Python Libraries / Projects
- Dask
- Python Implementations
- Jython
- IronPython
- PyPy (STM)
- skython
- CPython
- Gilectomy
- nogil
- other attempts to remove the GIL
- pyparallel
- Programming Languages
- Go
- Rust
- Elixir
- Erlang
- C#
- Java
- Lua
- ...
- Other
- better concurrency primitives for threaded programming
- add multi-core support to the asyncio event loop
- improve documentation
- fibers (Windows)
- other disciplines (esp. conceptual modeling, workflow analysis, etc.)...
- Knuth...
- Trent Nelson
- https://github.com/pyparallel/pyparallel
- https://bitbucket.org/tpn/pyparallel
- http://www.infoq.com/articles/PyParallel
- https://news.ycombinator.com/item?id=9862399
- https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores
- https://bitbucket.org/tpn/pyparallel/branches/compare/tpn/pyparallel:3.3-px%0Dpython_mirrors/sandbox-trent:3b5fdb5bc597#diff
- https://bitbucket.org/tpn/pyparallel/branches
- https://mail.python.org/pipermail/python-ideas/2015-September/035725.html
- https://www.reddit.com/r/programming/comments/3jhv80/pyparallel_an_experimental_proofofconcept_fork_of/
- https://speakerdeck.com/trent/pyparallel-pycon-2015-language-summit
- http://pyparallel.org/
- https://mail.python.org/pipermail/python-ideas/2015-June/034342.html
- http://en.wikipedia.org/wiki/Software_transactional_memory#Implementations
- https://gcc.gnu.org/wiki/TransactionalMemory
- https://scholar.google.com/scholar?q=software+transactional+memory&hl=en&as_sdt=0&as_vis=1&oi=scholart&sa=X&ei=ETFMVcGPEIWHsAXgioDoBQ&ved=0CBsQgQMwAA
- https://queue.acm.org/detail.cfm?id=1454466
- http://morepypy.blogspot.com/2012/06/stm-with-threads.html
- http://pypy.readthedocs.org/en/latest/stm.html
- https://bitbucket.org/pypy/stmgc/src/default/c7/
- http://www.kamaelia.org/STM.html
...
- Larry Hastings
- many possible solutions
- everyone's favorite whipping boy
- get rid of GIL but only with PYTHON_MAX_CORES set to something higher that 1 (a la Go)...
- Nick: "Threads aren't the problem, shared memory threading without appropriate access control primitives are the problem. Race conditions, especially around object life cycles, are the bane of both correctness and stability."
- send tasks atomically between threads/cores/processes? (apparently Go does something along these lines when a goroutine blocks)
- isolate task resources to just that task
- make task shared-memory read-only (a la pyparallels)
- threads/subinterpreters with COW
...
- multi-processing and asyncio...
> Would better education/documentation be sufficient? (I expect not.)
I think Dave Beazley's PyCon talk provided a compelling demonstration as
to why not - those performance numbers were brutal.
It would actually be nice to build a microbenchmark based on that talk
to quantify the problem we'd be aiming to solve.
--Nick
...
...
- https://www.google.com/search?q=Skython&oq=Skython&aqs=chrome..69i57&sourceid=chrome&es_sm=122&ie=UTF-8
- http://lwn.net/Articles/640177/
- http://stackoverflow.com/questions/796217/what-is-the-difference-between-a-thread-and-a-fiber
- https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661(v=vs.85).aspx
- primitives
- concurrency model/implementation
Elixir (aka Erlang that fits your brain) could be worth looking at on
that front. While I think subinterpreters and a hierarchical locking
model should form the technical foundation, that says very little about
what the high level API should look like.
--Nick
Also:
If I understand the heritage correctly, I believe Erlang is the origin
of the message passing design I learned in that original C++ job.
"Communicating sequential processes" is the backing domain from computer
science theory.
--Nick