Make mesa faster

[ghost]

What's the problem this feature will solve?
Sadly, Python belongs to the slowest programming languages, but it's friendly code makes it popular and extremely usable.
Agent-based simulations can involve very large number of agents (e.g. millions), where concurrency control, parallelism and run-time speed of simulations could be crucial.

Describe the solution you'd like
Using:

• PyPy
• Cython (like numpy, abcEconomics: an agent-based stuff in economics)
• numba

could help a lot in simulation speed ups.

[rht]

This is the first time PyPy is mentioned in this repo.
Disclaimer: I am also one of the maintainers of abcEconomics.

It should be easy to test if Mesa runs on PyPy with Travis (see e.g. https://github.com/AB-CE/abce/blob/master/.travis.yml#L24).

[rht]

In abcEconomics, there is an option to run the execution of each agents in a separate CPU (https://github.com/AB-CE/abce/blob/master/abcEconomics/scheduler/multiprocess.py). If there are 20 agents and 4 CPUs, each CPU could execute 5 agents sequentially, but the 4 CPUs are doing it in parallel independently of each other. See any pool.map in the file I linked before. Care is needed to ensure that the objects are able to be pickled so they can run in each CPUs.

[Corvince]

There is also a discussion about mesas performance in #798.

To recap my 2 cents on this issue: Mesa is mostly a framework and by itself it doesn't "do" much that could be improved by your suggestions. It really depends on the models it is used for (and if the actions are computationally expensive numba might be a very good and easy to use recommendation).

Re parallelism: Depending on the model running agents concurrently is either trivial or (almost) impossible. But usually you will run multiple simulations any way and running those in parallel is far easier, either manually or by using BatchrunnerMP (although it doesn't work at the moment, but it is being worked on).

[rht]

It's worth trying Cinder. Currently discussed in https://news.ycombinator.com/item?id=36612835.

[adamamer20]

addressing this issue, I started to develop an extension of mesa which makes use of vectorized operations on pandas dataframes to achieve faster runtimes. you can provide a feedback or features you'd like to be supported/introduced here

[EwoutH]

I'm going to close this issue as completed, now that we have mesa-frames officially under the Mesa umbrella:

• https://github.com/projectmesa/mesa-frames

If there are any specific issues or ideas for Mesa performance, feel free to open a new issue or discussion!