Upload wheel to PyPI

[ghost]

Could you upload wheel format to PyPI?

Helpful resource:

• https://pythonwheels.com/
• https://github.com/pypa/twine#using-twine

[bletham]

Is there a particular reason for preferring a wheel? Since you anyway have to have all of the build tools just to run pystan, we haven't seen a huge advantage to having a wheel. And conda-forge has binaries for fbprophet which cover anyone using Anaconda.

[ghost]

Thank you for this explanation.
What if pystan also provided wheel?

[bletham]

pystan does provide wheels, but you still have to have build tools because Stan models are translated to C++ and compiled on-the-fly.

[ghost]

So there must be a benefit from wheels ;) since pystan requires whole build kit and still provides wheels format.. . ;)
Thanks for explanation again :)

[bletham]

Yeah, we'll have to consider this more.

[chaitan94]

Is this being considered anytime soon?

fbprophet takes quite some time to build, in fact, in one of our application, it almost triples our build times. I am guessing by having a wheel in PyPI, will help improve this by a huge factor. If so, maybe I can try to figure out a way on how to this and make a contribution if that is okay with the maintainers. Any pointers to the same will be appreciated.

[CaselIT]

It would be really helpful for windows users, since installing all the required python build tool-chain is tedious and prone to errors.

In the meantime here's a link to the conda compiled version https://anaconda.org/conda-forge/fbprophet

[jgb]

+1 Could this be considered?
fbprophet/pystan is really a pain, it slow everything down and needs extra dependencies on the OS level.
Please consider building a wheel and publishing it to pypi to simplify everyone's life.

[bra-fsn]

+1 Could this be considered?
fbprophet/pystan is really a pain, it slow everything down and needs extra dependencies on the OS level.
Please consider building a wheel and publishing it to pypi to simplify everyone's life.

Or at least make it ccache friendly. :)

[gpsalt]

+1 Could this be considered?
fbprophet/pystan is really a pain, it slow everything down and needs extra dependencies on the OS level.
Please consider building a wheel and publishing it to pypi to simplify everyone's life.

Or at least make it ccache friendly. :)

For environmental reasons alone, we save a lot of CO2 if we can install a wheel instead of compiling pystan-prophet constantly in our CI/CD pipeline.... :-)

[burk]

+1. Wheels would greatly speed up installation locally and in CI/CD.

[tcuongd]

One method we can consider is using the GitHub Actions CI to build binaries and also publish them to PyPI.

• cibuildwheel documentation: https://cibuildwheel.readthedocs.io/en/stable/setup/#github-actions
• Setting up a manual workflow: https://docs.github.com/en/actions/reference/events-that-trigger-workflows#manual-events

I'm not sure yet how this works with different Python versions, but we could try it out and publish to https://test.pypi.org/ to verify.

Another thing to check is whether this can also handle the cmdstanpy backend as well (and if it doesn't handle cmdstanpy but can handle another backend like numpyro, that might help us decide on a canonical backend).

[CaselIT]

@tcuongd if you plan to use GitHub Action there are a lot of examples you can use as a starting point.
Like this one for sqlalchemy https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_23/.github/workflows/create-wheels.yaml
does wheels for windoes, osx (only x86 at the moment), linux (x86 and arm).

These are what's generated: https://pypi.org/project/SQLAlchemy/1.4.23/#files

[tcuongd]

I've started a branch for this here: #2010

• macOS seems to be working OK, I've pushed a test wheel here: https://test.pypi.org/project/prophet/#description for Python 3.8
• Not sure how to get the Linux distro working (see details in the PR)
• And I'm not sure it's worth the effort to support building on Windows, especially since we have a binary on conda-forge for Windows already.

Would love any help on the PR! Feel free to comment / branch off it.

[CaselIT]

And I'm not sure it's worth the effort to support building on Windows, especially since we have a binary on conda-forge for Windows already.

I would find it useful, since it allows standardizing on using pip instead of having to remember to install something from pip and something fron conda.

[tcuongd]

I've made some progress on macOS and Linux and got CI to build wheels for Python 3.6 - 3.8, see https://test.pypi.org/project/prophet/#files

If anyone wants to test them out (please do!):

• Start a brand new virtual environment
• Install the requirements from this file: https://github.com/facebook/prophet/blob/tcuongd-cmdstanpy-wheel/python/requirements.txt (python -m pip install -r [path to requirements txt] from within the virtual env)
• python -m pip install --index-url https://test.pypi.org/simple/ prophet
• Note that for now you have to install all the dependencies first, otherwise pip will try to find the dependencies on testpypi and they'll be out of date.

You should see Prophet install successfully without needing to compile any C++ code if you're on a supported platform. Then in your code, you should be able to use both Prophet(stan_backend='PYSTAN') and Prophet(stan_backend='CMDSTANPY') out of the box.

Note that if you're on Python 3.8 and want to use MCMC sampling with pystan, you'll want to set import multiprocessing; multiprocessing.set_start_method('fork') at the very beginning of your script. pystan has been reported to hang on 3.8, and this is an existing issue not related to the wheel.

I think we'll start working on polishing the release + publish process for macOS and Linux first, before trying to do Windows. I agree with you though @CaselIT that having a Windows PyPI wheel will be really beneficial, and I'm a bit less afraid of it now after realising that cmdstanpy already has tooling to make the Windows installation smoother. If anyone wants to branch off the existing PR and get it working for Windows that'd be amazing!

[tcuongd]

Got a first cut of the Windows wheels for PyPi, would appreciate some testing 👍 https://test.pypi.org/project/prophet/1.0.1/#files
Please follow the same instructions as above to install the pypi-test version of prophet.

[CaselIT]

Installed all right on python 3.8.8 on windows 10

(I had to use this command to install pip install --extra-index-url https://test.pypi.org/simple/ prophet though, otherwise pip would not find some dependencies in the test pypi)

Thanks for the effort!

[abitrolly]

@tcuongd it is better to upload the wheels to PyPI, because there is an open attack vector if test.pypi.org is used for public distribution when its database is purged as mentioned here https://packaging.python.org/en/latest/guides/using-testpypi/

[abitrolly]

Absence of wheels significantly delays CI builds and wastes resources. I came here after noticing that MindsDB docker files install the whole C++ toolset just for prophet.

https://github.com/mindsdb/mindsdb/blob/d53d8cf2b73195419f6d71d81893725aaab5a8d0/docker/dockerfile_release.template#L12-L15

So it is better to release wheels early. They can always be put down in case of problems.

[tcuongd]

Thanks everyone for your patience and help! With this release: https://github.com/facebook/prophet/releases/tag/v1.1
we now have wheels on PyPI (https://pypi.org/project/prophet/).