Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make prophet, lightgbm and catboost optional #1589

Merged
merged 14 commits into from
Aug 3, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 33 additions & 29 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,20 @@ Below, we detail how to install Darts using either `conda` or `pip`.
Currently only the x86_64 architecture with Python 3.8-3.10
is fully supported with conda; consider using PyPI if you are running into troubles.

To create a conda environment for Python 3.9
Create a conda environment (e.g., for Python 3.10):
(after installing [conda](https://docs.conda.io/en/latest/miniconda.html)):

conda create --name <env-name> python=3.9
conda create --name <env-name> python=3.10

Don't forget to activate your virtual environment
Activate your virtual environment

conda activate <env-name>

As some models have relatively heavy dependencies, we provide two conda-forge packages:

* Install darts with all available models (recommended): `conda install -c conda-forge -c pytorch u8darts-all`.
* Install core + neural networks (PyTorch): `conda install -c conda-forge -c pytorch u8darts-torch`
* Install core only (without neural networks or AutoARIMA): `conda install -c conda-forge u8darts`

For GPU support, please follow the instructions to install CUDA in the [PyTorch installation guide](https://pytorch.org/get-started/locally/).

* Install core only (without neural networks): `conda install -c conda-forge u8darts`

## From PyPI
Install darts with all available models: `pip install darts`.
Expand All @@ -37,31 +34,38 @@ we also maintain the `u8darts` package, which provides the following alternate l
* Install core + neural networks (PyTorch): `pip install "u8darts[torch]"`
* Install core + AutoARIMA: `pip install "u8darts[pmdarima]"`

### Enabling Support for LightGBM
## Other Information

To enable support for LightGBM in Darts, please follow the
[installation instructions](https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html) for your OS.
### Issues with LightGBM
If you run into issues with LightGBM when installing Darts, please follow the
[installation instructions](https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html) for your OS,
and then try re-installing Darts.

#### MacOS Issues with LightGBM
At the time of writing, there is an issue with ``libomp`` 12.0.1 that results in
[segmentation fault on Mac OS Big Sur](https://github.com/microsoft/LightGBM/issues/4229).
Here's the procedure to downgrade the ``libomp`` library (from the
[original Github issue](https://github.com/microsoft/LightGBM/issues/4229#issue-867528353)):
* [Install brew](https://brew.sh/) if you don't already have it.
* Install `wget` if you don't already have it : `brew install wget`.
* Run the commands below:
For instance, on MacOS you may have to run this (using [brew](https://brew.sh/)):
```
wget https://raw.githubusercontent.com/Homebrew/homebrew-core/fb8323f2b170bd4ae97e1bac9bf3e2983af3fdb0/Formula/libomp.rb
brew unlink libomp
brew install libomp.rb
brew install cmake
brew install libomp
```
and then again: `pip install darts`.

### Enabling support for Prophet
By default, as of 0.24.0, `darts` does not have Prophet as a dependency anymore, because its build
process was too often causing issues, [notably on Apple silicon](https://github.com/facebook/prophet/issues/2002).

If you want to use Darts' `Prophet` model, you will need to install the `prophet` package (version 1.1 or more recent).
We refer to the [Prophet README install guide](https://github.com/facebook/prophet#installation-in-python---pypi-release)

### Enabling GPU support
Darts relies on PyTorch for the neural network models.
For GPU support, please follow the instructions to install CUDA in the [PyTorch installation guide](https://pytorch.org/get-started/locally/).

#### Test environment Apple M1 processor
### Using an emulated x64 environment on Apple Silicon.
The installation of `darts` has been tested to work on Apple silicon (Python 3.10, OSX Ventura 13.2.1).
It requires following the instructions to make LightGBM work
[here](https://github.com/unit8co/darts/blob/master/INSTALL.md#issues-with-lightgbm).

We currently recommend to run Darts in an x_64 emulated environment on Mac computers with the Silicon M1 processor,
instead of trying to install directly with native arm64 packages, many of the dependent packages still have compatibility
issues. The following is a proposed procedure, if you tested other procedures on similar hardware and they worked,
please let us know about them by opening an issue or by updating this file and opening a PR.
If you still run into some issues with Apple silicon, you can consider using rosetta
to use an emulated x64 environment by following the steps below:

Before you start make sure that you have rosetta2 installed by running:
```
Expand All @@ -85,8 +89,8 @@ Below are the necessary instructions to create and configure the environment:
conda activate env_name
```
- Install darts: `pip install darts`
- With this method of installation, lightgbm might still have issues finding the libomp library.
The following procedure is to guarantee that the correct libomp (11.1.0) library is linked.
- If after this you still run into issues with lightgbm having issues finding the libomp library,
the following procedure guarantees that the correct libomp (11.1.0) library is linked.
- Unlink the existing libomp, from terminal : `brew unlink libomp`
- Setup a homebrew installer that is compatible with x_64 packages (follow this [blog](https://medium.com/mkdir-awesome/how-to-install-x86-64-homebrew-packages-on-apple-m1-macbook-54ba295230f)
post):
Expand All @@ -106,7 +110,7 @@ Below are the necessary instructions to create and configure the environment:
```
- Verify that your lightgbm works by importing lightgbm from your python env. It should not give library loading errors.

## Running the examples only, without installing:
### Running the examples only, without installing:

If the conda setup is causing too many problems, we also provide a Docker image with everything set up for you and ready-to-use Python notebooks with demo examples.
To run the example notebooks without installing our libraries natively on your machine, you can use our Docker image:
Expand Down
6 changes: 3 additions & 3 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ task installPipLatest {
commandLine "pip", "install", "pip-tools"
}
exec {
commandLine "pip-compile", "-o", "requirements-latest.txt"
commandLine "pip-compile", "requirements/core.txt", "requirements/notorch.txt", "requirements/torch.txt", "-o", "requirements-latest.txt"
}
exec {
commandLine "pip", "install", "-r", "requirements-latest.txt"
Expand All @@ -80,7 +80,7 @@ void createPipInstallTask(String flavour) {
}
}

String[] flavours = ["core", "dev", "torch", "release"];
String[] flavours = ["core", "dev", "notorch", "torch", "release"];

for(String flavour : flavours) {
createPipInstallTask(flavour);
Expand All @@ -94,7 +94,7 @@ task pipInstall() {
doFirst {
setupPip
}
dependsOn pip_core, pip_dev, pip_torch, pip_release
dependsOn pip_core, pip_dev, pip_notorch, pip_torch, pip_release
}

task lint_black(type: Exec) {
Expand Down
70 changes: 11 additions & 59 deletions darts/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
from darts.models.forecasting.tbats_model import BATS, TBATS
from darts.models.forecasting.theta import FourTheta, Theta
from darts.models.forecasting.varima import VARIMA
from darts.models.utils import NotImportedModule

try:
from darts.models.forecasting.block_rnn_model import BlockRNNModel
Expand All @@ -50,48 +51,17 @@
try:
from darts.models.forecasting.lgbm import LightGBMModel
except ModuleNotFoundError:
logger.warning(
"Support for LightGBM not available. "
"To enable LightGBM support in Darts, follow the detailed "
"install instructions for LightGBM in the README: "
"https://github.com/unit8co/darts/blob/master/INSTALL.md"
)
# TODO: simpler option would be to write LightGBMModel=None
# Prevent a second ImportError that would interrupt the import
class NotImportedLightGBM:
usable = False

LightGBMModel = NotImportedLightGBM()
LightGBMModel = NotImportedModule(module_name="LightGBM")

try:
from darts.models.forecasting.prophet_model import Prophet
except ImportError:
logger.warning(
"The prophet module could not be imported. "
"To enable support for Prophet model, follow "
"the instruction in the README: "
"https://github.com/unit8co/darts/blob/master/INSTALL.md"
)

class NotImportedProphet:
usable = False

Prophet = NotImportedProphet()
Prophet = NotImportedModule(module_name="Prophet")

try:
from darts.models.forecasting.catboost_model import CatBoostModel
except ModuleNotFoundError:
logger.warning(
"The catboost module could not be imported. "
"To enable support for CatBoostModel model, "
"follow the instruction in the README: "
"https://github.com/unit8co/darts/blob/master/INSTALL.md"
)

class NotImportedCatBoostModel:
usable = False

CatBoostModel = NotImportedCatBoostModel()
CatBoostModel = NotImportedModule(module_name="CatBoost")

try:
from darts.models.forecasting.croston import Croston
Expand All @@ -102,39 +72,21 @@ class NotImportedCatBoostModel:

except ImportError:
logger.warning(
"The statsforecast module could not be imported. "
"The StatsForecast module could not be imported. "
"To enable support for the StatsForecastAutoARIMA, "
"StatsForecastAutoETS and Croston models, please consider "
"installing it."
)

class NotImportedStatsForecastAutoARIMA:
usable = False

StatsForecastAutoARIMA = NotImportedStatsForecastAutoARIMA()

class NotImportedStatsForecastAutoETS:
usable = False

StatsForecastAutoETS = NotImportedStatsForecastAutoETS()

class NotImportedCroston:
usable = False

Croston = NotImportedCroston()
Croston = NotImportedModule(module_name="StatsForecast", warn=False)
StatsForecastAutoARIMA = NotImportedModule(module_name="StatsForecast", warn=False)
StatsForecastAutoCES = NotImportedModule(module_name="StatsForecast", warn=False)
StatsForecastAutoETS = NotImportedModule(module_name="StatsForecast", warn=False)
StatsForecastAutoTheta = NotImportedModule(module_name="StatsForecast", warn=False)

try:
from darts.models.forecasting.xgboost import XGBModel
except ImportError:
logger.warning(
"The xgboost module could not be imported. "
"To enable support for XGBoost model, install the xgboost package."
)

class NotImportedXGBModel:
usable = False

XGBModel = NotImportedXGBModel()
XGBModel = NotImportedModule(module_name="XGBoost")

from darts.models.filtering.gaussian_process_filter import GaussianProcessFilter
from darts.models.filtering.kalman_filter import KalmanFilter
Expand Down
12 changes: 1 addition & 11 deletions darts/models/forecasting/regression_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,6 @@

logger = get_logger(__name__)

try:
from catboost import CatBoostRegressor
except ModuleNotFoundError:
logger.warning(
"The catboost module could not be imported. "
"To enable support for CatBoostRegressor, "
"follow the instruction in the README: "
"https://github.com/unit8co/darts/blob/master/INSTALL.md"
)


class RegressionModel(GlobalForecastingModel):
def __init__(
Expand Down Expand Up @@ -533,7 +523,7 @@ def fit(
self.model = MultiOutputRegressor(
self.model, n_jobs=n_jobs_multioutput_wrapper
)
elif isinstance(self.model, CatBoostRegressor):
elif self.model.__class__.__name__ == "CatBoostRegressor":
if (
self.model.get_params()["loss_function"]
== "RMSEWithUncertainty"
Expand Down
22 changes: 22 additions & 0 deletions darts/models/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from darts.logging import get_logger, raise_log

logger = get_logger(__name__)


class NotImportedModule:
"""Helper class for handling import errors of optional dependencies."""

usable = False

def __init__(self, module_name: str, warn: bool = True):
self.error_message = (
f"The `{module_name}` module could not be imported. "
f"To enable {module_name} support in Darts, follow the detailed "
f"install instructions for LightGBM in the README: "
dennisbader marked this conversation as resolved.
Show resolved Hide resolved
f"https://github.com/unit8co/darts/blob/master/INSTALL.md"
)
if warn:
logger.warning(self.error_message)

def __call__(self, *args, **kwargs):
raise_log(ImportError(self.error_message), logger=logger)
Loading