Upgrade Python version and dependencies #520

PGijsbers · 2023-03-24T16:32:03Z

Updates the core dependencies of the benchmark with the move to Py3.9.

At this point I simply moved everything forward to keep things moving. In the future, it would be good to parametrize some of these things so that it's easy to e.g., build docker containers on older ubuntu versions or have the framework python version be independent from the amlb python version.

The following table reports what has been tested, only looking at whether or not they produce a result file without errors:

Framework	Ubuntu	MacOS (arm)	Windows (Docker)
constantpredictor	✅	✅	✅
randomforest	✅	✅	✅
tunedrandomforest	✅	✅	✅
DecisionTree	✅	✅	✅
AutoGluon	✅	✅ *	✅
AutoWEKA	☑️	x	☑️
GAMA	✅	✅	✅
H2OAutoML	✅	x	✅
MLNet	☑️	x	[2]
MLPlan	✅/❌	x	✅
TPOT	✅	✅	✅
autosklearn	✅	[1]	✅
flaml	✅	✅ *	✅
lightautoml	✅	✅ *	✅
mljar	✅	✅ *	❌
mlr3automl	✅	x	x
oboe	☑️	x	x
ranger	✅	x	✅

Legend:

✅ Works out of the box on this branch (possibly required patches)
☑️ Works partially
❌ Does not work (yet).
- Not yet tested.
x Won't try. For MacOS (arm) I don't want the extra global java/dotnet/r installs locally. For Windows (docker), mlr3automl should work but would need to change framework definition to forward a GITHUB_PAT, oboe shouldn't work since it doesn't work on Ubuntu host.

* Requires cmake install because wheels for lightgbm are not available

Verified the basic framework works with AWS (python runbenchmark.py constantpredictor -m aws with aws.docker: false runs fine).

Failures

It is not my intention to make sure all integrations work before the move, as some integrations were already broken before. That said, here is more information:

mlnet (ubuntu, 17): failure for binary classification
oboe (ubuntu, 38): requires monkey patching, regression not supported.
mlplansklearn (ubuntu, 19): no results, installation failures
mljarsupervised (Windows (docker)): issue when importing AutoML (base_automl.py#60).

Failures:

[1] autosklearn (MacOS (arm)): depends on scikit-learn 0.24.2 which does not have a wheel for the platform. Might work installing the right build tools, won't investigate.
[2] Fails to install on Windows (docker), seemingly due to a wrong ssl version, though openssl version within the docker image is the same as on Github actions (ubuntu) which runs "successfully" (see caveat above).

Also remove scipy pin, since later autosklearn needs higher scipy.

Specifically for mlr3automl integration

PGijsbers · 2023-03-31T10:00:04Z

@LittleLittleCloud The MLNet integration does not work for binary classification (see this run). I have tried to debug this, but unfortunately could not get a local or containerized setup to work (libssl issues). Would you be able to have a look if the integration script needs updating to support binary classification, or if there are other issues that cause the crash? If you also have time to figure out why MLNet does not install correctly in the docker image, that would be even better (perhaps need to change the frameworks/mlnet/setup.sh script).

@chengrunyang The oboe integration currently requires some monkey patching, based on these changes proposed by alanwilter in #496. As far as I can tell, these monkey patches are required because of oboe itself (perhaps most recent version of dependencies behave different than they did at the release of oboe==0.2.0). Can you have a look at his suggestions, and either release a new version with fixes, or suggest how to use oboe in the integration script without monkey patches? This will not be a blocking issue for this PR, but if the integration script will need to continue to rely on monkey patching, then eventually we will remove oboe from the benchmark as it goes against our design principles.

Innixma

LGTM! Had 1 comment

Innixma · 2023-03-31T14:35:21Z

requirements.txt

@@ -1,92 +1,86 @@
 #
-# This file is autogenerated by pip-compile
-# To update, run:
+# This file is autogenerated by pip-compile with Python 3.9


I'd recommend potentially adding fsspec and s3fs to requirements. These are useful packages that enable pd.read_csv to load files from s3. This is very useful to enable logic such as referencing a results.csv file from s3 when using --resume mode (I’ve implemented this already and plan to send a PR, it is very useful when running large-scale runs in AWS mode).

Thanks! I want to avoid adding new features to this PR, but would be open to looking at that later.

LittleLittleCloud · 2023-03-31T16:43:50Z

@PGijsbers Sure, I can take a look this weekend

PGijsbers · 2023-04-06T07:58:48Z

I will hold off on merging this for now as I will be out of office (and should there be an oversight, I won't be able to fix it on short notice). Planned merge is April 24th. Even after a merge, updates to oboe and mlnet to fix the known issues would be very welcome 🙏

LittleLittleCloud · 2023-04-06T19:12:50Z

@PGijsbers I'll put that under my note. My last week was too busy to fix ml.net issue. Please ping me if I forget to fix it by the end of next week

chengrunyang · 2023-04-06T22:34:47Z

@PGijsbers Sorry for my late reply. I've been busy but will take a look at this issue this weekend.

PGijsbers · 2023-04-07T08:01:35Z

Thanks to both of you! :)

chengrunyang · 2023-04-10T02:04:34Z

@PGijsbers I looked into @alanwilter's edit of https://github.com/openml/automlbenchmark/blob/master/frameworks/oboe/exec.py at #496 (comment). May I know the shape of X_test here (as I'm not sure how to run this exec.py)?

I'm asking because I do see in Oboe 0.2.0 that AutoLearner.predict() fails when X_test is a 1-D numpy array that corresponds to a single data point. But in this case, I had to do something like predictions = aml.predict(X_test.reshape(1, -1)) to get the code work, which seems to be the opposite of predictions = aml.predict(X_test.squeeze()).

And, is the change from auto_learner import AutoLearner -> from oboe.auto_learner import AutoLearner related to something I'd better fix in Oboe (like, would you suggest me to use a more elegant import path)?

Btw, the first fix in udellgroup/oboe@master...alanwilter:oboe:master is indeed needed. Thanks @alanwilter for catching the bug :)

PGijsbers · 2023-04-25T08:48:25Z

@chengrunyang

I don't know which dataset Alan was testing with that required him the squeeze. None of our datasets have only one feature or one row, and in the current working monkey patched version the tests run fine without it. I think it's safe to ignore that. As for the import statement, we also don't care which it should be.

Btw, the first fix in udellgroup/oboe@master...alanwilter:oboe:master is indeed needed. Thanks @alanwilter for catching the bug :)

As far as I can tell, if you incorporate this fix in a new release, we can remove the monkey patch from the integration script and everything should be OK.

PGijsbers · 2023-04-25T08:50:10Z

Merging this now. When a new release of oboe is available, we'll patch the integration script in a separate PR. Similarly MLNet updates will come in a separate PR.

alanwilter · 2023-04-25T08:56:56Z

@chengrunyang

I don't know which dataset Alan was testing with that required him the squeeze. None of our datasets have only one feature or one row, and in the current working monkey patched version the tests run fine without it. I think it's safe to ignore that. As for the import statement, we also don't care which it should be.

Btw, the first fix in udellgroup/oboe@master...alanwilter:oboe:master is indeed needed. Thanks @alanwilter for catching the bug :)

As far as I can tell, if you incorporate this fix in a new release, we can remove the monkey patch from the integration script and everything should be OK.

I was working with a big private data set for testing. AFAI remember, my solution there solved my issues and didn't break the autobenchmark tests.

* Add a workflow to tag latest `v*` release as `stable` (#399) Currenty limited to alphabetical ordering which means that any one number in the version can not exceed one digit. * Bump auto-sklearn to 0.14.0 (#400) * Update version to 2.0 * Revert "Update version to 2.0" This reverts commit 9e0791a. * Fix/docker tag (#404) * Add the version tag to the image name if present * Fix casing for MLNet framework definition * Sync stable-v2 and master (#407) * Update version to 2.0.2 * Revert version change * Add support for the OpenML test server (#423) * Add support for the OpenML test server * change domain from openmltestserver to test.openml * update error message * Apply suggestions from code review Co-authored-by: seb. <sebastien@h2o.ai> * fix syntax error due to online merging Co-authored-by: seb. <sebastien@h2o.ai> * Switch from release:created to release:published (#429) * Added support for dataset files stored on s3 (#420) * s3 functionality * Update amlb/datasets/fileutils.py Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> * OOD * add s3n * move boto3 import Co-authored-by: Weisu Yin <weisuyin96@gmail.com> Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> * Respect TMP, TMPDIR, TEMP (#442) * Respect tmpdir * Fixed submodule * feat: retain environment vars for framework venv * minor fix on compatibility (#454) Co-authored-by: Qingyun Wu <qxw5138@psu.edu> * Ignore decoding errors on Windows (#459) By default it can use cp1252 decoding which sometimes raises an error and halts the process. * Fix a typo (#462) will used -> will be used * Merge back stable-v2 to master (#472) * Add `stable` tag workflow, bump auto-sklearn (#401) * Add a workflow to tag latest `v*` release as `stable` (#399) Currenty limited to alphabetical ordering which means that any one number in the version can not exceed one digit. * Bump auto-sklearn to 0.14.0 (#400) * Fix/docker tag (#404) * Add the version tag to the image name if present * Fix casing for MLNet framework definition * Changed latest from master to main * Update version to 2.0.1 * Improv/aws meta (#413) * Add volume meta data to aws meta info * Add constraints for v2 benchmark (#415) * Add constraints for v2 benchmark For ease of reproducibility, we want to include our experimental setup in the constraints file. For our experiments we increase the volume size to 100gb and require gp3 volumes (general purpose SSD). * Update version to 2.0.2 * Fix AWS random cancel issue (#422) * let the job runner handle the rescheduling logic to ensure that the job is always can't be acted upon by current worker after being rescheduled * remove commented code * Add a GAMA configuration intended for benchmarking (#426) Made the previous version abstract to avoid accidentally running the wrong version of GAMA for the benchmark. * Unsparsify target variables for (Tuned)RF (#425) * Unsparsify target variables for (Tuned)RF Sparse targets are not supported in scikit-learn 0.24.2, and are used with tasks 360932 and 360933 (QSAR) in the benchmark. * cosmetic change to make de/serialization easier to debug Co-authored-by: Sebastien Poirier <sebastien@h2o.ai> * ensure that openml is configured when loading the tasks (#427) * Expect a possible `NoSuchProcess` error (#428) Since it's entirely possible that the processes were already terminating, but only completed termination between the process.children call and the proc.terminate/kill calls. * Reset version for versioning workflow * Update version to 2.0.3 * ensure that the docker images can be built from linux (#437) * Avoid querying terminated instance with CloudWatch (#438) * fixes #432 add precision to runtimes in results.csv (#433) * fixes #432 add precision to runtimes in results.csv * Update amlb/results.py Co-authored-by: seb. <sebastien@h2o.ai> Co-authored-by: seb. <sebastien@h2o.ai> * Iteratively build the forest to honor constraints (#439) * Iteratively build the forest to honor constraints In particular depending on the dataset size either memory or time constraints can become a problem which makes it unreliable as a baseline. Gradually growing the forest sidesteps both issues. * Make iterative fit default, parameterize execution * Step_size as script parameter, safer check if done When final_forest_size is not an exact multiple of step_size, randomforest should still terminate. Additionally step_size is escaped with an underscore as it is not a RandomForestEstimator hyperparameter. * Iterative fit for TunedRandomForest to meet memory and time constraints (#441) * Iterative fit to meet memory and time constraints Specifically for each value of `max_features` to try, an equal time budget is alloted, with one additional budget being reserved for the final fit. This does mean that different `max_features` can lead to different number of trees, but it keeps it simple. * Abort tuning when close to total time budget The first fit of each iterative fit for a `max_features` value was not guarded, which can lead to exceeding the total time budget. This adds a check before the first fit to estimate whether the budget will be exceeded, and if so aborting further tuning and continue with the final fit. * Make k_folds configurable * Add scikit-learn code with explanation * Modify cross_validate, allow 1 estimator per split This is useful when we maintain a warm_started model for each individual split. * Use custom cv function to allow warm-start By default estimators are cloned in any scikit-learn cross_validate function (which stops warm-start) and it is not possible to specify a specific estimator-object per fold (which stops warm-start). The added custom_validate module makes changes to the scikit-learn code to allow warm-starting to work in conjunction with the cross-validate functionality. For more info see scikit-learn#22044 and scikit-learn#22087. * Add parameter to set tune time, rest is for fit The previous iteration where the final fit was treated as an equivalent budget to any other optimization sometimes left too little time to train the final forest, in particular when the last fit took longer than expected. This would often lead to very small forests for the final model. The new system guarantees roughly 10% of budget for the final forest, guaranteeing a better final fit. * Revert version to _dev_version to prepare release (#444) * Update version to 2.0.4 * Signal to encode predictions as proba now works (#447) In a previous iteration it was encoded as a numpy file, but now it's serialized to JSON which means that results.probabilities is simply a string if imputation is required. * Monkeypatch openml to keep whitespace in features (#446) Technically monkeypatch xmltodict function used by openml when reading the features xml * fixe for mlr3automl (#443) * Reset version for Github workflow (#448) * Update version to 2.0.5 * Update mlr3automl to latest Was supposed to be included with #443 * Update MLR3 (#461) * Reset version for version bump * Updatet version because GA failed * Issue 416: fixing versioning workflow for releases and merges to master (#468) * change workflow to correctly modify the app version on releases and when forcing merged version back to master * protect main branch from accidental releases * fix stress test Co-authored-by: PGijsbers <p.gijsbers@tue.nl> Co-authored-by: eddiebergman <eddiebergmanhs@gmail.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Erin LeDell <erin@h2o.ai> Co-authored-by: Stefan Coors <stefan.coors@gmx.net> * useless workflow reintroduced during merge (#475) * tag all AWS entities (#469) * fixed parsing of int targets when loading file in CSV format (#467) * Avoid root owned files from docker (#464) * New site (#479) * First draft of new website * Add framework descriptions, papers and logos * Update footer with Github link * Remove under construction banner * Add redirect from old page to new one * Update page title * Add text links to new paper to be added later * Move static site to /docs * Whitelist documentation images * Remove temporary work directory * Add documentation images * Place holder for mobile * Move old notebooks and visualizations To make sure they are not confusing for new users, as these will no longer work out-of-the-box. New notebooks will be added soon but I don't have the files available right now. * Tell github this is not Jekyll * Update minimal responsiveness (#480) * Make results responsive (hacky) * Make Frameworks page more responsive * Make Home more responsive * Bare minimum mobile navbar * Make sure phones report fake width * Link to arxiv paper (#481) * Update to support AutoGluon v0.4 (#455) * Update to support AutoGluon v0.4 * Address comments * Updated setup.py for `hyperoptsklearn` as it no longer uses PyPi (also now accepts shas) (#410) * Updated hyper opt not to use PyPi and accept shas * case-sensitive PIP command in setup Co-authored-by: Sebastien Poirier <sebastien@h2o.ai> * AutoGluon TimeSeries Support (first version) (#494) * Add AutoGluon TimeSeries Prototype * AutoMLBenchmark TimeSeries Prototype. (#6) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * Update readme * Autogluon timeseries, addressed comments by sebhrusen (#7) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * no outer context manipulation, add dataset subdir * add more datasets * include error raising for too large pred. length. * mergin AutoGluonTS framework folder into AutoGluon * renaming ts.yaml to timeseries.yaml, plus ext. * removing presets, correct latest config for AGTS * move dataset timeseries ext to datasets/file.py * dont bypass test mode * move quantiles and y_past_period_error to opt_cols * remove whitespaces * deleting merge artifacts * delete merge artifacts * renaming prediction_length to forecast_range_in_steps * use public dataset, reduced range to maximum * fix format string works * fix key error bug, remove magic time limit * Addressed minor comments, and fixed version call for tabular and timeseries modularities (#8) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * no outer context manipulation, add dataset subdir * add more datasets * include error raising for too large pred. length. * mergin AutoGluonTS framework folder into AutoGluon * renaming ts.yaml to timeseries.yaml, plus ext. * removing presets, correct latest config for AGTS * move dataset timeseries ext to datasets/file.py * dont bypass test mode * move quantiles and y_past_period_error to opt_cols * remove whitespaces * deleting merge artifacts * delete merge artifacts * renaming prediction_length to forecast_range_in_steps * use public dataset, reduced range to maximum * fix format string works * fix key error bug, remove magic time limit * swapped timeseries and tabular to set version * make warning message more explicit * remove outer context manipulation * split timeseries / tabular into functions Co-authored-by: Leo <LeonhardSommer96@gmail.com> * Add workflow to manually run `runbenchmark.py` on Github Actions (#516) * Add workflow for manually running a test benchmark * Use built-in context for getting the branch * Add more info to step names * Add ability to specify options * Fixed user and sudo under docker (#495) * Fixed user and sudo under docker * Reverted format * Update docker.py * Addressing #497 #497 * Keep wget quiet * Use :, . is deprecated Co-authored-by: seb. <sebastien@h2o.ai> * Set username and userid in Dockerfile generation * Install HDF5 to Docker for tables * Avoid using unix-specific workarounds on Windows * Re-enable caching for building docker images --------- Co-authored-by: seb. <sebastien@h2o.ai> Co-authored-by: PGijsbers <p.gijsbers@tue.nl> * [no-ci] Fix broken link (#514) * Remove autoxgboost, add `removed` field for frameworks (#519) * Add redirect for dataset page (#521) * Upgrade Python version and dependencies (#520) * Remove usage of np.float alias and just use float * Bump to Py3.9 * Update requirements for March 2023, Py3.9 * Pin packaging, since LegacyVersion was removed. Also remove scipy pin, since later autosklearn needs higher scipy. * Install packages to ranger/lib * Set secret PAT used when installing with R remotes Specifically for mlr3automl integration * Update usage for oct 21 release * Disable custom installed packages * Remove installation of reqiurements altogether * Insert oboe example * Add monkeypatch * Make error matrix numpy array * Upgrade to Ubuntu 22.04 from 18.04 * Update pip cache to look at 3.9 directory * Add Github PAT to run_all_frameworks script * bump github action versions * Adding tarfile member sanitization to extractall() (#508) * Included lightautoml in frameworks_stable (#412) * Included lightautoml in frameworks_stable * Added MLNet to frameworks_latest * Added mlr3 to both stable and latest * copy/paste fix * Remove travis file (#529) * Remove travis file since it is not used * Update readme to reflect Python 3.9 support * Add github action workflow to replace old travis file * Add job id, improve name * Fix bug where task inference would lead to KeyError * Update type data for new openml/pandas Probably ought to remove the specific check if we don't enforce it. * Write numeric categories as str, see renatopp/liac-arff/issues/126 * [Open for review] Store results after each job completion (#526) * ensure that results are solved progressively in all situations instead of only when all jobs are completed * rename config flag * don't forget to cleanup job runner exec thread * Improve type hints * Adding file lock on global results file (#453) * adding file lock on global results file * fix imports * fix amlb.utils export * cosmetic * clranup util imports (also magic strings) + remove ruamel dependency in subprocesses --------- Co-authored-by: Sebastien Poirier <sebastien@h2o.ai> * Update the requirements files to exclude yaml and include filelock The remainder of dependencies are not re-generated to avoid additional changes in the PR. * Add missing import * Add fallback for when job is not started * Return an empty dataframe if dataframe is empty This avoids a bug where an empty dataframe is indexed. * Inform the user result summary is not available in AWS mode As results are processed in a different manner (files are directly copied over from S3). This avoids a bug where a benchmark results.csv file tries to be accessed. * Separate scoreboard generation to two lines instead Which makes it easier to tell which part of the generation generates an error, if any. * re-enable logging * Provide a warning and return early if no process output is detected This avoids potentially crashing if the logging is configured incorrectly. In the future, we should expand this to first check how logging is configured in order to see whether or not the issue should be reported and possibly give a more detailed warning if it is likely the cause of an error. --------- Co-authored-by: Sebastien Poirier <sebastien@h2o.ai> Co-authored-by: seb <sebastien.poirier@h2o.ai> * maint: upgrade AMI to Ubuntu 22.04 #512 (#525) * Add `flaml_benchmark` (#528) * dont discard setup_args if it already is a list * Add flaml and flaml_benchmark It is not added to latest since install from latest seems to be broken * Set up alternative way for benchmark mode of flaml This is only temporarily allowed - we expect an easily configurable algorithm, instead of having to carefully install specific dependencies. * limit install, since >2 incompatible * Measure inference time (#532) Add the option to measure inference time (disabled by default) for most frameworks. For those frameworks, inference time is measured capturing both the data loading and the inference. This is done to make things more equal between the different frameworks (as some _need_ to read the file if they don't operator in Python). Inference time is measured multiple times for different batch sizes (configurable). By default, the median is reported in the results file (as it is less sensitive to e.g., cold-starts) but all measured inference times are stored in the predictions folder of a run. For Python frameworks, inference time for in-memory single row predictions is also measured. * Upload to OpenML (#523) Adds a script that allows uploading run results to openml. Additional metadata is stored in the task information to be able to provide a complete description for openml upload. Additional parameters are added to `run_benchmark` to allow runs to automatically be tagged, and to connect to the test server. Also fixes TPOT integration for newer versions, where if a model has no `predict_proba` an `AttributeError` is raised instead of a `RuntimeError`. * Fix a race condition of checking vs adding results (#535) Specifically, adding results was queued in a job executor, while checking results was directly called by the worker threads. If the worker thread checks before the executor had added results, it is possible to get into a deadlock condition. The deadlock arises from the fact that the `stop` condition is never called and the main thread will continue to wait for its END_Q signal. * Add scikit_safe inference time measurement files (#537) * Add scikit_safe inference time measurement files These files have categorical values numerically encoded and missing values imputed, which makes them usable for any scikit-learn algo. * Only generate inference measurement files if enabled * Optionally limit inference time measurements by dataset size (#538) * Add versions 2023 q2 (#539) * Fix versions for June 2023 benchmark * Add 2023Q2 framework tag * Use encoded values for inference * Add us-east-2 AMI * Run docker as root on AWS * Add option to add build options for docker build command * Remove 'infer_speed' artifact as it is not supported in main repo * Fix pandas 2 not compatible with autosklearn 2 see askl#1672 --------- Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de> Co-authored-by: seb. <sebastien@h2o.ai> Co-authored-by: Weisu Yin <weisy@amazon.com> Co-authored-by: Weisu Yin <weisuyin96@gmail.com> Co-authored-by: Eddie Bergman <eddiebergmanhs@gmail.com> Co-authored-by: Qingyun Wu <qw2ky@virginia.edu> Co-authored-by: Qingyun Wu <qxw5138@psu.edu> Co-authored-by: Robinnibor <robinksskss@gmail.com> Co-authored-by: Erin LeDell <erin@h2o.ai> Co-authored-by: Stefan Coors <stefan.coors@gmx.net> Co-authored-by: Alan Silva <3899850+alanwilter@users.noreply.github.com> Co-authored-by: Nick Erickson <neerick@amazon.com> Co-authored-by: Leo <LeonhardSommer96@gmail.com> Co-authored-by: TrellixVulnTeam <112716341+TrellixVulnTeam@users.noreply.github.com> Co-authored-by: seb <sebastien.poirier@h2o.ai>

PGijsbers added 4 commits March 23, 2023 22:07

Remove usage of np.float alias and just use float

316d75c

Bump to Py3.9

bde5518

Update requirements for March 2023, Py3.9

66f5514

Pin packaging, since LegacyVersion was removed.

468cda4

Also remove scipy pin, since later autosklearn needs higher scipy.

PGijsbers added framework For issues with frameworks in the current benchmark WIP Work In Progress: PR is not ready for merge dependencies Pull requests that update a dependency file labels Mar 24, 2023

PGijsbers changed the title ~~Upgrade core~~ Upgrade Python version and dependencies Mar 24, 2023

PGijsbers added 8 commits March 27, 2023 14:28

Install packages to ranger/lib

199bfe4

Set secret PAT used when installing with R remotes

5a9b056

Specifically for mlr3automl integration

Update usage for oct 21 release

ab39dc2

Disable custom installed packages

8a4cecd

Remove installation of reqiurements altogether

010e567

Insert oboe example

9e05fe7

Add monkeypatch

6680a3c

Make error matrix numpy array

c6c4797

PGijsbers marked this pull request as ready for review March 31, 2023 09:49

PGijsbers requested a review from sebhrusen March 31, 2023 09:49

PGijsbers added 3 commits March 31, 2023 12:02

Upgrade to Ubuntu 22.04 from 18.04

f8fed56

Update pip cache to look at 3.9 directory

a2e3219

Add Github PAT to run_all_frameworks script

635b560

Innixma approved these changes Mar 31, 2023

View reviewed changes

Innixma mentioned this pull request Mar 31, 2023

Switch to using Python 3.8 instead of Python 3.7 #511

Closed

bump github action versions

355ac4c

PGijsbers merged commit 7fc5314 into master Apr 25, 2023

PGijsbers deleted the upgrade_core branch April 25, 2023 09:58

PGijsbers mentioned this pull request Apr 25, 2023

Update mlnet #522

Open

PGijsbers mentioned this pull request May 30, 2023

[Open for review] Store results after each job completion #526

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Python version and dependencies #520

Upgrade Python version and dependencies #520

PGijsbers commented Mar 24, 2023 •

edited

Loading

PGijsbers commented Mar 31, 2023 •

edited

Loading

Innixma left a comment

Innixma Mar 31, 2023

PGijsbers Mar 31, 2023

LittleLittleCloud commented Mar 31, 2023

PGijsbers commented Apr 6, 2023

LittleLittleCloud commented Apr 6, 2023

chengrunyang commented Apr 6, 2023

PGijsbers commented Apr 7, 2023

chengrunyang commented Apr 10, 2023

PGijsbers commented Apr 25, 2023

PGijsbers commented Apr 25, 2023

alanwilter commented Apr 25, 2023

Upgrade Python version and dependencies #520

Upgrade Python version and dependencies #520

Conversation

PGijsbers commented Mar 24, 2023 • edited Loading

Failures

PGijsbers commented Mar 31, 2023 • edited Loading

Innixma left a comment

Choose a reason for hiding this comment

Innixma Mar 31, 2023

Choose a reason for hiding this comment

PGijsbers Mar 31, 2023

Choose a reason for hiding this comment

LittleLittleCloud commented Mar 31, 2023

PGijsbers commented Apr 6, 2023

LittleLittleCloud commented Apr 6, 2023

chengrunyang commented Apr 6, 2023

PGijsbers commented Apr 7, 2023

chengrunyang commented Apr 10, 2023

PGijsbers commented Apr 25, 2023

PGijsbers commented Apr 25, 2023

alanwilter commented Apr 25, 2023

PGijsbers commented Mar 24, 2023 •

edited

Loading

PGijsbers commented Mar 31, 2023 •

edited

Loading