v0.4.4

What's Changed

This is a patch release to fix the AutoGluon source install issue due to spacy update.

• Fix AutoMM by adding uv by @prateekdesai04 in #108
• Update release workflow by @suzhoum in #109

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's Changed

• Add the support of testing on multiple datasets after training on one dataset. by @Harry-zzh in #69
• Bench update by @suzhoum in #70
• Updates for evaluation and aggregation based on AG 1.0 by @Innixma in #71
• Add metric to fill missing logic by @Innixma in #72
• Update for eval refactor by @Innixma in #73
• Update multimodal dep by @suzhoum in #75

New Contributors

• @Harry-zzh made their first contribution in #69

Full Changelog: v0.4.2...v0.4.3

v0.4.2

This is a minor update from the previous release v0.4.1.

Main Features

1. Support few shot learning and custom metrics in config generation script.
2. Bug fix to use the correct file paths when installing autogluon.bench automatically from source or PyPI.
3. Some minor bug fixes.

This release contains 6 commits from 2 contributor.

Full Contributor List (ordered by # of commits):
@prateekdesai04 @suzhoum

This release supports Python versions 3.9. and 3.10.

v0.4.1

We're excited to announce the release of AutoGluon-Bench 0.4.1!

For more details, please check out the README.

Main Features

1. Support custom metrics for multimodal module
2. Improvements on evaluation module
3. Improvements on infrastructure

This release contains 32 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@Innixma @prateekdesai04 @suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.4.0

We're excited to announce the release of AutoGluon-Bench 0.4.0!

For more details, please check out the README.

Main Features

1. timeseries module is now supported! Similar as tabular module, timeseries benchmarks are running on the AMLB framework but with better support in cloud infrastructure. Custom datasets and frameworks are supported as well.
2. Hardware metrics including CPUUtilizaiton and GPUUtilization can now be queried from CloudWatch with the added option --save-hardeware-metrics in agbench run. They are pulled and saved to S3 bucket immediately after all jobs have run into completion.
3. Updates in evaluation logic

This release contains 4 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum @Innixma @shreyash2106

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.3.0

We're excited to announce the release of AutoGluon-Bench 0.3.0!

For more details, please check out the README.

Main Features

1. Introduced framework and constraint concepts in multimodal module so that metadata is better logged in the results.
2. Added evaluation support for multimodal module.

This release contains 2 commits from 1 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.2.2

We're excited to announce the release of AutoGluon-Bench 0.2.2!

For more details, please check out the README.

Main Features

1. Support custom dataset with custom data loader for multimodal module
2. Added model failure support to evaluation output_context.py
3. Added median relative error diff to evaluation evaluate_results.py
4. Added detailed hardware monitoring support with 5s intervals
5. Support CPU base image when using non-GPU machines
6. Updated base images to reduce image size
7. Bug fix for running benchmark on CPU machine.

This release contains 11 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@Innixma @suzhoum @yinweisu

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.2.1

We're excited to announce the release of AutoGluon-Bench 0.2.1!

For more details, please check out the README.

Main Features

1. Supports AMLB user_dir by providing amlb_user_dir in tabular configs.
2. Supports custom AMLB branch installation.
3. Supports AMLB task fold parallelization runs in aws mode, i.e. each instance runs only 1 fold of task
4. Pre-installed framework in docker container for time saving in large scale benchmark.
5. Removed Python3.8 support to align with AMLB.

This release contains 2 commits from 1 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.2.0

We're excited to announce the release of AutoGluon-Bench 0.2.0!

For more details, please check out the README.

Main Features

1. Push GPUUtilization to AWS CloudWatch
2. Added support for python3.10
3. Fix broken python3.8 support
4. Support unit test runs under different python environments
5. Migrate from setup.py to pyproject.toml with automatic versioning and clearer structure

This release contains 2 commits from 1 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum

This release supports Python versions 3.8, 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.1.0

We're excited to announce the release of AutoGluon-Bench 0.1.0!

For more details, please check out the README.

Main Features

1. Added evaluation support for tabular module
2. Restructured multimodal results and supported aggregating the results
3. Use validation data in MultiModalPredictor when available
4. Added --wait option in agbench run to block the process for CI integration
5. CLI to generate cloud config file
6. Added --dev-branch in agbench run to support easy development testing
7. Supports querying instance information for any instance type
8. Adds --config-file option to agbench --destroy-stack
9. Bug fixes

This release contains 13 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum @gidler @yinweisu

This release supports Python versions 3.8, and 3.9.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.