Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Remove Cython - Reduce tech debt #346

Merged
merged 105 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
ff0f2b2
uncythonize, pytest update
eddiebergman Dec 18, 2023
e915964
tests passing; fix add_conditions
eddiebergman Dec 19, 2023
2a29120
Fixup tests
eddiebergman Dec 19, 2023
79decbd
refactor: UniformFloat
eddiebergman Dec 19, 2023
add84b5
optim: UniformFloat get neighbors
eddiebergman Dec 19, 2023
a3f4297
refactor: deprecate UniformFloat is_legal_vector
eddiebergman Dec 19, 2023
60dc4db
big overhaul
eddiebergman Mar 6, 2024
279cac2
fix: depercation wraning of `np.integer` for dtype
eddiebergman Mar 6, 2024
3bc359f
refactor: Big internal rewrite, passing tests
eddiebergman Mar 12, 2024
b7f5d9d
test passing checkpoint
eddiebergman Mar 15, 2024
fb368c4
pythonize conditionals
eddiebergman Mar 16, 2024
e4064cf
optim: Topological sort and vectorize cond/forbiddens
eddiebergman Mar 25, 2024
5ebd21f
optim: Further small optimizations
eddiebergman Mar 25, 2024
a4916be
optim: Transpose sampling matrix for cache locality
eddiebergman Mar 26, 2024
4ab1c80
optim: Add test script for neighbor sampling
eddiebergman Mar 26, 2024
584b8ef
test: Repass tests
eddiebergman Mar 26, 2024
87eb45d
tmpcommmit - lets pray
eddiebergman Mar 26, 2024
fceffa6
refactor: Factor out DAG structure into dedicated class
eddiebergman Mar 27, 2024
ba1cb59
refactor: Move things out of `c_util`
eddiebergman Mar 27, 2024
26430fd
fix: Minimum spanning
eddiebergman Mar 27, 2024
5fb53f6
fix: min-cond-span sorted by min(depth of all children)
eddiebergman Mar 28, 2024
82e3fab
refactor: All tests and benchmarks pass
eddiebergman Apr 8, 2024
147e6ec
Merge branch 'main' into typing-cython-3
eddiebergman Apr 11, 2024
07687f2
refactor: Change deprecated test_util
eddiebergman Apr 8, 2024
99cd93b
Revert "refactor: Change deprecated test_util"
eddiebergman Apr 11, 2024
0df82ec
doc: Remove mention of cython
eddiebergman Apr 11, 2024
8da216d
build: Remove the use "build" in deps
eddiebergman Apr 11, 2024
55b9190
ci: Update test workflow
eddiebergman Apr 11, 2024
fb3fc42
ci: Remove conda tests since we no longer build w/ cython
eddiebergman Apr 11, 2024
68a0fbb
fix(typing): Use typing extension for future TypeAlias
eddiebergman Apr 11, 2024
a75708e
ci: Use caching for env across tests
eddiebergman Apr 11, 2024
ef96506
fix(typing): Use `Union` instead of 3.10 syntax
eddiebergman Apr 11, 2024
5af481f
ci: Remove git porcelain check
eddiebergman Apr 11, 2024
83cb54c
fix(typing): Use typing for inheritance to support <= 3.9
eddiebergman Apr 11, 2024
0a11215
fix: Remove `strict=True` from `zip` to support <= 3.10
eddiebergman Apr 11, 2024
b699216
fix(typing): Using `typing.Mapping` for inheritance
eddiebergman Apr 11, 2024
940c403
fix: Move benchmarked functions as main functions to use
eddiebergman Apr 11, 2024
f58a387
style: Update ruff to use numpy syntax
eddiebergman Apr 11, 2024
cb335b9
fix(functional): `linspace_chunked` acts more like `np.linspace`
eddiebergman Apr 11, 2024
7e2f97f
doc: Run `ruff --fix`
eddiebergman Apr 11, 2024
fab4b07
style: Fix ruff checks
eddiebergman Apr 11, 2024
f913bc2
refactor: Convert to `src` layout
eddiebergman Apr 11, 2024
cf2dd67
ci: Ignore use of `eval` in test
eddiebergman Apr 11, 2024
0bbd7a9
style: Fix remaning ruff errors
eddiebergman Apr 11, 2024
60791ab
refactor: Json encoding
eddiebergman Apr 15, 2024
2a7cbdd
refactor: Sorry, big dump
eddiebergman Apr 15, 2024
f9b0cf7
optim: Further speedups to one exchange
eddiebergman Apr 15, 2024
631a1e5
optim: Further speedups
eddiebergman Apr 16, 2024
4fb2787
fix: Hyperparameter attributes are simple types
eddiebergman Apr 16, 2024
9fe50d9
fix: Sampling from empty configuration spaces
eddiebergman Apr 16, 2024
79682ea
fix: Revert changes from different branch
eddiebergman Apr 16, 2024
ba65910
fix: Test for default_value type
eddiebergman Apr 16, 2024
574d355
fix: Deserialization with `"r"`
eddiebergman Apr 16, 2024
4ab87f3
fix: Conjunctions match clauses for equality #334
eddiebergman Apr 16, 2024
ac906a4
fix: Import TypeAlias from typing_extensions
eddiebergman Apr 16, 2024
d88c6bd
fix: Remove unreliable __hash__ from ConfigurationSpace
eddiebergman Apr 16, 2024
945df4c
fix: Ruff errors
eddiebergman Apr 19, 2024
693d64d
typing: Soft deprecate `pcs` and `pcs_new`
eddiebergman Apr 19, 2024
451766f
typing: Fix decoder type for json read and write
eddiebergman Apr 19, 2024
4515a8a
ci: Move to ruff for linting
eddiebergman Apr 19, 2024
3b54021
fix: Better condition cloning in replace hyperparameters
eddiebergman Apr 19, 2024
3fe243b
fix: Use 3.8 compatable isinstance checks
eddiebergman Apr 19, 2024
cb74fde
typing: `configuration_space.py`
eddiebergman Apr 19, 2024
470071b
typing: `dictionary.py`
eddiebergman Apr 19, 2024
77522df
typing: `_condition_tree.py`
eddiebergman Apr 19, 2024
222b097
typing: `forbidden.py`
eddiebergman Apr 19, 2024
5f65e5a
ci: Add `scipy.*` to list of mypy ignore missing import
eddiebergman Apr 19, 2024
636f0b6
typing: `conditions.py`
eddiebergman Apr 19, 2024
9c8a0c9
typing: Fix hyperparameter related issues
eddiebergman Apr 19, 2024
560ada3
test: Fixups
eddiebergman Apr 19, 2024
08b8cf9
refactor: Remove `nx` vendored lib
eddiebergman Apr 19, 2024
ad79b5c
test: Use new `add` as old was deprecated
eddiebergman Apr 19, 2024
4cdb5b3
refactor: Remove deprecations
eddiebergman Apr 19, 2024
9d66f56
fix: Use `isinstance(..., (type1, type2))`
eddiebergman Apr 19, 2024
d144026
fix: Better casting for reading from pcs
eddiebergman Apr 19, 2024
a4467f4
debug: Check windows issue of not legal
eddiebergman Apr 24, 2024
f773910
ci: Don't use cache for installation
eddiebergman Apr 24, 2024
215256a
fix: Change error type to IllegalVectorizedValue
eddiebergman Apr 24, 2024
01c68c6
debug: More debug prints for ci
eddiebergman Apr 24, 2024
e94825d
ci: Exclude 3.8 3.9 mac
eddiebergman Apr 24, 2024
05fbe4b
debug: More ci debugging statements
eddiebergman Apr 24, 2024
8a9d51f
fix: UnitScaler promotes to 64bit to prevent overflow on windows
eddiebergman Apr 24, 2024
f6d1343
fix: Remove non assigned type
eddiebergman Apr 26, 2024
dbaab3f
fix: Address minor comments
eddiebergman Apr 30, 2024
f84cbd4
fix: Minor review comments
eddiebergman Apr 30, 2024
5b65f45
doc: Overhaul to mkdocs, partially complete
eddiebergman Jun 20, 2024
669b108
doc: Fix missing comma
eddiebergman Jun 20, 2024
107204b
doc: Typo fixes
eddiebergman Jun 20, 2024
2f495d1
doc: typo fix again...
eddiebergman Jun 20, 2024
8a4ac0b
doc: Major doc overhaul to mkdocs
eddiebergman Jun 27, 2024
369803c
fix: Add backwards compatible `check_forbidden`
eddiebergman Jul 1, 2024
5152f9e
fix: Allow backwards compat `"default"` key in encoded spaces
eddiebergman Jul 1, 2024
cc5fd78
fix: Backwards compatible private methods
eddiebergman Jul 1, 2024
cf011bf
refactor: Use `check_` instead of `is_` as method raises
eddiebergman Jul 1, 2024
bc3afcd
doc: Forbiddens
eddiebergman Jul 1, 2024
2a48d08
test: fix test funcion name
eddiebergman Jul 1, 2024
995c417
fix(Forbidden): Uncomment value checking on `ForbiddenEqualsClaus`
eddiebergman Jul 1, 2024
09beefd
ci: Update doc and test workflows
eddiebergman Jul 1, 2024
debe675
test: Remove stray function
eddiebergman Jul 12, 2024
897bec7
doc: Type fix
eddiebergman Jul 12, 2024
b1f1fb9
doc: Add note to `_unsafe` code in `hp_components`
eddiebergman Jul 12, 2024
80e2c7c
doc: Typo fix
eddiebergman Jul 12, 2024
d23879d
doc: Improve code doc attribute clarity
eddiebergman Jul 12, 2024
de0b36e
doc: Change `note` to `todo`
eddiebergman Jul 12, 2024
82d66fd
doc: Fix documentation for `continuous_neighborhood`
eddiebergman Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 13 additions & 58 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -1,72 +1,27 @@
# This workflow is just to test that the docs build successfully.
name: docs

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
# Trigger manually
workflow_dispatch:

# Trigger on any push to the main
push:
branches:
- main
- development

# Trigger on any push to a PR that targets main
pull_request:
branches:
- main
- development

permissions:
contents: write

env:
name: "ConfigSpace"

jobs:
build-and-deploy:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Setup Python
uses: actions/setup-python@v3
with:
python-version: "3.8"

- name: Install dependencies
run: |
pip install build
pip install ".[docs]"

- name: Make docs
run: |
make clean
make docs

- name: Pull latest gh-pages
if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
run: |
cd ..
git clone https://github.com/${{ github.repository }}.git --branch gh-pages --single-branch gh-pages

- name: Copy new docs into gh-pages
if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
run: |
branch_name=${GITHUB_REF##*/}
cd ../gh-pages
rm -rf $branch_name
cp -r ../${{ env.name }}/docs/build/html $branch_name

- name: Push to gh-pages
if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
run: |
last_commit=$(git log --pretty=format:"%an: %s")
cd ../gh-pages
branch_name=${GITHUB_REF##*/}
git add $branch_name/
git config --global user.name 'Github Actions'
git config --global user.email 'not@mail.com'
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
git commit -am "$last_commit"
git push
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: "Install dependancies"
run: python -m pip install -e ".[dev]"
- name: "Build Docs"
run: mkdocs build --clean --strict
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, is there a reason you removed the pushing of the documentation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, we moved away from Sphinx and I didn't know how to automate doc deployments given this setup. I will document how to do it in manual way in CONTRIBUTING.md

16 changes: 4 additions & 12 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,9 @@ env:
--durations=20
-v


jobs:
test:

# General unit tests
source-test:
name: ${{ matrix.python-version }}-${{ matrix.os }}

runs-on: ${{ matrix.os }}
Expand All @@ -49,12 +47,6 @@ jobs:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
exclude:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this work now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently so

# These don't seem to be available anymore with setup-python?
- python-version: "3.8"
os: "macos-latest"
- python-version: "3.9"
os: "macos-latest"

steps:
- name: Checkout
Expand All @@ -64,13 +56,13 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: pyproject.toml

- name: Install ${{ env.package-name }}
run: |
python -m pip install --upgrade pip
python -m pip install -e ".${{ env.extra-requires }}"

- name: Tests
timeout-minutes: 45
run: |
pytest ${{ env.pytest-args }} ${{ env.test-dir }}
pytest ${{ env.test-dir }}
35 changes: 4 additions & 31 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,13 @@

.PHONY: help install-dev install-test install-docs pre-commit clean clean-doc clean-build build docs links publish test clean-test

A_DIFFERENT_CHANGE

help:
@echo "Makefile ConfigSpace"
@echo "* install-dev to install all dev requirements and install pre-commit"
@echo "* pre-commit to run the pre-commit check"
@echo "* check to run the pre-commit check"
@echo "* docs to generate and view the html files"
@echo "* linkcheck to check the documentation links"
@echo "* publish to help publish the current branch to pypi"
@echo "* test to run the tests"

Expand Down Expand Up @@ -39,48 +40,20 @@ install-dev:
$(PIP) install -e ".[dev]"
pre-commit install

install-test:
$(PIP) install -e ".[test]"

install-docs:
$(PIP) install -e ".[docs]"

check:
$(PRECOMMIT) run --all-files

check-types:
mypy ConfigSpace

fix:
black --quiet ConfigSpace test
ruff --silent --exit-zero --no-cache --fix ConfigSpace test

build:
python -m build

test:
$(PYTEST) test

clean-build:
rm -rf ${BUILD}

clean-docs:
$(MAKE) -C ${DOCDIR} clean

clean: clean-build clean-docs

clean-test: clean-build build test

# Running build before making docs is needed all be it very slow.
# Without doing a full build, the doctests seem to use docstrings from the last compiled build
docs: clean build
$(MAKE) -C ${DOCDIR} html
@echo
@echo "View docs at:"
@echo ${INDEX_HTML}

links:
$(MAKE) -C ${DOCDIR} linkcheck
mkdocs serve

# Publish to testpypi
# Will echo the commands to actually publish to be run to publish to actual PyPi
Expand Down
24 changes: 0 additions & 24 deletions docs/Makefile

This file was deleted.

53 changes: 0 additions & 53 deletions docs/api/conditions.rst

This file was deleted.

5 changes: 0 additions & 5 deletions docs/api/configuration.rst

This file was deleted.

5 changes: 0 additions & 5 deletions docs/api/configurationspace.rst

This file was deleted.

26 changes: 0 additions & 26 deletions docs/api/forbidden_clauses.rst

This file was deleted.

Loading
Loading