Merge pull request #802 from gchq/feature/benchmarking

Feature/benchmarking

.cspell/custom_misc.txt

@@ -11,24 +11,34 @@ coresets
 coresubset
 coresubsets
 cubature
+delaxes
 diag
 docstrings
+ecolor
 eigendecomposition
 elementwise
+errorbar
+figtext
+fontsize
 forall
 GCHQ
 Gramian
 gramians
+horizontalalignment
+hspace
 infty
 kbar
 kdtree
 kernelised
 kernelized
 KSD
+linestyle
 linewidth
 mapsto
 Matern
+maxs
 ml.p3.8xlarge
+MNIST
 ndmin
 parsable
 PCIMQ
@@ -43,6 +53,7 @@ recomb
 refs
 regulariser
 RKHS
+rngs
 RPCHOLESKY
 sdist
 sigmas
@@ -52,3 +63,7 @@ TLDR
 typecheck
 venv
 WMMD
+wspace
+xticks
+yerr
+yscale

.cspell/library_terms.txt

@@ -24,6 +24,7 @@ cmap
 color
 Concretization
 cuda
+densmap
 dispname
 dollarmath
 dtype
@@ -63,6 +64,7 @@ leggauss
 linalg
 linkcheck
 literalinclude
+llvmlite
 logpdf
 lstsq
 mathbb
@@ -107,6 +109,7 @@ pyqt
 pyright
 pyroma
 pytest
+pytorch
 pytree
 pytrees
 quickstart
@@ -129,11 +132,14 @@ tensordot
 texttt
 toctree
 tomli
+torch
+torchvision
 tqdm
 triu
 ttest
 tuplegetter
 typehints
+umap
 undoc
 unflatten
 unsrt

.github/workflows/unittests.yml

@@ -22,12 +22,6 @@ jobs:
           - ubuntu-latest
           - windows-latest
           - macos-latest
-        # macos-latest does not support python 3.9
-        # https://github.com/actions/setup-python/issues/696#issuecomment-1637587760
-        exclude:
-        - { python-version: "3.9", os: "macos-latest" }
-        include:
-        - { python-version: "3.9", os: "macos-13" }
     runs-on: ${{ matrix.os }}
     env:
       # Set the Python version that `uv` will use for its virtual environment.

CHANGELOG.md

@@ -10,9 +10,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - Contributor-facing features:
-  - Automated testing of the built package before release (https://github.com/gchq/coreax/issues/854)
+  - Automated testing of the built package before release
+    (https://github.com/gchq/coreax/issues/854)
 - Added an analytical test for SteinThinning, and associated documentation in
   `tests.unit.test_solvers`. (https://github.com/gchq/coreax/pull/842)
+- Added benchmarking scripts:
+  - MNIST (train a classifier on coreset of training data, test on testing data)
+    (https://github.com/gchq/coreax/pull/802)
+  - Blobs (generate synthetic data using `sklearn.datasets.make_blobs` and compare MMD
+    and KSD metrics) (https://github.com/gchq/coreax/pull/802)
 
 
 ### Fixed

benchmark/__init__.py

@@ -0,0 +1,15 @@
+# © Crown Copyright GCHQ
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Different benchmarking tests to compare performance of different algorithms."""