v6.10.0: CPU efficiency improvements, refactoring

✨ Major features and improvements

• Provisional CUDA 9 support. CUDA 9 removes a compilation flag we require for CUDA 8. As a temporary workaround, you can build on CUDA 9 by setting the environment variable CUDA9=1. For example:

CUDA9=1 pip install thinc==6.10.0

• Improve efficiency of NumpyOps.scatter_add, when the indices only have a single dimension. This function was previously a bottle-neck for spaCy.
• Remove redundant copies in backpropagation of maxout non-linearity
• Call floating-point versions of sqrt, exp and tanh functions.
• Remove calls to tensordot, instead reshaping to make 2d dot calls.
• Improve efficiency of Adam optimizer on CPU.
• Eliminate redundant code in thinc.optimizers. There's now a single Optimizer class. For backwards compatibility, SGD and Adam functions are used to create optimizers with the Adam recipe or vanilla SGD recipe.

👥 Contributors

Thanks to @RaananHadar for the pull request!

v6.9.0: Reorganize layers, bug fix to Layer Normalization

✨ Major features and improvements

• Add new namespace modules thinc.v2v, thinc.i2v, thinc.t2t, thinc.t2v that group layer implementations by input and output type v indicates vector, i indicates integer ID, t indicates tensor. The input type refers to the logical unit, i.e. what constitutes a sample.

🔴 Bug fixes

• Fix bug in layer normalization. The bug fix means that models trained with Thinc 6.8 are incompatible with Thinc 6.9. For convenience, a backwards compatibility flag has been added, which can be set with thinc.neural._classes.layernorm.set_compat_six_eight. This flag is off by default.

v6.8.2: Fix packaging of gpu_ops

🔴 Bug fixes

• Fix incorrect packaging of thinc.neural.gpu_ops, introduced in v6.8.1.
• Fix bad data type in thinc.extra.search.MaxViolation, which caused segfaults on some platforms.

v6.8.1: Fix Windows support

✨ Major features and improvements

• Add new foreach layer combinator, which maps a layer across elements of a sequence.
• Add support for predict methods to more layers, for use during decoding.
• Improve correctness of batch normalization. Previously, some layers would force batch normalization to run in training mode, even during prediction. This led to decreased accuracy in some situations.
• Improved efficiency of Maxout layer.

🔴 Bug fixes

• Fix compiler flags for MSVC
• Remove unnecessary Chainer dependency. Now depends on Chainer's cupy package.
• Fix LSTM layer.
• Small bug fixes to beam search

v6.8.0: SELU layer, attention, improved GPU/CPU compatibility

✨ Major features and improvements

• Add SELU layer, from Klambauer et al. (2017).
• Add parametric soft attention layer, as in Yang et al. (2016).
• New higher-order function uniqued, which wraps layers giving them a per-batch cache.
• Improve batch normalization, by tracking activation moving averages.

🔴 Bug fixes

• Fix GPU usage in pooling operations.
• Add optimized code for extracting ngram features.
• Improve CPU/GPU compatibility.
• Improve compatibility of LinearModel class.

👥 Contributors

Thanks to @tammoippen for the pull request!

v6.7.3: Fix convolution on GPU

🔴 Bug fixes

• Convolution is now computed the same on CPU and GPU.

v6.7.2: Bug fixes to serialization

🔴 Bug fixes

• Make order of dicts stable when serializing model.

v6.7.1: Improve serialization

✨ Major features and improvements

• Temporarily revert change to CuPy.
• Improve efficiency of Adam optimizer.

v6.7.0: Fixes to serialization, hash embeddings and flatten ops

✨ Major features and improvements

• Add Model.to_bytes() and Model.from_bytes() methods, to support serialization that's compatible between Python versions.
• Remove code depending on Chainer, and instead depend explicitly on the new cupy subpackage, for simpler GPU installation.
• Improve accuracy for HashEmbed table, by using 4 conditionally independent keys.
• Support padding in flatten and with_flatten ops.
• Use the same hash function on both CPU and GPU, for model compatibility.

🔴 Bug fixes

• HashEmbed now returns correct results for arrays of length not divisible by 16.
• Provide .cu source files in the source distribution.
• Remove unnecessary allocations from the CPU maxout op.
• Fix issue #27: Remove Python2-specific code from setup.py.

v6.6.0: Improved GPU usage and examples

✨ Major features and improvements

• Add GPU kernels for max and mean pool using variable-length sequences.
• thinc.api.FeatureExtractor, for getting features from spaCy Doc objects.

🔴 Bug fixes

• Improve multi-device handling
• thinc.api.add now accepts a variable number of layers.
• Improve Residual class.

⚠️ Backwards incompatibilities

• Some of the example code may be out of date.

📖 Documentation and examples

• Add reader for WikiNER corpora.
• Add example for Twitter NER.
• Add Siamese network example.