0.12.10

🐛 Bug Fixes

Correct the axis which temporal merges are applied to in tracking model @msschwartz21 (#718)

Also adds a flag to make the image normalization layer in the tracking appearance head configurable

🧰 Maintenance

Update DNN dataset and Caliban models @msschwartz21 (#721)

• Update Dynamic Nuclear Net with addition of Hela CTC movies to version 1.1
• Update nuclear segmentation and tracking models after retraining on the updated dataset

General maintenance to make CI happy @rossbar (#719)

What

Some general maintenance to deal with failures cropping up from dependencies either updating or falling out-of-sync with one another

Documentation

• Force suppression of warnings from nbsphinx which break doc tests. Note: nbsphinx breaks sphinx caching (hence the warning)
• Add doc dependencies that nbsphinx needs but are not specified in their deps list
• Modify mocking to fix broken access of numpy features during doc build. This means the docs will no longer build without numpy installed

Linting

The newest version of ruff has checks for non-f-string based string formatting. I've silenced these for now - there are much more important issues than string formatting.

• Updated ruff configuration and invocation to match latest release.

Why

• General maintenance, keep CI green

Rm pins from doc dependencies. @rossbar (#708)

@msschwartz21 IIRC the motivation for many of the pins was issues with list rendering in the datasets and applications galleries. They seem to be resolved locally, if you could double-check the dock preview below and confirm these look as expected then I think we're good to go!

What

• Moving to latest versions of doc deps seems to resolve the issues that motivated the pins in the first place.

Why

• Software maintenance best-practices

CI: Update action versions and ruff incantation @rossbar (#707)

What

• Updates action versions and modifies the ruff incantation to use the new flag now that ruff-format exists

Why

• The inexorable march of time

Add SpotNet v1.1 @elaubsch (#706)

This PR adds SpotNet version 1.1, which adds Airlocalize to the set of spot detection methods used to create consensus annotations.

📚️ Documentation

DOC: Increase gallery thumbnail size @rossbar (#695)

What

• Minor cosmetic adjustments to increase the size of the thumbnails for the gallery. These values seem like a good balance for the RTD theme.
• Also fixes missing mesmer application thumbnail image

Why

• Improve gallery examples

Rm pins from doc dependencies. @rossbar (#708)

@msschwartz21 IIRC the motivation for many of the pins was issues with list rendering in the datasets and applications galleries. They seem to be resolved locally, if you could double-check the dock preview below and confirm these look as expected then I think we're good to go!

What

• Moving to latest versions of doc deps seems to resolve the issues that motivated the pins in the first place.

Why

• Software maintenance best-practices

0.12.9

🐛 Bug Fixes

Make creation of base download location more robust. @rossbar (#692)

What

• Fixup for fetch_data when a cachedir is provided but the parent directory has not yet been created.

Why

• Should make the downloading of assets (i.e. models/datasets) more robust to order and existing directory structure.

0.12.8

🚀 Features

Authenticated SpotNet dataset @elaubsch (#688)

This PR adds the SpotNet dataset to the list of new authenticated datasets, following the established pattern. It also adds a SpotNetExampleData class, which loads the example data for the Polaris example notebooks. This class deviates slightly from the existing pattern for loading data in order to accommodate for the different file types that need to be loaded. The SpotNet dataset has also been added to the datasets gallery in the documentation.

0.12.6

🐛 Bug Fixes

Fix untiling corner-case when no padding in one dimension @rossbar (#673)

What

Closes #665. As noted there, the issue arises from pads of (0, 0) which subsequently lead to 0:0 slices in untiling, giving arrays with shape 0 for that dimension.

AFAICT, the only way to hit this corner case is when one of the dimensions is smaller than model_image_shape, but the other is exactly model_image_shape. If both dimensions are >= model_image_shape, then padding should be False if I'm following the logic correctly.

Why

Bugfix

🧰 Maintenance

Bump version for 0.12.6 release @msschwartz21 (#674)

Update nuclear segmentation and tracking notebooks @msschwartz21 (#671)

This PR updates the three notebooks that are associated with the tracking paper to match our current scripts for training and running the application. I tested each notebook to verify that everything runs.

Update nuclear tracking and segmentation model paths @msschwartz21 (#666)

This PR updates the models used by the NuclearSegmentation and CellTracking applications to the latest versions associated with the upcoming paper.

Cleanup related to transitioning from pylint -> ruff @rossbar (#661)

What

Update codebase to reflect the adoption of ruff instead of pylint.

• Removes pylint configs & pylint ignore flags
• Updates the contributor guide

Why

General cleanup.

Apply pyupgrade to update code style @rossbar (#662)

What

This PR applies pyupgrade (via ruff) to automatically modernize some coding patterns. The way this works: you tell pyupgrade what minimum version of Python you support (3.7 in our case), then it automatically applies linting patterns based on language features in the minimum supported version.

The changes here generally fall in the following categories:

• Stop inheriting from object
• Remove from __future__ imports
• Remove super() args in cases of single inheritance
• Automatically convert older-style string formatting to fstrings.

Why

The main improvement is removing cruft related to old Python versions - perhaps the most notable is the removal of the from __future__ imports and related dels in the __init__.py files. The automatic switch to f-strings is also (IMO) a nice improvement.

📚️ Documentation

Update nuclear segmentation and tracking notebooks @msschwartz21 (#671)

This PR updates the three notebooks that are associated with the tracking paper to match our current scripts for training and running the application. I tested each notebook to verify that everything runs.

0.12.5

Main highlights:

• Add support for Python 3.10
• Add support for numpy version >= 1.24
• Pin scikit-image to <0.20 for compatibility

🐛 Bug Fixes

Correct crop mode bug in tracking application and improve model metadata organization @msschwartz21 (#629)

• The CellTracker had a bug where the crop mode was not being set during inference. This update to the tracking application sets the crop mode correctly.
• Reorganizes model metadata and parameters to set of global variables in each application so that they are easier to maintain and update.

Warning: This PR is dependent on a tracking release after merging vanvalenlab/deepcell-tracking#108.

🧰 Maintenance

Add CI testing against dev branches of deepcell-toolbox and deepcell-tracking @rossbar (#636)

This should help catch incompatibilities between unreleased versions of libraries in the deepcell ecosystem.

What

• Code remains unaffected - this PR is just dedicated to bolstering testing infrastructure

Why

• The deepcell- libraries are interdependent: deepcell-tf depends on deepcell-tracking and deepcell-toolbox. If there is a change in one of these dependencies, there is no way to tell in the automated test running whether this will break something in deepcell-tf until the underlying libraries are released. Testing against the dev branches will catch potential issues sooner, at the expense of being noisier and reducing test specificity (failures can originate from either deepcell-tf or the dependencies). Overall however I think this should improve the ability to things consistent across libraries.

Prepare for 0.12.5 release @rossbar (#659)

What

• Bump version numbers in final step before next patch release

Update copyright notice to 2023. @rossbar (#658)

What

• Update year in copyright notice. Accomplished with:

find . -type f -exec sed -i "s/2016-2022/2016-2023/g" {} \;

Why

• I plan to do a 0.12.5 release soon (primarily for the scikit-image pin) so I figured I'd get this in as well.

Update pydot dependency: soft dep for `keras.utils.plot_model` @rossbar (#647)

What

pydot is listed as a dependency, but is not actually used in the project, so should be safe to remove.

Why

Decreasing the dependency footprint is always beneficial. Doubly-so in the case of pydot, which has not been actively maintained in a while, see e.g. networkx/networkx#5723

Bump action versions to avoid deprecation warnings. @rossbar (#653)

What

The builtin github actions checkout, setup-python, and cache have all been updated to a later version of node. There are now deprecation warnings for the previous versions in the actions logs.

Why

General maintenance to keep the CI in good shape.

Note there may be other actions that need to be updated, but I'm starting with the main ones so I can see what remains in the logs after these updates.

Pin scikit-image to avoid expired deprecations. @rossbar (#656)

What

• Fix for #655 .

Why

My vote is to pin scikit-image then do a patch release. For the next minor release the pin should be updated to >=0.19.

Deprecate entire `export_utils` module @rossbar (#649)

What

A followup to #648. With the deprecation of export_model_to_tflite, it is now the case that every function in the export_utils module has been deprecated. Therefore I propose to deprecate the entire module. We can do this using the module getattr to emit warnings if a user ever tries to access the two public names (i.e. export_model or export_utils).

In practice this means import patterns like:

>>> from deepcell.utils import export_model  # or export utils

will now raise a deprecation warning as well.

The module getattr was added in Python 3.7 - see PEP 562 for details.

Why

Further cleanup related to the export_model functions, all of which are deprecated in favor of using tf.keras.models.save_model directly.

Deprecate export\_model\_to\_tflite @rossbar (#648)

What

Deprecate export_utils.export_model_to_tflite.

Why

Notify users who may still be using this function to switch to tf.keras.models.save_model. Closes gh-645.

Lint with ruff @rossbar (#646)

What

Adopt ruff as a linter for the project. See also: vanvalenlab/deepcell-toolbox#137 and vanvalenlab/deepcell-tracking#113.

Why

Primarily to add automated linting for future code submissions, though this PR also contains a few minor fixups to address existing issues.

This one's a bit of a bear in terms of files touched and lines modified - I'm more than happy to split this up into smaller PRs to make review easier, just LMK!

Update test to use second num\_semantic\_classes input. @rossbar (#644)

What

Closes gh-643. If there's a reason not to use the nsc2 input, then alternatively we can delete that var.

Why

See gh-643 for context.

Add python3.10 support @rossbar (#639)

What

Add support for Python 3.10

Why

General software updates. See also vanvalenlab/deepcell-tracking#111 and vanvalenlab/deepcell-toolbox#128

Fix failures due to invalid numpy scalars. @rossbar (#634)

What

Fixes the failures in deepcell-tf for numpy v1.24 by updating to use the

Why

Remove the upper bound on numpy.

Marking as draft for now, as this depends on vanvalenlab/deepcell-tracking#112 as well. The tracking tests in deepcell-tf will continue to fail until those changes make it into a release.

Update Dockerfile to jupyter lab @rdilip (#637)

What

• Removed jupyter notebook call

A collection of minor documentation updates @rossbar (#633)

What

The major change is removing the pins to sphinx, docutils, etc. AFAICT the motivating factors for the pins are no longer relevant - see e.g. #320 and #526.

Some other minor changes include:

• Minor configuration updates to get rid of warnings
• Updating intersphinx to point to the stable Python docs instead of 3.7
• Modifying the heading levels in one of the example notebooks to fix the toctree nav column

Why

Sphinx 2.3.1 is 2 major releases behind stable - being pinned this far back will make it difficult to reliably change/update the docs.

📚️ Documentation

Rm blurb about Python2/TensorFlow 1 from README. @rossbar (#651)

What

Removing blurb from README about running with Python2/TensorFlow 1.

Why

The chances of this working out of a containerized environment is practically nil and certainly not worth the effort for users.

I'm also using this change as a test for the RTD docs preview feature in CI.

A collection of minor documentation updates @rossbar (#633)

What

The major change is removing the pins to sphinx, docutils, etc. AFAICT the motivating factors for the pins are no longer relevant - see e.g. #320 and #526.

Some other minor changes include:

• Minor configuration updates to get rid of warnings
• Updating intersphinx to point to the stable Python docs instead of 3.7
• Modifying the heading levels in one of the example notebooks to fix the toctree nav column

Why

Sphinx 2.3.1 is 2 major releases behind stable - being pinned this far back will make it difficult to reliably change/update the docs.

0.12.4

🧰 Maintenance

MAINT: Pin numpy to avoid errors due to expired scalar deprecations @rossbar (#630)

Resolve failures in deepcell due to code that depends on numpy features that were removed in numpy v1.24.

Bump model versions for nuclear segmentation and tracking @msschwartz21 (#627)

Updates the nuclear segmentation model from model-registry # 34 and the tracking model from model-registry # 36

0.12.3

🧰 Maintenance

Swap m2r with m2r2 and update version number to 0.12.3 @msschwartz21 (#623)

Bump version number for new release

Also includes a change from m2r to m2r2 for our documentation pipeline. m2r is no longer being maintained so it has been replaced with a fork with more active maintenance. https://github.com/CrossNox/m2r2

Add support for EfficientNetV2 backbones to the get_backbone utility function @msschwartz21 (#619)

What

• Add support for EfficientNetV2 backbones to the get_backbone utility function

Why

• Exposes EfficientNetV2 backbones for use in deepcell model architectures

0.12.2

🐛 Bug Fixes

Add matplotlib to setup.py @msschwartz21 (#610)

What

• Add matplotlib requirement to setup.py

Why

• Pip installations used the requirements listed in setup.py so currently matplotlib is not installed when pip installing deepcell

Update mesmer post-processing args @ngreenwald (#609)

What

Updated the post-processing parameters for the Mesmer model. Also updates the notebook to describe how post-processing can be modified.

Why

The newly retrained model has different parameters that give the best results. In addition, I've gotten questions from a few different people about how to tweak the model output, having it in the notebook will make it easy for people to see the effects.

0.12.1

🚀 Features

Create TFRecords for tracking datasets @vanvalen (#602)

What

• Added functionality to create TFRecords for tracking datasets

Why

• As the training datasets grow in size, they are no longer able to fit in memory (as is the case with image generators). Adding functionality for TFRecords will let us train on larger datasets as they are loaded dynamically from disk during training rather than into memory all at once.

🐛 Bug Fixes

Fix tracking model bug that pinned n_filters, encoder_dim and embedding_dim to 64 @vanvalen (#606)

What

• Fixed a bug that required the tracking model to have n_filters, encoder_dim, and embedding_dim be pinned to 64

Why

• Model optimization is going to require us to change these parameters to improve performance and reduce model size. This pull request makes that substantially easier by fixing this bug.

🧰 Maintenance

Bump version to 0.12.1 @msschwartz21 (#605)

Expose option for fixed crops in the Track data object @vanvalen (#607)

What

• Modify the Track class so that it allows hooks into get_image_features for crop_mode and norm

Why

• Creating the appearance image feature by cropping and resizing removes information about cell size that the model can use to make more accurate tracking predictions. A previous update to deepcell-tracking (vanvalenlab/deepcell-tracking#98) introduced the crop_mode (either 'fixed' or 'resize') and norm flags to get_image_features. This pull request exposes these flags to the Track class.

Bump `deepcell-tracking` to 0.6.0 @msschwartz21 (#603)

What

• Bump deepcell-tracking to the new minor release
• Update imports to match the reorganization introduced in this release

Update the docstring for `format_output_mesmer` @curlup (#601)

What

Doc-string for format_output_mesmer is now correctly saying "ValueError: if model output list is not len(4)"

Why

Because format_output_mesmer code diverged from the doc in what is expected length of model output list

0.12.0

🚀 Features

Introduce functions for reading and writing TF Records for segmentation data @vanvalen (#597)

What

Included functions to save datasets as tfrecords and load them into tf.data.Dataset objects

Why

As our training datasets grow, it is becoming difficult to load full datasets into memory. By introducing support for tfrecords, we can load portions of datasets from disk on the fly during training.

🧰 Maintenance

Update models after retraining on deepcell 0.12.0rc @msschwartz21 (#599)

What

• Update models with versions that were trained on tensorflow 2.8 (vanvalenlab/model-registry#17)

Why

• Models should use the same version of tensorflow for predictions as they were trained on

Add option for either batch or layer norm in tracking model @msschwartz21 (#598)

What

• Provide the option to select either BatchNormalization or LayerNormalization in GNNTrackingModel

Why

• This option makes it possible to train the model with a batch size of 1 when layer normalization is enabled.

Update TF_VERSION build arg in docker build workflow @msschwartz21 (#596)

The TF_VERSION build arg has to be updated manually

Update Tensorflow to 2.8 @msschwartz21 (#595)

This PR updates tensorflow to 2.8 and drops support for python 3.6. The following changes were necessary to make this upgrade possible:

• Change imports from tensorflow.python.keras to tensorflow.keras which was a change introduced with tensorflow 2.6
• Remove convolutional recurrent layers and their functionality from featurenet and panopticnet. Key functions that were used in the convolutional recurrent layer are no longer available in keras.
• Change imports from tensorflow.keras to keras: keras_parameterized, conv_utils, test_utils
• Drop support for python 3.6

I retrained the nuclear model in the model-registry using this branch of deepcell and the performance was comparable.