Group part delete propagation (#899)

* Create class for group parts to help propagate deletes

* spelling

* update changelog

* Part delete edits (#946)

* Add spyglass version to created analysis nwb files (#897)

* Add sg version to created analysis nwb files

* update changelog

* Change existing source script to spyglass version (#900)

* Add pynapple support (#898)

* Preliminary code

* Add retrieval of file names

* Add get_nwb_table function

* Update docstrings

* Update CHANGELOG.md

* Hot fixes for clusterless `get_ahead_behind_distance` and `get_spike_times` (#904)

* Squeeze results

* Make method and not class method

* Update CHANGELOG.md

* fix bugs in fetch_nwb (#913)

* Check for entry in merge part table prior to insert (#922)

* check for entry in merge part table prior to insert

* update changelog

* Kachery fixes (#918)

* Prioritize datajoint filepath for getting analysis file abs_path

* remove deprecated kachery tables

* update changelog

* fix lint

---------

Co-authored-by: Samuel Bray <samuelbray@som-dfvnn9m-lt.ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@users.noreply.github.com>

* remove old tables from init (#925)

* Fix improper uses of strip (#929)

Strip will remove leading characters

* Update CHANGELOG.md

* Misc Issues (#903)

* #892

* #885

* #879

* Partial address of #860

* Update Changelog

* Partial solve of #886 - Ask import

* Fix failing tests

* Add note on order of inheritace

* #933

* Could not replicate fill_nan error. Reverting except clause

* Export logger (#875)

* WIP: rebase Export process

* WIP: revise doc

* ✅ : Generate working export script

* Cleanup: Expand notebook, migrate export process from graph class to export

* Revert dj_chains related edits

* Update changelog

* Revise doc

* Address review comments #875

* Remove walrus in  eval

* prevent log on preview

* Fix arg order on fetch, iterate over restr

* Add upstream analysis files during cascade. Address false positive fetch

* Avoid regen file list on revisit node

* Bump Export.Table.restr to mediumblob

* Revise Export.Table uniqueness to include export_id

* Spikesorting quality of life helpers (#910)

* add utitlity function for finding spikesorting merge ids

* add option to select v1 sorts that didn't go through artifact detection

* add option to return merge keys as dicts for future restrictions

* Add tool to get brain region and electrode info for a spikesorting merge id

* update changelog

* style cleanup

* style cleanup

* fix restriction bug for curation_id

* account for change or radiu_um argument name in spikeinterface

* only do joins with metric curastion tables if have relevant keys in the restriction

* Update tutorial to use spikesorting merge table helper functions

* fix spelling

* Add logging of AnalysisNwbfile creation time and file size (#937)

* Add logging for any func that creates AnalysisNwbfile

* Migrate create to top of respective funcs

* Use pathlib for file size. Bump creation time to top of  in spikesort

* Clear pre_create_time on create

* get/del -> pop

* Log when file accessed (#941)

* Add logging for any func that creates AnalysisNwbfile

* Fix bug on empty delete in merge table (#940)

* fix bug on empty delete in merge table

* update changelog

* fix spelling

---------

Co-authored-by: Chris Brozdowski <Chris.Broz@ucsf.edu>

* Remove master restriction

* Part delete takes restriction from self

---------

Co-authored-by: Samuel Bray <sam.bray@ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@users.noreply.github.com>
Co-authored-by: Samuel Bray <samuelbray@som-dfvnn9m-lt.ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@bu.edu>

* Fix linting

---------

Co-authored-by: Chris Brozdowski <Chris.Broz@ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@users.noreply.github.com>
Co-authored-by: Samuel Bray <samuelbray@som-dfvnn9m-lt.ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@bu.edu>

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Change Log
 
+## [0.5.3] (Unreleased)
+
+### Release Notes
+
+<!-- Running draft to be removed immediately prior to release. -->
+
+### Infrastructure
+
+- Create class `SpyglassGroupPart` to aid delete propagations #899
+
 ## [0.5.2] (April 22, 2024)
 
 ### Infrastructure
@@ -227,3 +237,4 @@
 [0.5.0]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.5.0
 [0.5.1]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.5.1
 [0.5.2]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.5.2
+[0.5.3]: https://github.com/LorenFrankLab/spyglass/releases/tag/0.5.3

src/spyglass/decoding/v1/clusterless.py

@@ -32,7 +32,7 @@
 )  # noqa: F401
 from spyglass.position.position_merge import PositionOutput  # noqa: F401
 from spyglass.settings import config
-from spyglass.utils import SpyglassMixin, logger
+from spyglass.utils import SpyglassMixin, SpyglassMixinPart, logger
 
 schema = dj.schema("decoding_clusterless_v1")
 
@@ -44,7 +44,7 @@ class UnitWaveformFeaturesGroup(SpyglassMixin, dj.Manual):
     waveform_features_group_name: varchar(80)
     """
 
-    class UnitFeatures(SpyglassMixin, dj.Part):
+    class UnitFeatures(SpyglassMixinPart):
         definition = """
         -> UnitWaveformFeaturesGroup
         -> UnitWaveformFeatures

src/spyglass/decoding/v1/core.py

@@ -13,7 +13,7 @@
     restore_classes,
 )
 from spyglass.position.position_merge import PositionOutput  # noqa: F401
-from spyglass.utils import SpyglassMixin
+from spyglass.utils import SpyglassMixin, SpyglassMixinPart
 
 schema = dj.schema("decoding_core_v1")
 
@@ -94,7 +94,7 @@ class PositionGroup(SpyglassMixin, dj.Manual):
     position_variables = NULL: longblob # list of position variables to decode
     """
 
-    class Position(SpyglassMixin, dj.Part):
+    class Position(SpyglassMixinPart):
         definition = """
         -> PositionGroup
         -> PositionOutput.proj(pos_merge_id='merge_id')

src/spyglass/spikesorting/analysis/v1/group.py

@@ -6,7 +6,7 @@
 
 from spyglass.common import Session  # noqa: F401
 from spyglass.spikesorting.spikesorting_merge import SpikeSortingOutput
-from spyglass.utils.dj_mixin import SpyglassMixin
+from spyglass.utils.dj_mixin import SpyglassMixin, SpyglassMixinPart
 
 schema = dj.schema("spikesorting_group_v1")
 
@@ -51,7 +51,7 @@ class SortedSpikesGroup(SpyglassMixin, dj.Manual):
     sorted_spikes_group_name: varchar(80)
     """
 
-    class Units(SpyglassMixin, dj.Part):
+    class Units(SpyglassMixinPart):
         definition = """
         -> master
         -> SpikeSortingOutput.proj(spikesorting_merge_id='merge_id')

src/spyglass/utils/__init__.py

@@ -1,5 +1,5 @@
 from spyglass.utils.dj_merge_tables import _Merge
-from spyglass.utils.dj_mixin import SpyglassMixin
+from spyglass.utils.dj_mixin import SpyglassMixin, SpyglassMixinPart
 from spyglass.utils.logging import logger
 
-__all__ = ["_Merge", "SpyglassMixin", "logger"]
+__all__ = ["_Merge", "SpyglassMixin", "SpyglassMixinPart", "logger"]

src/spyglass/utils/dj_mixin.py

@@ -737,3 +737,15 @@ def file_like(self, name=None, **kwargs):
             logger.error(f"No file-like field found in {self.full_table_name}")
             return
         return self & f"{attr} LIKE '%{name}%'"
+
+
+class SpyglassMixinPart(SpyglassMixin, dj.Part):
+    """
+    A part table for Spyglass Group tables. Assists in propagating
+    delete calls from upstreeam tables to downstream tables.
+    """
+
+    def delete(self, *args, **kwargs):
+        """Delete master and part entries."""
+        restriction = self.restriction or True  # for (tbl & restr).delete()
+        (self.master & restriction).delete(*args, **kwargs)