Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fault-permitting insert and print-out #212

Closed
khl02007 opened this issue Apr 13, 2022 · 5 comments
Closed

Fault-permitting insert and print-out #212

khl02007 opened this issue Apr 13, 2022 · 5 comments
Assignees
Labels

Comments

@khl02007
Copy link
Collaborator

@MichaelCoulter

@khl02007 khl02007 changed the title print our more detailed output message when inserting a new session print out more detailed output message when inserting a new session Apr 13, 2022
@edeno
Copy link
Collaborator

edeno commented Apr 14, 2022

@MichaelCoulter could you leave a more detailed description of what you want in the output?

@MichaelCoulter
Copy link
Collaborator

it would be great to have: a list of associated and video files that are linked to the nwb file and actually exist, a list of intervals of the position data (including estimated time in minutes), a list of intervals of the neural data (including estimated time in minutes), a check that the timestamps of the position and neural intervals are overlapping.

@edeno edeno added the common label Oct 13, 2023
@edeno edeno assigned CBroz1 and unassigned zoldello Dec 23, 2023
@CBroz1 CBroz1 changed the title print out more detailed output message when inserting a new session Fault-permitting insert and print-out Feb 1, 2024
@CBroz1
Copy link
Member

CBroz1 commented Feb 1, 2024

One approach here would be to run the insert permissively (if one insert fails, note it and keep moving), and then run generate a printout with TableChains showing the connections between NwbFile and the respective endpoints of insert_all_common

If you have some time, @MichaelCoulter, I would be interested in your feedback on the the current printouts of TableChain.

from spyglass.utils.dj_chains import TableChain
from spyglass.common import Nwbfile, OtherTable

my_chain = TableChain(Nwbfile, OtherTable)
my_restr = 'nwb_file_name LIKE "name%"'
print(str(my_chain)) # quick  A -> D
print(repr(my_chain)) # longer A -> B -> C -> D 
my_chain.join(my_restr) # join of all A-D with restriction

Is this a useful representation of the data for the inserted data? Or would it be better to have:

New IntervalList: 2, 'name1', 'name2'
New VideoFile: 1, 'video_name'
etc.

@CBroz1
Copy link
Member

CBroz1 commented Feb 9, 2024

I'm currently implementing a solution that will keep track of errors, and not log successes. I plan to close this issue when merged, but if a successes or combined printout is needed, lmk and we can reopen

CBroz1 added a commit to CBroz1/spyglass that referenced this issue Feb 9, 2024
edeno pushed a commit that referenced this issue Feb 10, 2024
#824)

* #719, #804, #212

* #768

* Add merge delete and populate

* Changes following PR review @edeno

* Replace delayed import of ImportedSpikeSorting
@edeno
Copy link
Collaborator

edeno commented Feb 10, 2024

Fixed by #824

@edeno edeno closed this as completed Feb 10, 2024
edeno added a commit that referenced this issue Feb 20, 2024
* Fault-permit insert and remove mutual exclusivity protections on Merge (#824)

* #719, #804, #212

* #768

* Add merge delete and populate

* Changes following PR review @edeno

* Replace delayed import of ImportedSpikeSorting

* Update CITATION.cff (#826)

* Update CITATION.cff

* Update change log

* Update ref

* Add MUA notebook and fix numbering.

* Only apply include labels filter if include_labels not empty (#827)

* dont skip unit if include_labels list is empty

* update check for np array size

* gh-actions docs fixes (#828)

* Update 'latest' in docs deploy

* Docs bugfix. Rename Link action, incorporate cspell.

* MUA as own heading

* Update README.md

* Fix citation

* Fix citation

* include all relevant restrictions on video file

* Proposed structure for user roles. (#832)

* Add roles

* Remove use of unix user group. Add note for retroactive role assign

* Add docs on roles and external tables. Reduce key length

* Fix test for update of position tools (#835)

Related to single LED halving the data bug

* fix build error in mamba and restriction for dlc

* flush stdout before converting mp4

* Fix notebook name (#840)

* remove deprecated yaml.safe_load function

* Fix test

* replace deprecated yaml.safe_load function

* only call no_transaction_make if video key not present

---------

Co-authored-by: Chris Brozdowski <Chris.Broz@ucsf.edu>
Co-authored-by: Eric Denovellis <edeno@users.noreply.github.com>
Co-authored-by: Eric Denovellis <edeno@bu.edu>
edeno added a commit that referenced this issue Feb 21, 2024
* allow for  multicamera epochs

* change VideoFile definition

* update video_file_num iteration

* add update methods

* change VideoFile restriction in get_video_path

* modify formatting in position_dlc_project

* allow selective video selection in pose estimation

* add NotImplementedError to DLCProject insert

* allow for video file addition after proj creation

* fix add_video_files

* fix video_file_num determination in VideoFile

* modify add_video_files method

* change call to add_video_files

* modify interval_list_name call

* Fix linting errors

* Tested DLC multicam pipeline  (#841)

* Fault-permit insert and remove mutual exclusivity protections on Merge (#824)

* #719, #804, #212

* #768

* Add merge delete and populate

* Changes following PR review @edeno

* Replace delayed import of ImportedSpikeSorting

* Update CITATION.cff (#826)

* Update CITATION.cff

* Update change log

* Update ref

* Add MUA notebook and fix numbering.

* Only apply include labels filter if include_labels not empty (#827)

* dont skip unit if include_labels list is empty

* update check for np array size

* gh-actions docs fixes (#828)

* Update 'latest' in docs deploy

* Docs bugfix. Rename Link action, incorporate cspell.

* MUA as own heading

* Update README.md

* Fix citation

* Fix citation

* include all relevant restrictions on video file

* Proposed structure for user roles. (#832)

* Add roles

* Remove use of unix user group. Add note for retroactive role assign

* Add docs on roles and external tables. Reduce key length

* Fix test for update of position tools (#835)

Related to single LED halving the data bug

* fix build error in mamba and restriction for dlc

* flush stdout before converting mp4

* Fix notebook name (#840)

* remove deprecated yaml.safe_load function

* Fix test

* replace deprecated yaml.safe_load function

* only call no_transaction_make if video key not present
---------

Co-authored-by: dpeg22 <dpeg22@gmail.com>
Co-authored-by: CBroz1 <CBrozdowski@yahoo.com>
Co-authored-by: Samuel Bray <sam.bray@ucsf.edu>
Co-authored-by: Chris Brozdowski <Chris.Broz@ucsf.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants