Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Datajoint pipeline updates to main #73

Closed
wants to merge 306 commits into from
Closed

Datajoint pipeline updates to main #73

wants to merge 306 commits into from

Conversation

iamamutt
Copy link
Contributor

@iamamutt iamamutt commented Feb 7, 2022

Draft PR to merge DJ updates into main.

Other outstanding PR's to main should be resolved first. Once those are merged I can pull those changes, resolve conflicts, and submit for review.

iamamutt and others added 30 commits August 16, 2021 15:59
@iamamutt
python entrypoint
1f18210
@iamamutt
convert ingestion from process.py into python package script
14a130e
Merge pull request #20 from iamamutt/ingestion
289a7b8 
Changes to ingestion routine script
@iamamutt
move load_arena_setup out of __init__ so as to not trigger db conne…
02a4301 
…ction prematurely
added TaskProtocol
18c248d
move QC-related tables to a qc schema
0026da3
Merge branch 'datajoint_pipeline' of https://github.com/vathes/aeon_m…
4a42818 
…echa into datajoint_pipeline
rename experiment schema to acquisition
e5687cb
rename TimeBin -> Chunk and SessionEpoch -> TimeSlice
5e38c3d
Epoch -> TimeSlice rename
ab54cf3
@iamamutt
doc update
fe3316f
@iamamutt
updates for containerization
994b6d3
@iamamutt
add datajoint config defaults specific to container
fb83f05
@iamamutt
path parsing
68a9658
@iamamutt
use docker-compose for environment vars
0ea1311
@iamamutt
add dj config to dockerfile
841c700
@iamamutt
update image building inst
7e1ffc0
@iamamutt
docker updates
3b72e93
Merge pull request #22 from ttngu207/datajoint_pipeline
553f7b8 
Modified schema based on newly agreed terminology
Merge branch 'datajoint_pipeline' into datajoint_pipeline
29dce2e
Merge pull request #3 from vathes/datajoint_pipeline
1d7983c 
terminology changes
@iamamutt
move stuff out of __init__ that triggers db connection on package load
20d8583
@iamamutt
add additional input arguments
b9ed571
@iamamutt
metavar info update
5205829
Merge pull request #23 from iamamutt/datajoint_pipeline
86aa863 
Ingestion process
@iamamutt
remove import that no longer exists
6b9c783
@iamamutt
find path to yml from diff locations
84c7e10
@iamamutt
insert metadata option
3264315
@iamamutt
update more epoch changes to time slice
836d04c
@iamamutt
update time_slice
eb2aba2
Thinh Nguyen and others added 26 commits February 16, 2022 15:33
add docker-compose for labbook
2d665bb
Session related tables are now part of the analysis.ina_arena module
be81b2f
more rename Session -> InArena
b230c80
update comment
866b9fb
Merge branch 'datajoint_pipeline' of https://github.com/SainsburyWell…
e6db726 
…comeCentre/aeon_mecha into datajoint_pipeline
Merge branch 'datajoint_pipeline' of https://github.com/ttngu207/aeon…
7d3edc9 
…_mecha into datajoint_pipeline
@iamamutt
Merge pull request #84 from ttngu207/datajoint_pipeline
3ac82c9 
more comprehensive design for position tracking - supporting multi-animal position ingestion
@iamamutt
fix(gh action): rebuild container for arm64 platform
208acf0 
Revert to stable buildx and limit qemu platforms to only those needed.
Merge pull request #85 from SainsburyWellcomeCentre/joseph
a9b0062 
fix(gh action): rebuild container for arm64 platform
@iamamutt
ci(gh-actions): clean up before build
42202dd
@iamamutt
Merge branch 'joseph' into datajoint_pipeline
2e71a57
minor bugfix
f85df65
update docker-compose for labbook
7522d7c
Merge branch 'datajoint_pipeline' into separate_analysis
d184948
Merge branch 'datajoint_pipeline' of https://github.com/SainsburyWell…
4547102 
…comeCentre/aeon_mecha into datajoint_pipeline
Merge branch 'datajoint_pipeline' into separate_analysis
c1e853e
remove position_z from tracking
291a44e
@iamamutt
Merge pull request #86 from ttngu207/separate_analysis
dca044e 
Separating the analyses to a submodule `analysis` with `Session` renamed to `InArena`
Create clone_and_freeze_exp01.py
f34e6b9
add method to ingest epochs - code refactoring
6afb11f
code cleanup/refactoring - preparing for exp0.2 ingestion
2fd3e97
include Epoch ingestion in process
bf75f30
exp0.2 mata data ingest - still WIP
0c3043d
Merge branch 'main' of https://github.com/SainsburyWellcomeCentre/aeo…
466c153 
…n_mecha into datajoint_pipeline
epoch and chunk ingestion for exp0.2
5beb3bf
@iamamutt
Merge pull request #89 from ttngu207/datajoint_pipeline
e22c755 
Code cleanup and refactoring - preparing for exp 0.2 ingestion - script to clone and freeze exp 0.1 data
jkbhagatio
jkbhagatio reviewed Mar 12, 2022
View reviewed changes
aeon/dj_pipeline/docs/notebooks/Example_queries_exp0.1.ipynb
@@ -1,224 +0,0 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ttngu207 @iamamutt hey guys, is there a reason these example notebooks were deleted?

@jkbhagatio jkbhagatio marked this pull request as ready for review March 12, 2022 15:59
@jkbhagatio
Copy link
Member

merged 41ce4eb

@jkbhagatio jkbhagatio closed this Mar 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkbhagatio jkbhagatio jkbhagatio left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@iamamutt @jkbhagatio @ttngu207 @glopesdev