-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[EEG-BIDS] Add support for events.json file & HED Tags #769
Merged
Merged
Changes from 15 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
cfbe859
refactoring how bids_import handles event files: fetch, insert, archi…
be6c745
bugfixes!
4cf62c6
add event tables to database_lib
58b77f3
lint
514320f
missed one
4574f71
?
jesscall 022707b
some more fixes after testing import with db libs
d76288f
conflicts
45128f0
hed-assemble events
3285b93
lint
ee680a6
more lint edits
9c5622c
rm prints
08b9b7e
rm print
jesscall 073250e
fixed bug with storing events.json and naming of archive
5b654bf
Apply suggestions from code review
jesscall af956f7
Fix values for save to parameter_category_level
jesscall 601ef9b
lint
jesscall 45db229
trying to fix flake8
jesscall fe81761
flake 8?
jesscall 199e702
Update python/lib/eeg.py
jesscall 6214ffe
fixing grwp eventFileID function
174b47b
fix EOL error
jesscall 36475ee
keep making syntax errors :(
jesscall File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
"""This class performs database queries for the physiological_event_archive table""" | ||
|
||
|
||
__license__ = "GPLv3" | ||
|
||
|
||
class PhysiologicalEventArchive: | ||
|
||
def __init__(self, db, verbose): | ||
""" | ||
Constructor method for the PhysiologicalEventArchive class. | ||
|
||
:param db : Database class object | ||
:type db : object | ||
:param verbose : whether to be verbose | ||
:type verbose : bool | ||
""" | ||
|
||
self.db = db | ||
self.table = 'physiological_event_archive' | ||
self.verbose = verbose | ||
|
||
def grep_from_physiological_file_id(self, physiological_file_id): | ||
""" | ||
Gets rows given a physiological_file_id | ||
|
||
:param physiological_file_id : Physiological file's ID | ||
:type physiological_file_id : int | ||
|
||
:return : list of dict containing rows data if found or None | ||
:rtype : list | ||
""" | ||
|
||
return self.db.pselect( | ||
query="SELECT * FROM " + self.table + " WHERE PhysiologicalFileID = %s", | ||
args=(physiological_file_id,) | ||
) | ||
|
||
def insert(self, physiological_file_id, blake2, archive_path): | ||
""" | ||
Inserts a new entry in the physiological_event_archive table. | ||
|
||
:param physiological_file_id : Physiological file's ID | ||
:type physiological_file_id : int | ||
|
||
:param blake2 : blake2b hash | ||
:type blake2 : string | ||
|
||
:param archive_path : Archive's path | ||
:type archive_path : string | ||
""" | ||
|
||
self.db.insert( | ||
table_name = self.table, | ||
column_names = ('PhysiologicalFileID', 'Blake2bHash', 'FilePath'), | ||
values = (physiological_file_id, blake2, archive_path) | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
"""This class performs database queries for the physiological_event_file table""" | ||
|
||
|
||
__license__ = "GPLv3" | ||
|
||
|
||
class PhysiologicalEventFile: | ||
|
||
def __init__(self, db, verbose): | ||
""" | ||
Constructor method for the PhysiologicalEventFile class. | ||
|
||
:param db : Database class object | ||
:type db : object | ||
:param verbose : whether to be verbose | ||
:type verbose : bool | ||
""" | ||
|
||
self.db = db | ||
self.table = 'physiological_event_file' | ||
self.verbose = verbose | ||
|
||
def insert(self, physiological_file_id, event_file_type, event_file): | ||
""" | ||
Inserts a new entry in the physiological_event_file table. | ||
|
||
:param physiological_file_id : physiological file's ID | ||
:type physiological_file_id : int | ||
|
||
:param event_file_type : type of the event file | ||
:type event_file_type : str | ||
|
||
:param event_file : path of the event file | ||
:type event_file : str | ||
|
||
:return : id of the row inserted | ||
:rtype : int | ||
""" | ||
|
||
return self.db.insert( | ||
table_name = self.table, | ||
column_names = ('PhysiologicalFileID', 'FileType', 'FilePath'), | ||
values = (physiological_file_id, event_file_type, event_file), | ||
get_last_id = True | ||
) | ||
|
||
def grep_event_paths_from_physiological_file_id(self, physiological_file_id): | ||
""" | ||
Gets the FilePath given a physiological_file_id | ||
|
||
:param physiological_file_id : Physiological file's ID | ||
:type physiological_file_id : int | ||
|
||
:return : list of FilePath if any or None | ||
:rtype : list | ||
""" | ||
|
||
event_paths = self.db.pselect( | ||
query = "SELECT DISTINCT FilePath " | ||
"FROM physiological_event_file " | ||
"WHERE PhysiologicalFileID = %s", | ||
args=(physiological_file_id,) | ||
) | ||
|
||
event_paths = [event_path['FilePath'] for event_path in event_paths] | ||
|
||
return event_paths | ||
|
||
def grep_event_file_id_from_event_path(self, event_file_path): | ||
""" | ||
Gets the EventFileID given a FilePath | ||
|
||
:param event_file_path : FilePath of physiological event file | ||
:type event_file_path : str | ||
|
||
:return : id of the file specified | ||
:rtype : int | ||
""" | ||
event_file_id = self.db.pselect( | ||
query = "SELECT EventFileID " | ||
"FROM physiological_event_file " | ||
"WHERE FilePath = %s", | ||
args = (event_file_path,) | ||
) | ||
|
||
return event_file_id[0]['EventFileID'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
"""This class performs database queries for the physiological_event_parameter table""" | ||
|
||
|
||
__license__ = "GPLv3" | ||
|
||
|
||
class PhysiologicalEventParameter: | ||
|
||
def __init__(self, db, verbose): | ||
""" | ||
Constructor method for the PhysiologicalEventParameter class. | ||
|
||
:param db : Database class object | ||
:type db : object | ||
:param verbose : whether to be verbose | ||
:type verbose : bool | ||
""" | ||
|
||
self.db = db | ||
self.table = 'physiological_event_parameter' | ||
self.verbose = verbose | ||
|
||
def insert(self, event_file_id, parameter_name, description, long_name, units, is_categorical, hed): | ||
""" | ||
Inserts a new entry in the physiological_event_parameter table. | ||
|
||
:param event_file_id : event file's ID | ||
:type event_file_id : int | ||
|
||
:param parameter_name : Name of the event parameter | ||
:type parameter_name : string | ||
|
||
:param description : Description of the events | ||
:type description : string | ||
|
||
:param long_name : Full name of the event parameter | ||
:type long_name : string | ||
|
||
:param units : Event parameter's units | ||
:type units : string | ||
|
||
:param is_categorical : Whether event has categorical levels ('Y' || 'N') | ||
:type is_categorical : string | ||
|
||
:param hed : Event parameter's HED tag if not categorical | ||
:type hed : string | ||
|
||
:return : id of the row inserted | ||
:rtype : int | ||
""" | ||
|
||
return self.db.insert( | ||
table_name = self.table, | ||
column_names = ('EventFileID', 'ParameterName', 'Description', 'LongName', | ||
'Units', 'isCategorical', 'HED'), | ||
values = (event_file_id, parameter_name, description, long_name, units, is_categorical, hed), | ||
get_last_id = True | ||
) |
67 changes: 67 additions & 0 deletions
67
python/lib/database_lib/physiologicaleventparametercategorylevel.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
"""This class performs database queries for the physiological_event_parameter table""" | ||
|
||
|
||
__license__ = "GPLv3" | ||
|
||
|
||
class PhysiologicalEventParameterCategoryLevel: | ||
|
||
def __init__(self, db, verbose): | ||
""" | ||
Constructor method for the PhysiologicalEventParameter class. | ||
|
||
:param db : Database class object | ||
:type db : object | ||
:param verbose : whether to be verbose | ||
:type verbose : bool | ||
""" | ||
|
||
self.db = db | ||
self.table = 'physiological_event_parameter_category_level' | ||
self.verbose = verbose | ||
|
||
def insert(self, event_parameter_id, level_name, description, hed): | ||
""" | ||
Inserts a new entry in the physiological_event_parameter table. | ||
|
||
:param event_parameter_id : event parameter's ID | ||
:type event_parameter_id : int | ||
|
||
:param level_name : Name of the event parameter's categorical level | ||
:type level_name : string | ||
|
||
:param description : Description of the event parameter's categorical level | ||
:type description : string | ||
|
||
:param hed : Event parameter's categorical HED tag | ||
:type hed : string | ||
""" | ||
|
||
self.db.insert( | ||
table_name = self.table, | ||
column_names = ('EventParameterID', 'LevelName', 'Description', 'HED'), | ||
values = (event_parameter_id, level_name, description, hed), | ||
get_last_id = True | ||
) | ||
|
||
def grep_id_from_physiological_file_id(self, physiological_file_id): | ||
""" | ||
Gets the EventParameterID given a physiological_file_id | ||
|
||
:param physiological_file_id : Physiological file's ID | ||
:type physiological_file_id : int | ||
|
||
:return : id of the row if found or None | ||
:rtype : int | ||
""" | ||
|
||
paramID = self.db.pselect( | ||
query="SELECT EventParameterID " | ||
"FROM " + self.table + " p " | ||
"JOIN physiological_event_file f ON f.EventFileID = p.EventFileID " | ||
"WHERE f.PhysiologicalFileID = %s " | ||
"LIMIT 1", | ||
args=(physiological_file_id,) | ||
) | ||
|
||
return paramID[0]['EventParameterID'] if paramID else None |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is
event_file_path
supposed to be the TSV file?Also, what would happen if an event file is common to all physiological files? Maybe the wrong eventFileID could be returned?
Example:
looking at the return of the function, it looks like it would return
EventFileID
= 1 (for TSV) or 2 (for JSON) which are linked toPhysiologicalFileID
1 even though this isPhysiologicalFileID
2 that is under process.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes it takes in the event_tsv file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok. But what happens in the example above where the same TSV file links to two different
PhysiologicalFileID
? Would it just return the firstPhysiologicalFileID
encountered? If so, sounds like a possible major bug, no?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where do you see the case where two tsv point to the same physioFile?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
when the event.tsv file is identical for all EEG files and therefore put directly under root. I believe this is possible in BIDS.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The event.json file can be shared by all subjects. I don't think the events.tsv can ? This means that ALL recordings have the same events/ stimulus? Seems unlikely, but I could be wrong. Do you have an example of this in the spec somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the BIDS specs: https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/05-task-events.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed by passing in physioFileID as a param in the query