-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ENH] BEP 020 Eye Tracking #1128
base: master
Are you sure you want to change the base?
Conversation
correction of text
(NOTE: I'll cross-post this message across several BEP threads) Hi there, just a quick notification that we have just merged #918 and it may be interesting to look at the implications for this BEP. We are introducing "BIDS URIs", which unify the way we refer to and point to files in BIDS datasets (as opposed to "dataset-relative" or "subject-relative" or "file-relative" links). If the diff and discussion in the PR is unclear, you can also read the rendered version: https://bids-specification.readthedocs.io/en/latest/02-common-principles.html#bids-uri Perhaps there are things in the BEP that need adjusting now, but perhaps also not -- in any case it's good to be aware of this new feature! Let me know if there are any questions, comments, or concerns. |
<<a href="../glossary.html#data_type-common_principles">datatype</a>>/ | ||
<matches>[_<a href="../appendices/entities.html#recording">recording</a>-<<a href="../glossary.html#label-common_principles">label</a>>]_<<a href="../glossary.html#physio-suffixes">physio</a>|<a href="../glossary.html#physioevents-suffixes">physioevents</a>><a href="../glossary.html#json-extensions">.json</a> | ||
<matches>[_<a href="../appendices/entities.html#recording">recording</a>-<<a href="../glossary.html#label-common_principles">label</a>>]_<<a href="../glossary.html#physio-suffixes">physio</a>|<a href="../glossary.html#physioevents-suffixes">physioevents</a>><a href="../glossary.html#tsvgz-extensions">.tsv.gz</a> | ||
</code></pre></div> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAIK shouldn't be HTML here -- I can't read it... Use MACRO as below or demonstrate on a simple Text
example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Problem is that that matches
is not in the glossary so we can't have macro yet.
Opened this so we can have it in the glossary and then after make it easier to MACRO the 💩 out of this.
#1781
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See also: #1128 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and also... all PRs are rendered as HTML: https://bids-specification--1128.org.readthedocs.build/en/1128/modality-specific-files/physiological-recordings.html#physiology-events
😉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAIK shouldn't be HTML here -- I can't read it... Use MACRO as below or demonstrate on a simple
Text
example?
Yes, we're using this to set some expectation for the macros to deal with this.
"Columns": ["onset", "duration", "trial_type", "blink", "message"], | ||
"Description": "Messages logged by the measurement device", | ||
"ForeignIndexColumn": "timestamp", | ||
"blink": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shameless plug: this is the situation I would like to avoid in
where I guess such file would look like
{
"Description": "Messages logged by the measurement device",
"ForeignIndexColumn": "timestamp",
"Columns": {
"onset": { "Index": 1},
"duration": {"Index": 2},
"trial_type": {
"Description": "Event type as identified by the eye-tracker's model (either 'n/a' if not applicabble, 'fixation', or 'saccade')."
"Index": 3},
"blink": {
"Description": "One indicates if the eye was closed, zero if open."
"Index" : 4},
"message": {
"Description": "String messages logged by the eye-tracker.",
"Index": 5}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm +1 on this and the solution.
However, I understand that can only be implemented in BIDS 2.0 because would introduce backward breaking changes, right?
If so, I think this is something we cannot resolve here :(
Sorry, I haven't been following this, but if adding a timestamp column is an issue, have you looked at blood for prior art, which has a |
It is not an issue (other than whether we are correctly configuring the schema #1128 (comment) and allowing an optional column to be placed first in the file when there are two mandatory columns after) It seems that most of the ET devices do generate a timestamp/time column, so that would not be problematic at all. |
Some metadata of the `StimulusPresentation` object become REQUIRED with the presence of | ||
[eye-tracking data](physiological-recordings.md#eye-tracking) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As mentioned yesterday in the meeting: only for on-screen eyetracking.
Makes the validation of those a bit more complicated (one extra If).
I wonder if we should not have a similar warning in the eyetracking section, or at least a link to the task page.
@rwblair can we get your input about allowing/validating optional columns to be present (when present) before mandatory columns? |
|
The number of calibrations corresponding to this run. | ||
type: integer | ||
minimum: 0 | ||
CalibrationPosition: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assumes the same position for all calibrations / validations
} | ||
``` | ||
|
||
Content of `sub-01_task-VisualSearch_events.json`: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this really the _events.json file or is it the _physioevents.json?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is correct, it annotates the events.tsv file, as opposed to the new _physioevents.json.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So basically in the _physioevents.json file I only have the column names plus their description? And do I need a _events.json file if I do not have an _events.tsv? Because Martins dataset on Open Neuro does not have it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, gotcha -- you've actually hit an edge case we do not currently cover:
- We say that some metadata belonging in the
_events.json
file becomes mandatory if eye tracking is present - However, BIDS allows
task-rest
to dismiss the events.tsv file. This BEP could make it mandatory for rest, even if theevents.tsv
file is just empty (only the header row).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or include it in the physioevents.json
? (if this does not disturb too much other stuff) Because that would be easy to include in the converter as we are already asking users to put some metadata manually in themetadata.yml
which is read in by our code. The current code actually does that because I understood it wrong, cf my initial question...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure that is a good idea. If I were to move the metadata from events.json, I would think it'd be better in the physio.json file rather than physioevents.json.
These metadata are about stimulus presentation, so they are likely relevant beyond eye tracking exclusively. Perhaps a reasonable alternative would be to update the specs so these metadata are encoded within stim.json files instead of the events.json files. Either way, they need to allow a json be defined without the corresponding .tsv[.gz] file, or allow empty tables (okay for .tsv but dangerous for .tsv.gz).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'm of course fine with the solution that fits best to BIDS in general but I agree that we should keep this information somehow.
allow a json be defined without the corresponding .tsv[.gz] file
would probably the best solution for this particular BEP as we agreed on .tsv.gz
```JSON | ||
{ | ||
"DeviceSerialNumber": "17535483", | ||
"Columns": ["timestamp", "x_coordinate", "y_coordinate", "pupil_size"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the src/schema/objects/columns.yaml and src/schema/rules/tabular_data/physio.yaml it says "et_timestamp", here it says "timestamp", which is correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both are correct, these are two different things:
-
src/schema/objects/columns.yaml
,src/schema/rules/tabular_data/physio.yaml
define aet_timestamp
column name. This is so to allow the timestamp be different from the already existingtimestamp
column. This identifier is internal to the schema, so it does not propagate into actual column names. -
the line 672 you commented on is just the necessary specification of the
"Columns"
metadata, the column name is arbitrary, and in this case it is named"timestamp"
but other names would be equally supported.
{ | ||
"SamplingFrequency": 100.0, | ||
"StartTime": -22.345, | ||
"Columns": ["cardiac", "timestamp"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe better if timestamp comes first, no?
selectors: | ||
- suffix == "physioevents" | ||
columns: | ||
onset: required |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need new definitions for onset and duration because now the one used here assumed that the unit is in seconds.
`sub-01_task-visualSearch_recording-eye1_physio.json` sidecar | ||
could read: | ||
|
||
```JSON |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
insert in example here
Discussed previously during some meetings with @oesteban @mszinte Note that if the source data for single run was acquired by turning the eyetracker only during trials instead of keeping the eyetracking recording for the whole duration of the run, this will lead to discontinuous timestamps. @julia-pfarr and I are encountering the issue in some of the datasets we are converting. The decision for now is to pad the output files with rows of NaNs for the missing time points. Wonder if this consequence of using physio data for this kind of eyetracking acquisition should be mentioned somewhere in the spec, or if this is more a converter implementation detail + best practice recommendations for data acquistion... |
Hi everyone,
Frustratingly, this is at least somewhat common in EyeLink eytrackers. Eyelink will also stop recording any time you enter a calibration sequence.
This is exactly what we did in the eyelink reader in MNE. I was also unsure if it was the right thing to do.. It makes me feel a little better about that decision, seeing you all arrive to the same conclusion independently 🙂
Just a heads up that our decision to pad with NaNs in MNE has caused its fair share of headaches, especially with signal processing routines (e.g. if you filter your pupil size signal, one of those NaN's could obliterate the whole signal!). |
fields: | ||
EnvironmentCoordinates: required | ||
RecordedEye: required | ||
SampleCoordinateUnits: required |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the unit may also described in the description of each column of the physio file so so this may conflict with also having it here, no?
Hi everyone thanks for the hard work for pushing this forward! I have a question/minor suggestion: when bids-specification/src/modality-specific-files/physiological-recordings.md Lines 670 to 691 in 26c463a
|
type: object | ||
items: | ||
type: object | ||
EyeTrackerDistance: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of EyeTrackerDistance
metadata object we can employ a much holistic EyetrackingGeometry
object that records the spatial description of eyetracking setup. This is necessary information for correcting pupil foreshortening error during analysis.
It could look something like
"EyetrackingGeometry" : {
"EyeToCameraX": 0.05,
"EyeToCameraY": 0.30,
"EyeToCameraZ": 0.40,
"EyeToScreenTopLeftX": 0.15,
"EyeToScreenTopLeftY": 0.1,
"EyeToScreenTopLeftZ": 0.50
}
where X, Y are the axes that run along the width and height of the screen, while Z is the axis along which the subject, the eyetracker and the screen are located.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would add these in addition to the simpler EyeTrackerDistance
, noting that it is recommended to have these better metadata values. I'd suggest also:
{
"EyeToCameraXYZ": [0.05, 0.30, 0.40],
"EyeToScreenTopLeftXYZ": [0.15, 0.1, 0.5],
}
for more information, see https://pre-commit.ci
src/schema/objects/metadata.yaml
Outdated
EncodingTechnique: | ||
name: EncodingTechnique | ||
display_name: Encoding Technique | ||
description: | | ||
The encoding technique used during readout. | ||
For example, `"Cartesian"`, `"EPSI"`, `"Spiral"`, | ||
or `"Density-weighted concentric ring trajectory"`. | ||
type: string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo due to merge conflict resolution
Here is the specifications of the BEP 020 about eye tracking.
Note
We meet regularly and everyone is welcome :
Next meeting October 3Rd 2024 4pm CET on zoom.
Note that if you consider joining but this time or day doesn't suits you, reach me (@mszinte) and I will arrange another appointment.
Notes of last meeting
Chat and discussions also happening on matrix
Tip
HTML preview of this BEP
Issues for: