All motion params

[tergeorge]

Fixes

Fixes #1576 by @sgiavasis

Description

The pull request is used to condense the motion parameters into a two .tsv files.

Technical details

changes made to func_preproc.py and generate_motion_statistics.py.
All .1D files mentioned below combined into 1 .tsv file:
sub-_ses-_task-rest_run-_framewise-displacement-jenkinson.1D
sub-_ses-_task-rest_run-_framewise-displacement-power.1D
sub-_ses-_task-rest_run-_max-displacement.1D
sub-_ses-_task-rest_run-_rels-displacement.1D
sub-_ses-_task-rest_run-*_movement-parameters.1D

All 2D matrix values mentioned below combined into 2nd .tsv file:
sub-_ses-_task-rest_run-_motion-params.txt
sub-_ses-_task-rest_run-_power-params.txt

The new files generated are motion.tsv and summary-motion.tsv.

Tests

Run the PR with any BIDS dataset and the two .tsv files should be generated in the working and output directory.

Checklist

• My pull request has a descriptive title (not a vague title like Update index.md).
• My pull request targets the develop branch of the repository.
• My commit messages follow best practices.
• My code follows the established code style of the repository.
• I added tests for the changes I made (if applicable).
• I updated the changelog.
• I added or updated documentation (if applicable).
• I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

[sgiavasis]

There seem to be four files that might have been accidentally created - viewing the Files Changed will show 4 empty files, with names of our usual files but in the wrong places:

• CPAC/func_preproc.py
• CPAC/generate_motion_statistics.py
• func_preproc.py
• generate_motion_statistics.py

You'll need to remove these before merging.

[tergeorge]

Deleted the empty files and tested and updated the PR.

changes made

[shnizzedy]

It would be good to condense all of this data into 3 files. There are three kinds:

One row of numbers with a header denoting each one. We can merge these into one file and change it to a .tsv:

sub-*_ses-*_task-rest_run-*_motion-params.txt
sub-*_ses-*_task-rest_run-*_power-params.txt

A 1D file with a single vector, with a row/value per time point from the time series. Again, merge these files, but also create a new header describing each vector, and make it a .tsv file:

sub-*_ses-*_task-rest_run-*_dvars.1D
sub-*_ses-*_task-rest_run-*_framewise-displacement-jenkinson.1D
sub-*_ses-*_task-rest_run-*_framewise-displacement-power.1D
sub-*_ses-*_task-rest_run-*_max-displacement.1D
sub-*_ses-*_task-rest_run-*_rels-displacement.1D

The movement-parameters.1D output, which is often a 6xN matrix, with N = timepoints. This file is used widely downstream as its own output, in the format it is written in, so this can remain the same, although we can also convert it to a .tsv file:

sub-*_ses-*_task-rest_run-*_movement-parameters.1D

― #1576 (comment)

changes made to func_preproc.py and generate_motion_statistics.py. All .1D files mentioned below combined into 1 .tsv file:
sub-*_ses-*_task-rest_run-*_framewise-displacement-jenkinson.1D
sub-*_ses-*_task-rest_run-*_framewise-displacement-power.1D
sub-*_ses-*_task-rest_run-*_max-displacement.1D
sub-*_ses-*_task-rest_run-*_rels-displacement.1D
[sub-*_ses-*_task-rest_run-*_movement-parameters.1D]

All 2D matrix values mentioned below combined into 2nd .tsv file:
sub-*_ses-*_task-rest_run-*_motion-params.txt
sub-*_ses-*_task-rest_run-*_power-params.txt

The new files generated are [motion.tsv] and [desc-summary_motion.tsv].

― #1624 (comment)

Flips the priority in the logic from filtered-movement-parameters being the special case to unfiltered-movement-parameters being the special case (with movement-parameters being the other, default case both before and after)

― #1989 (comment)

Currently testing that outputs are

• motion.tsv
• desc-summary_motion.tsv
• movement-parameters.tsv
  and potentially
• unfiltered-movement-parameters.tsv
  (will update the latter to become desc-movementParameters_motion.tsv and desc-unfilteredMovementParameters_motion.tsv)

Note: we'll probably want to rename these again soon since BIDS now uses motion as the suffix for motion capture data

[sgiavasis]

Thank you!

Note: we'll probably want to rename these again soon since BIDS now uses motion as the suffix for motion capture data

Do we want to just do this now? Is there a defined proper name to use in the spec other than motion?

[shnizzedy]

I asked in bids-standard/bids-specification#519 (comment)

[shnizzedy]

Do we want to just do this now? Is there a defined proper name to use in the spec other than motion?

Looks like _motion is still the suffix to use, at least for now.

[shnizzedy]

7b33c49 changes the format of desc-summary_motion.tsv from

Parameters	Values
Mean_Relative_RMS_Displacement	0.14145283984038426
Max_Relative_RMS_Displacement	1.389282568761795
Movements_gt_threshold	212.0
Mean_Relative_Mean_Rotation	0.017968538324420677
Mean_Relative_Maxdisp	0.19005614973262028
Max_Relative_Maxdisp	3.832
Max_Abs_Maxdisp	4.367
Max Relative_Roll	0.06999999999999999
Max_Relative_Pitch	1.5071999999999999
Max_Relative_Yaw	0.3244
Max_Relative_dS-I	1.4773
Max_Relative_dL-R	0.05430000000000001
Max_Relative_dP-A	0.2651
Mean_Relative_Roll	0.008331283422459894
Mean_Relative_Pitch	0.04126631016042781
Mean_Relative_Yaw	0.026191978609625667
Mean_Relative_dS-I	0.21911791443850265
Mean_Relative_dL-R	0.009462299465240643
Mean_Relative_dP-A	0.10449518716577538
Max_Abs_Roll	0.0924
Max_Abs_Pitch	1.7731
Max_Abs_Yaw	0.3401
Max_Abs_dS-I	1.581
Max_Abs_dL-R	0.102
Max_Abs_dP-A	0.2047
Mean_Abs_Roll	0.04281546666666667
Mean_Abs_Pitch	0.09135386666666667
Mean_Abs_Yaw	0.0942888
Mean_Abs_dS-I	0.18470373333333334
Mean_Abs_dL-R	0.0356336
Mean_Abs_dP-A	0.06858533333333333
MeanFD_Power	0.3981497083287398
MeanFD_Jenkinson	0.26231529997333336
rootMeanSquareFD	0.5121672578107013
FDquartile(top1/4thFD)	0.46138294591397855
MeanDVARS	84.98434029333335

(36 rows + a header × 2 columns)

to

Mean_Relative_RMS_Displacement	Max_Relative_RMS_Displacement	Movements_gt_threshold	Mean_Relative_Mean_Rotation	Mean_Relative_Maxdisp	Max_Relative_Maxdisp	Max_Abs_Maxdisp	Max Relative_Roll	Max_Relative_Pitch	Max_Relative_Yaw	Max_Relative_dS-I	Max_Relative_dL-R	Max_Relative_dP-A	Mean_Relative_Roll	Mean_Relative_Pitch	Mean_Relative_Yaw	Mean_Relative_dS-I	Mean_Relative_dL-R	Mean_Relative_dP-A	Max_Abs_Roll	Max_Abs_Pitch	Max_Abs_Yaw	Max_Abs_dS-I	Max_Abs_dL-R	Max_Abs_dP-A	Mean_Abs_Roll	Mean_Abs_Pitch	Mean_Abs_Yaw	Mean_Abs_dS-I	Mean_Abs_dL-R	Mean_Abs_dP-A	MeanFD_Power	MeanFD_Jenkinson	rootMeanSquareFD	FDquartile(top1/4thFD)	MeanDVARS
0.14145283984038426	1.389282568761795	212.0	0.017968538324420677	0.19005614973262028	3.832	4.367	0.06999999999999999	1.5071999999999999	0.3244	1.4773	0.05430000000000001	0.2651	0.008331283422459894	0.04126631016042781	0.026191978609625667	0.21911791443850265	0.009462299465240643	0.10449518716577538	0.0924	1.7731	0.3401	1.581	0.102	0.2047	0.04281546666666667	0.09135386666666667	0.0942888	0.18470373333333334	0.0356336	0.06858533333333333	0.3981497083287398	0.26231529997333336	0.5121672578107013	0.46138294591397855	84.98434029333335

(1 row + a header × 36 columns)

[sgiavasis]

Amazing! Very happy to merge this in after almost 2 years 😆