set up file logger with timestamp

[pierre-nedelec]

Fixes part of #914 by adding the ability to log to file and have timestamps.
Next potential steps:

• option to have logs within BIDS structure?
• it's not easy to fully understand what stage of the pyAFQ pipeline we're at just reading the current logs

[pierre-nedelec]

unless really necessary, it'd be better not to redefine the logger level in a file like that

[36000]

This is an example, so its showing what the user can do. So I think it's good to show the getlogger and setlevel stuff. This file is not imported by any other file so it shouldn't affect anything else. That said I am not sure why there is this root logger thing.

[pierre-nedelec]

Example log file generated with current config:

2022-10-26 17:29:06,220 - root:INFO:Starting pyAFQ
2022-10-26 17:29:06,237 - root:INFO:Starting to create the GroupAFQ object
2022-10-26 17:32:35,669 - AFQ.api.group:WARNING:No dwi found for subject SUBJ1 and session 2YearFollowUpYArm1. Skipping.
2022-10-26 17:33:02,578 - AFQ.tasks.segmentation:INFO:Saving AFQ_test/derivatives/det/sub-SUBJ1/ses-baselineYear1Arm1/clean_bundles/sub-SUBJ1_ses-baselineYear1Arm1_run-01_dwi_space-RASMM_model-detDKI_algo-AFQ_desc-ATRL_tractography.trk
2022-10-26 17:33:02,618 - AFQ.tasks.segmentation:INFO:Saving AFQ_test/derivatives/det/sub-SUBJ1/ses-baselineYear1Arm1/clean_bundles/sub-SUBJ1_ses-baselineYear1Arm1_run-01_dwi_space-RASMM_model-detDKI_algo-AFQ_desc-ATRR_tractography.trk
...
2022-10-26 17:41:36,192 - AFQ.api.group:WARNING:To make plots in pyAFQ, you will need to install the relevant plotting software packages.You can do that by installing pyAFQ with `pip install AFQ[plot]`
2022-10-26 17:41:48,839 - AFQ.viz.utils:INFO:Loading Volume...
2022-10-26 17:41:50,192 - AFQ.viz.utils:INFO:Loading Volume...
2022-10-26 17:41:50,198 - AFQ.viz.utils:INFO:Loading Stateful Tractogram...
2022-10-26 17:41:50,662 - AFQ.viz.utils:INFO:Generating colorful lines from tractography...
2022-10-26 17:41:55,592 - AFQ.viz.utils:INFO:Loading Volume...
2022-10-26 17:41:55,646 - AFQ.viz.utils:INFO:Loading Volume...
2022-10-26 17:41:55,650 - AFQ.viz.utils:INFO:Loading Stateful Tractogram...
2022-10-26 17:41:56,060 - AFQ.viz.utils:INFO:Generating colorful lines from tractography...
2022-10-26 17:41:58,324 - AFQ.tasks.viz:INFO:Generating ATR_L visualization...
...
2022-10-26 17:48:27,432 - AFQ.api.group:INFO:Time taken for export all: 947.104297876358

I think we should add subject/session for each entry. Also currently, if the pipeline works on a few subjects but not all in the GroupAFQ, it will still display that last message, but there's no way to tell from the log that it didn't complete for others.

[36000]

I think this PR looks good to go as is, but there are a few changes we can do here or in another PR.

We can remove these two lines from the example:

root = logging.getLogger()
root.setLevel(logging.ERROR)

For the idea of printing subject/session for each log output, I think that is good and could be handled in a separate PR or posted as an issue. We could add a handler to the logger before running for a given subject, then remove it afterwards. Here is an example of adding a prefix to the logger object: https://stackoverflow.com/questions/28994448/how-to-add-a-prefix-to-an-existing-python-logging-formatter . I think we could do it in GroupAFQ's export function, wrapping this line:

pyAFQ/AFQ/api/group.py

Line 431 in a44d570

results[subject][session] = wf_dict[attr_name]

(not sure how to handle the parallel case)

For some subjects not completing with export_all, I am not sure what you mean. If any subject has a fatal error the whole pipeline should stop.

@pierre-nedelec if you rebase this PR on master we can merge this PR.

[arokem]

@pierre-nedelec : if you rebase this, we can merge this to include in the upcoming release.

[arokem]

Pushing this to the next 1.2 release (which hopefully will be soon too!)