CTF data loading with non-zero compensation grade error

[Yaocoo]

I was trying to load CTF data. I only put the .ds dir as the first parameter, all other parameters are default value. Then meet a runtime error that "RuntimeError: Reference magnetometer channel with non-zero compensation grade 1".

the code and the results are in the below

Example:

%matplotlib tk
import mne
import matplotlib.pyplot as plt
import os.path as op
import os
import glob
import numpy as np

sub = "A1922"
data_path = op.join('/Users/yaocongd/local/Data/MEEG_2018/',sub);
dir_list = os.listdir(data_path)
mne.io.read_raw_ctf(op.join(data_path,dir_list[2]), system_clock='truncate', preload=True, clean_names=False, verbose=None)

Actual results

ds directory : /Users/yaocongd/local/Data/MEEG_2018/A1922/A1922_GlasgowFace_20190619_05.ds
    res4 data read.
    hc data read.
    Separate EEG position data file not present.
    Quaternion matching (desired vs. transformed):
       4.47   82.95    0.00 mm <->    4.47   82.95    0.00 mm (orig :  -67.80   49.75 -257.65 mm) diff =    0.000 mm
      -4.47  -82.95    0.00 mm <->   -4.47  -82.95    0.00 mm (orig :   44.38  -72.48 -266.51 mm) diff =    0.000 mm
     104.26    0.00    0.00 mm <->  104.26    0.00    0.00 mm (orig :   60.64   61.29 -243.12 mm) diff =    0.000 mm
    Coordinate transformations established.
    Polhemus data for 3 HPI coils added
    Device coordinate locations for 3 HPI coils added
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-015f3654086c> in <module>
      2 data_path = op.join('/Users/yaocongd/local/Data/MEEG_2018/',sub);
      3 dir_list = os.listdir(data_path)
----> 4 mne.io.read_raw_ctf(op.join(data_path,dir_list[2]), system_clock='truncate', preload=True, clean_names=False, verbose=None)

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/io/ctf/ctf.py in read_raw_ctf(directory, system_clock, preload, clean_names, verbose)
     62     """
     63     return RawCTF(directory, system_clock, preload=preload,
---> 64                   clean_names=clean_names, verbose=verbose)
     65 
     66 

</Users/yaocongd/anaconda3/envs/mne/lib/python3.6/site-packages/mne/externals/decorator.py:decorator-gen-150> in __init__(self, directory, system_clock, preload, verbose, clean_names)

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/utils/_logging.py in wrapper(*args, **kwargs)
     87             with use_log_level(verbose_level):
     88                 return function(*args, **kwargs)
---> 89         return function(*args, **kwargs)
     90     return FunctionMaker.create(
     91         function, 'return decfunc(%(signature)s)',

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/io/ctf/ctf.py in __init__(self, directory, system_clock, preload, verbose, clean_names)
    115 
    116         # Compose a structure which makes fiff writing a piece of cake
--> 117         info = _compose_meas_info(res4, coils, coord_trans, eeg)
    118         info['dig'] += digs
    119         info['bads'] += _read_bad_chans(directory, info)

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/io/ctf/info.py in _compose_meas_info(res4, coils, trans, eeg)
    442         if trans['t_ctf_head_head'] is not None:
    443             info['ctf_head_t'] = trans['t_ctf_head_head']
--> 444     info['chs'] = _convert_channel_info(res4, trans, eeg is None)
    445     info['comps'] = _convert_comp_data(res4)
    446     if eeg is None:

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/io/ctf/info.py in _convert_channel_info(res4, t, use_eeg_pos)
    219             if cch['sensor_type_index'] == CTF.CTFV_REF_MAG_CH:
    220                 ch['kind'] = FIFF.FIFFV_REF_MEG_CH
--> 221                 _check_comp_ch(cch, 'Reference magnetometer')
    222                 ch['coil_type'] = FIFF.FIFFV_COIL_CTF_REF_MAG
    223                 nref += 1

~/anaconda3/envs/mne/lib/python3.6/site-packages/mne/io/ctf/info.py in _check_comp_ch(cch, kind, desired)
    138         if cch['grad_order_no'] != 0:
    139             raise RuntimeError('%s channel with non-zero compensation grade %s'
--> 140                                % (kind, cch['grad_order_no']))
    141     else:
    142         if desired is None:

RuntimeError: Reference magnetometer channel with non-zero compensation grade 1

[larsoner]

Can you upload this dataset somewhere? It's weird that there is a reference magnetometer channel that has had gradient compensation applied to it.

Do you know where this dataset was acquired, and with which CTF acquisition software version?

[dengemann]

Eric this issue comes from one of our attendees in Gasgow. We can try to take this offline if data sharing would turn out to be difficult. Perhaps we should also try reading it with MNE-C.

[agramfort]

+1 to try with mne-c

…

[Yaocoo]

Thank you all very much. This data has both MEG and EEG recorded, I guess the problem is might because the EEG channel. Let me check the validation of data with fieldtrip first tomorrow.

[paulineduret]

Hi! I got the same error with a mixed EEG/MEG dataset. Maybe the CTF system uses the eeg as a ref and thus all magnetometers are compensated, even the REF ones.

[agramfort]

@bloyl as our CTF expert do you have any thought on this? do we some tiny dataset to replicate this?

[bloyl]

I agree that it seems odd that a ref magnetometer would have gradient compensation applied.
I've never seen this issue on our MEG only data and unfortunately I don't have any datasets that have both MEG and EEG so I can't replicate the issue.

Is there a dataset available?

[Nichalas]

Hi! I have the same error with a dataset containing only MEG data.
Here you can find the dataset:
https://drive.google.com/file/d/1s3Ofe4RCkV5xp-UG-EdNVLtp3c6o8z6h/view?usp=sharing

[agramfort]

I can replicate the problem too. I have no clue why we impose a compensation grade of 0 for reference channels.
in the shared dataset there are 27 reference channels with a compensation grade of 1 and makes our reader crash.

@bloyl or @larsoner any idea?

[bloyl]

I didn't think the reference channels changed when you applied gradient compensation. So I'm not sure how/why the reference channels get marked as having compensation applied. Our data that is acquired with 3rd order compensation turned on doesn't have this problem. Maybe if I turn it on/off on processing I could get a dataset in this state. Nichalas, is this raw ctf data or has it been processed at all? If so what software did you use to do the processing. I'll try this week to open this dataset using ctf's software and see what happens.

…

On Sun, Jul 5, 2020, 10:26 AM Alexandre Gramfort ***@***.***> wrote: I can replicate the problem too. I have no clue why we impose a compensation grade of 0 for reference channels. in the shared dataset there are 27 reference channels with a compensation grade of 1 and makes our reader crash. @bloyl <https://github.com/bloyl> or @larsoner <https://github.com/larsoner> any idea? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6488 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKTXHL5M4XGWYSOD3JJXITR2CEP5ANCNFSM4H3JZ4SA> .

[Nichalas]

We tried to apply an offline 3rd order compensation and the error remained.
Then, we did a test measurement with 1st and 3rd order (data attached) and the 3rd order dataset could be successfully imported.

dataset: https://drive.google.com/file/d/16yZ0fdT24i3lK3D6jvgpn70VWBmZXFG1/view?usp=sharing

[agramfort]

so you have a pb with data recorded using 1st order without any processing applied. If so we should fix this. Can you read the files easily with Brainstorm or Fieldtrip?

[Nichalas]

Yes, I think this is the case.
I didn't have any issue loading the data with Fieldtrip (haven't tried with Brainstorm).

[larsoner]

I added this back in #3440. Perhaps we can try disabling the check and check to see if forwards computed in both cases match...?