Dandiset 000012 does not load properly with load_namespaces=True

[yarikoptic]

Might be just another reincarnation of #1335 but I have decided to file separately since error message is different

helper script

#!/usr/bin/env python3
import hdmf
import h5py
import pynwb
import sys

from pynwb import NWBHDF5IO

print(f"h5py: {h5py.__version__}; hdmf: {hdmf.__version__};  pynwb: {pynwb.__version__}")

for path in sys.argv[1:]:
    print(path)
    for kw in ({}, {'load_namespaces': True}):
        try:
            with NWBHDF5IO(path, "r", **kw) as io:
                nwb = io.read()
                print(f" {kw}: OK  {nwb}")
        except Exception as exc:
            print(f" {kw}: FAIL  {exc}")

produces

$> ~/proj/dandi/tools/pynwb-read.py /mnt/backup/dandi/dandisets/000012/sub-1/sub-1_ses-18516000_tis-1-1.nwb
h5py: 2.10.0; hdmf: 2.3.0;  pynwb: 1.4.0
/mnt/backup/dandi/dandisets/000012/sub-1/sub-1_ses-18516000_tis-1-1.nwb
/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/pynwb/icephys.py:309: UserWarning: Unit 'V' for  ZeroClampSeries 'index_0' is ignored and will be set to 'volts' as per NWB 2.1.0.
  warnings.warn(
 {}: FAIL  "'ndx-dandi-icephys' not a namespace"
/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/hdmf/spec/namespace.py:484: UserWarning: Ignoring cached namespace 'core' version 2.0.1 because version 2.2.5 is already loaded.
  warn("Ignoring cached namespace '%s' version %s because version %s is already loaded."
 {'load_namespaces': True}: FAIL  (root/general/DandiIcephysMetadata GroupBuilder {'attributes': {'namespace': 'ndx-dandi-icephys', 'neurodata_type': 'DandiIcephysMetadata', 'object_id': '0f50a70a-1e3b-42a5-bb94-1fc23f88c110', 'tissue_sample_id': '1-1'}, 'groups': {}, 'datasets': {}, 'links': {}}, "Could not construct DandiIcephysMetadata object due to: TypeMap.__get_cls_dict.<locals>.__init__: missing argument 'help'")

[t-b]

Does the file validate?
The error message "Could not construct DandiIcephysMetadata object due to: TypeMap.__get_cls_dict.<locals>.__init__: missing argument 'help'" suggests not.

[yarikoptic]

here is a clamp of nwb validation + then failure of us to load it:

*$> dandi validate /mnt/backup/dandi/dandisets/000012/sub-1/sub-1_ses-18516000_tis-1-1.nwb
/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/hdmf/spec/namespace.py:484: UserWarning: Ignoring cached namesp
ace 'core' version 2.0.1 because version 2.2.5 is already loaded.
  warn("Ignoring cached namespace '%s' version %s because version %s is already loaded."
/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/pynwb/icephys.py:309: UserWarning: Unit 'V' for IZeroClampSerie
s 'index_0' is ignored and will be set to 'volts' as per NWB 2.1.0.
  warnings.warn(
/mnt/backup/dandi/dandisets/000012/sub-1/sub-1_ses-18516000_tis-1-1.nwb: 6 error(s)
  VectorIndex/description (general/intracellular_ephys/sweep_table/series_index): argument missing
  VectorIndex (general/intracellular_ephys/sweep_table/series_index): incorrect type - expected 'uint8', got 'int64'
  VectorIndex/description (general/intracellular_ephys/sweep_table/series_index): argument missing
  VectorIndex (general/intracellular_ephys/sweep_table/series_index): incorrect type - expected 'uint8', got 'int64'
  warning: nwb_version '2.0b' is not a proper semantic version. See http://semver.org
  Failed to read metadata: (root/general/DandiIcephysMetadata GroupBuilder {'attributes': {'namespace': 'ndx-dandi-icephys', 'neu
rodata_type': 'DandiIcephysMetadata', 'object_id': '0f50a70a-1e3b-42a5-bb94-1fc23f88c110', 'tissue_sample_id': '1-1'}, 'groups': 
{}, 'datasets': {}, 'links': {}}, "Could not construct DandiIcephysMetadata object due to: TypeMap.__get_cls_dict.<locals>.__init
__: missing argument 'help'")
Summary: Validation errors in 1 out of 1 files

FWIW: file(s) could be downloaded from https://gui.dandiarchive.org/#/dandiset/000012/draft/files?_id=5e739c1d368d3c79a80064a2&_modelType=folder or with datalad from https://github.com/dandisets/000012

[oruebel]

@yarikoptic can this issue be closed?

[yarikoptic]

I don't know -- could you load that file now?

well, I have tried now, had to recover that file from backup since it was replaced with a "fixed up" version. Both are opening ok now with the script I provided and h5py: 3.6.0; hdmf: 3.1.1; pynwb: 2.0.0 even though validation still fails.

We also just got reported similar validation errors from another user in dandi/helpdesk#77 , inquiring how those files were produced.