Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How should I make config file to train with my customized dataset? #70

Open
HitmansGO opened this issue Jul 28, 2024 · 8 comments
Open

Comments

@HitmansGO
Copy link

I had already prepared my customized dataset with S3DIS format. And I also referred to mmdection3d's config file https://mmdetection3d.readthedocs.io/en/latest/user_guides/new_data_model.html
but when I run train.py with my own config file, I got this erro:

Traceback (most recent call last):
  File "/home/hfut108/oneformer3d/tools/train.py", line 135, in <module>
    main()
  File "/home/hfut108/oneformer3d/tools/train.py", line 131, in main
    runner.train()
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1777, in train
    model = self.train_loop.run()  # type: ignore
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/mmengine/runner/loops.py", line 96, in run
    self.run_epoch()
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/mmengine/runner/loops.py", line 112, in run_epoch
    for idx, data_batch in enumerate(self.dataloader):
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 681, in __next__
    data = self._next_data()
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
    return self._process_data(data)
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
    data.reraise()
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
Exception: Caught Exception in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/mmengine/dataset/dataset_wrapper.py", line 171, in __getitem__
    return self.datasets[dataset_idx][sample_idx]
  File "/home/hfut108/anaconda3/envs/openmmlab3d/lib/python3.10/site-packages/mmengine/dataset/base_dataset.py", line 418, in __getitem__
    raise Exception(f'Cannot find valid image after {self.max_refetch}! '
Exception: Cannot find valid image after 1000! Please check your image path and pipeline

and here is my config file:

import sys
sys.path.append("/home/hfut108")

_base_ = [
    'mmdet3d::_base_/default_runtime.py',
]
custom_imports = dict(imports=['oneformer3d.oneformer3d.oneformer3d'])

# model settings
num_channels = 64
num_instance_classes = 2
num_semantic_classes = 2
class_names = ['part','bgpart']
metainfo = dict(classes=class_names)
num_points = 4096

model = dict(
    type='S3DISOneFormer3D',
    data_preprocessor=dict(type='Det3DDataPreprocessor'),
    in_channels=6,
    num_channels=num_channels,
    voxel_size=0.05,
    num_classes=num_instance_classes,
    min_spatial_shape=128,
    backbone=dict(
        type='SpConvUNet',
        num_planes=[num_channels * (i + 1) for i in range(5)],
        return_blocks=True),
    decoder=dict(
        type='QueryDecoder',
        num_layers=3,
        num_classes=num_instance_classes,
        num_instance_queries=400,
        num_semantic_queries=num_semantic_classes,
        num_instance_classes=num_instance_classes,
        in_channels=num_channels,
        d_model=256,
        num_heads=8,
        hidden_dim=1024,
        dropout=0.0,
        activation_fn='gelu',
        iter_pred=True,
        attn_mask=True,
        fix_attention=True,
        objectness_flag=True),
    criterion=dict(
        type='S3DISUnifiedCriterion',
        num_semantic_classes=num_semantic_classes,
        sem_criterion=dict(
            type='S3DISSemanticCriterion',
            loss_weight=5.0),
        inst_criterion=dict(
            type='InstanceCriterion',
            matcher=dict(
                type='HungarianMatcher',
                costs=[
                    dict(type='QueryClassificationCost', weight=0.5),
                    dict(type='MaskBCECost', weight=1.0),
                    dict(type='MaskDiceCost', weight=1.0)]),
            loss_weight=[0.5, 1.0, 1.0, 0.5],
            num_classes=num_instance_classes,
            non_object_weight=0.05,
            fix_dice_loss_weight=True,
            iter_matcher=True,
            fix_mean_loss=True)),
    train_cfg=dict(),
    test_cfg=dict(
        topk_insts=450,
        inst_score_thr=0.0,
        pan_score_thr=0.4,
        npoint_thr=300,
        obj_normalization=True,
        obj_normalization_thr=0.01,
        sp_score_thr=0.15,
        nms=True,
        matrix_nms_kernel='linear',
        num_sem_cls=num_semantic_classes,
        stuff_cls=[1],
        thing_cls=[0]))

# dataset settings
dataset_type = 'S3DISSegDataset_'
data_root = 'data/s3dis/'
data_prefix = dict(
    pts='points',
    pts_instance_mask='instance_mask',
    pts_semantic_mask='semantic_mask')

train_area = [1, 2, 3, 4, 6]
test_area = 5

train_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='DEPTH',
        shift_height=False,
        use_color=True,
        load_dim=6,
        use_dim=[0, 1, 2, 3, 4, 5]),
    dict(
        type='LoadAnnotations3D',
        with_label_3d=False,
        with_bbox_3d=False,
        with_mask_3d=True,
        with_seg_3d=True),
    dict(
        type='PointSample_',
        num_points=num_points),
    dict(type='PointInstClassMapping_',
        num_classes=num_instance_classes),
    dict(
        type='RandomFlip3D',
        sync_2d=False,
        flip_ratio_bev_horizontal=0.5,
        flip_ratio_bev_vertical=0.5),
    dict(
        type='GlobalRotScaleTrans',
        rot_range=[0.0, 0.0],
        scale_ratio_range=[0.9, 1.1],
        translation_std=[.1, .1, .1],
        shift_height=False),
    dict(
        type='NormalizePointsColor_',
        color_mean=[127.5, 127.5, 127.5]),
    dict(
        type='Pack3DDetInputs_',
        keys=[
            'points', 'gt_labels_3d',
            'pts_semantic_mask', 'pts_instance_mask'
        ])
]
test_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='DEPTH',
        shift_height=False,
        use_color=True,
        load_dim=6,
        use_dim=[0, 1, 2, 3, 4, 5]),
    dict(
        type='LoadAnnotations3D',
        with_bbox_3d=False,
        with_label_3d=False,
        with_mask_3d=True,
        with_seg_3d=True),
    dict(
        type='MultiScaleFlipAug3D',
        img_scale=(1333, 800),
        pts_scale_ratio=1,
        flip=False,
        transforms=[
            dict(
                type='NormalizePointsColor_',
                color_mean=[127.5, 127.5, 127.5])]),
    dict(type='Pack3DDetInputs_', keys=['points'])
]

# run settings
train_dataloader = dict(
    batch_size=2,
    num_workers=3,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=True),
    dataset=dict(
            type='ConcatDataset',
            datasets=([
                dict(
                    type=dataset_type,
                    data_root=data_root,
                    ann_file=f's3dis_infos_Area_{i}.pkl',
                    pipeline=train_pipeline,
                    metainfo=metainfo,
                    filter_empty_gt=True,
                    data_prefix=data_prefix,
                    box_type_3d='Depth',
                    backend_args=None) for i in train_area])))

val_dataloader = dict(
    batch_size=1,
    num_workers=1,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file=f's3dis_infos_Area_{test_area}.pkl',
        pipeline=test_pipeline,
        metainfo=metainfo,
        test_mode=True,
        data_prefix=data_prefix,
        box_type_3d='Depth',
        backend_args=None))
test_dataloader = val_dataloader


label2cat = {i: name for i, name in enumerate(class_names)}
metric_meta = dict(
    label2cat=label2cat,
    ignore_index=[num_semantic_classes],
    classes=class_names,
    dataset_name='S3DIS')
sem_mapping = [0, 1]

val_evaluator = dict(
    type='UnifiedSegMetric',
    stuff_class_inds=[1],
    thing_class_inds=[0],
    min_num_points=1,
    id_offset=2**16,
    sem_mapping=sem_mapping,
    inst_mapping=sem_mapping,
    submission_prefix_semantic=None,
    submission_prefix_instance=None,
    metric_meta=metric_meta)
test_evaluator = val_evaluator

optim_wrapper = dict(
    type='OptimWrapper',
    optimizer=dict(type='AdamW', lr=0.0001, weight_decay=0.05),
    clip_grad=dict(max_norm=10, norm_type=2))
param_scheduler = dict(type='PolyLR', begin=0, end=512, power=0.9)

custom_hooks = [dict(type='EmptyCacheHook', after_iter=True)]
default_hooks = dict(
    checkpoint=dict(
        interval=16,
        max_keep_ckpts=1,
        save_best=['all_ap_50%', 'miou'],
        rule='greater'))

load_from = 'work_dirs/tmp/instance-only-oneformer3d_1xb2_scannet-and-structured3d.pth'

train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=512, val_interval=16)
val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')

I don't know how to fix this erro, so I really need some help.
Expecting for the solution, thx!

@oneformer3d-contributor
Copy link
Collaborator

Please debug smth like for b in YourDataset: print b. Because now you have a problem with dataset. It tries to get i-th element 1000 times, all of the attempts fail (probably bug with path to point clouds or to annotation), so you get this error.

@zhaohongxiang-seu
Copy link

@HitmansGO I encountered the same issue, have you resolved it?

@ustbwang1jie
Copy link

@HitmansGO have you resolved it?

@Lizhinwafu
Copy link

[> @HitmansGO have you resolved it?

](#9 (comment))

@clawCa
Copy link

clawCa commented Oct 7, 2024

Encountered the same problem. It seems like there was an error in the dataset preprocessing. Did you use the conversion script from mmdetection3d for the S3DIS dataset? In that script, you would need to modify instance categories such as sofa and table to your custom categories.

@accoumar12
Copy link

I had the same issue. And solved it by taking care of the good format of the input data.

@xiaoxiae
Copy link

xiaoxiae commented Dec 3, 2024

In my case, these are the changes I had to make in mmdetection3d and the oneformer3d directories, respectively:

mmdetection3d (only used when generating the dataset)

diff --git a/data/s3dis/collect_indoor3d_data.py b/data/s3dis/collect_indoor3d_data.py
index 13f5dc67..c7c1050e 100644
--- a/data/s3dis/collect_indoor3d_data.py
+++ b/data/s3dis/collect_indoor3d_data.py
@@ -25,18 +25,6 @@ anno_paths = [osp.join(args.data_dir, p) for p in anno_paths]
 output_folder = args.output_folder
 mmengine.mkdir_or_exist(output_folder)
 
-# Note: there is an extra character in the v1.2 data in Area_5/hallway_6.
-# It's fixed manually here.
-# Refer to https://github.com/AnTao97/dgcnn.pytorch/blob/843abe82dd731eb51a4b3f70632c2ed3c60560e9/prepare_data/collect_indoor3d_data.py#L18  # noqa
-revise_file = osp.join(args.data_dir,
-                       'Area_5/hallway_6/Annotations/ceiling_1.txt')
-with open(revise_file, 'r') as f:
-    data = f.read()
-    # replace that extra character with blank space to separate data
-    data = data[:5545347] + ' ' + data[5545348:]
-with open(revise_file, 'w') as f:
-    f.write(data)
-
 for anno_path in anno_paths:
     print(f'Exporting data from annotation file: {anno_path}')
     elements = anno_path.split('/')
diff --git a/data/s3dis/meta_data/anno_paths.txt b/data/s3dis/meta_data/anno_paths.txt
index 0ad2f259..bd89d0b8 100644
--- a/data/s3dis/meta_data/anno_paths.txt
+++ b/data/s3dis/meta_data/anno_paths.txt
@@ -1,272 +1,3 @@
-Area_1/conferenceRoom_1/Annotations
-Area_1/conferenceRoom_2/Annotations
-Area_1/copyRoom_1/Annotations
-Area_1/hallway_1/Annotations
-Area_1/hallway_2/Annotations
-Area_1/hallway_3/Annotations
-Area_1/hallway_4/Annotations
-Area_1/hallway_5/Annotations
-Area_1/hallway_6/Annotations
-Area_1/hallway_7/Annotations
-Area_1/hallway_8/Annotations
-Area_1/office_10/Annotations
-Area_1/office_11/Annotations
-Area_1/office_12/Annotations
-Area_1/office_13/Annotations
-Area_1/office_14/Annotations
-Area_1/office_15/Annotations
-Area_1/office_16/Annotations
-Area_1/office_17/Annotations
-Area_1/office_18/Annotations
-Area_1/office_19/Annotations
-Area_1/office_1/Annotations
-Area_1/office_20/Annotations
-Area_1/office_21/Annotations
-Area_1/office_22/Annotations
-Area_1/office_23/Annotations
-Area_1/office_24/Annotations
-Area_1/office_25/Annotations
-Area_1/office_26/Annotations
-Area_1/office_27/Annotations
-Area_1/office_28/Annotations
-Area_1/office_29/Annotations
-Area_1/office_2/Annotations
-Area_1/office_30/Annotations
-Area_1/office_31/Annotations
-Area_1/office_3/Annotations
-Area_1/office_4/Annotations
-Area_1/office_5/Annotations
-Area_1/office_6/Annotations
-Area_1/office_7/Annotations
-Area_1/office_8/Annotations
-Area_1/office_9/Annotations
-Area_1/pantry_1/Annotations
-Area_1/WC_1/Annotations
-Area_2/auditorium_1/Annotations
-Area_2/auditorium_2/Annotations
-Area_2/conferenceRoom_1/Annotations
-Area_2/hallway_10/Annotations
-Area_2/hallway_11/Annotations
-Area_2/hallway_12/Annotations
-Area_2/hallway_1/Annotations
-Area_2/hallway_2/Annotations
-Area_2/hallway_3/Annotations
-Area_2/hallway_4/Annotations
-Area_2/hallway_5/Annotations
-Area_2/hallway_6/Annotations
-Area_2/hallway_7/Annotations
-Area_2/hallway_8/Annotations
-Area_2/hallway_9/Annotations
-Area_2/office_10/Annotations
-Area_2/office_11/Annotations
-Area_2/office_12/Annotations
-Area_2/office_13/Annotations
-Area_2/office_14/Annotations
-Area_2/office_1/Annotations
-Area_2/office_2/Annotations
-Area_2/office_3/Annotations
-Area_2/office_4/Annotations
-Area_2/office_5/Annotations
-Area_2/office_6/Annotations
-Area_2/office_7/Annotations
-Area_2/office_8/Annotations
-Area_2/office_9/Annotations
-Area_2/storage_1/Annotations
-Area_2/storage_2/Annotations
-Area_2/storage_3/Annotations
-Area_2/storage_4/Annotations
-Area_2/storage_5/Annotations
-Area_2/storage_6/Annotations
-Area_2/storage_7/Annotations
-Area_2/storage_8/Annotations
-Area_2/storage_9/Annotations
-Area_2/WC_1/Annotations
-Area_2/WC_2/Annotations
-Area_3/conferenceRoom_1/Annotations
-Area_3/hallway_1/Annotations
-Area_3/hallway_2/Annotations
-Area_3/hallway_3/Annotations
-Area_3/hallway_4/Annotations
-Area_3/hallway_5/Annotations
-Area_3/hallway_6/Annotations
-Area_3/lounge_1/Annotations
-Area_3/lounge_2/Annotations
-Area_3/office_10/Annotations
-Area_3/office_1/Annotations
-Area_3/office_2/Annotations
-Area_3/office_3/Annotations
-Area_3/office_4/Annotations
-Area_3/office_5/Annotations
-Area_3/office_6/Annotations
-Area_3/office_7/Annotations
-Area_3/office_8/Annotations
-Area_3/office_9/Annotations
-Area_3/storage_1/Annotations
-Area_3/storage_2/Annotations
-Area_3/WC_1/Annotations
-Area_3/WC_2/Annotations
-Area_4/conferenceRoom_1/Annotations
-Area_4/conferenceRoom_2/Annotations
-Area_4/conferenceRoom_3/Annotations
-Area_4/hallway_10/Annotations
-Area_4/hallway_11/Annotations
-Area_4/hallway_12/Annotations
-Area_4/hallway_13/Annotations
-Area_4/hallway_14/Annotations
-Area_4/hallway_1/Annotations
-Area_4/hallway_2/Annotations
-Area_4/hallway_3/Annotations
-Area_4/hallway_4/Annotations
-Area_4/hallway_5/Annotations
-Area_4/hallway_6/Annotations
-Area_4/hallway_7/Annotations
-Area_4/hallway_8/Annotations
-Area_4/hallway_9/Annotations
-Area_4/lobby_1/Annotations
-Area_4/lobby_2/Annotations
-Area_4/office_10/Annotations
-Area_4/office_11/Annotations
-Area_4/office_12/Annotations
-Area_4/office_13/Annotations
-Area_4/office_14/Annotations
-Area_4/office_15/Annotations
-Area_4/office_16/Annotations
-Area_4/office_17/Annotations
-Area_4/office_18/Annotations
-Area_4/office_19/Annotations
-Area_4/office_1/Annotations
-Area_4/office_20/Annotations
-Area_4/office_21/Annotations
-Area_4/office_22/Annotations
-Area_4/office_2/Annotations
-Area_4/office_3/Annotations
-Area_4/office_4/Annotations
-Area_4/office_5/Annotations
-Area_4/office_6/Annotations
-Area_4/office_7/Annotations
-Area_4/office_8/Annotations
-Area_4/office_9/Annotations
-Area_4/storage_1/Annotations
-Area_4/storage_2/Annotations
-Area_4/storage_3/Annotations
-Area_4/storage_4/Annotations
-Area_4/WC_1/Annotations
-Area_4/WC_2/Annotations
-Area_4/WC_3/Annotations
-Area_4/WC_4/Annotations
-Area_5/conferenceRoom_1/Annotations
-Area_5/conferenceRoom_2/Annotations
-Area_5/conferenceRoom_3/Annotations
-Area_5/hallway_10/Annotations
-Area_5/hallway_11/Annotations
-Area_5/hallway_12/Annotations
-Area_5/hallway_13/Annotations
-Area_5/hallway_14/Annotations
-Area_5/hallway_15/Annotations
-Area_5/hallway_1/Annotations
-Area_5/hallway_2/Annotations
-Area_5/hallway_3/Annotations
-Area_5/hallway_4/Annotations
-Area_5/hallway_5/Annotations
-Area_5/hallway_6/Annotations
-Area_5/hallway_7/Annotations
-Area_5/hallway_8/Annotations
-Area_5/hallway_9/Annotations
-Area_5/lobby_1/Annotations
-Area_5/office_10/Annotations
-Area_5/office_11/Annotations
-Area_5/office_12/Annotations
-Area_5/office_13/Annotations
-Area_5/office_14/Annotations
-Area_5/office_15/Annotations
-Area_5/office_16/Annotations
-Area_5/office_17/Annotations
-Area_5/office_18/Annotations
-Area_5/office_19/Annotations
-Area_5/office_1/Annotations
-Area_5/office_20/Annotations
-Area_5/office_21/Annotations
-Area_5/office_22/Annotations
-Area_5/office_23/Annotations
-Area_5/office_24/Annotations
-Area_5/office_25/Annotations
-Area_5/office_26/Annotations
-Area_5/office_27/Annotations
-Area_5/office_28/Annotations
-Area_5/office_29/Annotations
-Area_5/office_2/Annotations
-Area_5/office_30/Annotations
-Area_5/office_31/Annotations
-Area_5/office_32/Annotations
-Area_5/office_33/Annotations
-Area_5/office_34/Annotations
-Area_5/office_35/Annotations
-Area_5/office_36/Annotations
-Area_5/office_37/Annotations
-Area_5/office_38/Annotations
-Area_5/office_39/Annotations
-Area_5/office_3/Annotations
-Area_5/office_40/Annotations
-Area_5/office_41/Annotations
-Area_5/office_42/Annotations
-Area_5/office_4/Annotations
-Area_5/office_5/Annotations
-Area_5/office_6/Annotations
-Area_5/office_7/Annotations
-Area_5/office_8/Annotations
-Area_5/office_9/Annotations
-Area_5/pantry_1/Annotations
-Area_5/storage_1/Annotations
-Area_5/storage_2/Annotations
-Area_5/storage_3/Annotations
-Area_5/storage_4/Annotations
-Area_5/WC_1/Annotations
-Area_5/WC_2/Annotations
-Area_6/conferenceRoom_1/Annotations
-Area_6/copyRoom_1/Annotations
-Area_6/hallway_1/Annotations
-Area_6/hallway_2/Annotations
-Area_6/hallway_3/Annotations
-Area_6/hallway_4/Annotations
-Area_6/hallway_5/Annotations
-Area_6/hallway_6/Annotations
-Area_6/lounge_1/Annotations
-Area_6/office_10/Annotations
-Area_6/office_11/Annotations
-Area_6/office_12/Annotations
-Area_6/office_13/Annotations
-Area_6/office_14/Annotations
-Area_6/office_15/Annotations
-Area_6/office_16/Annotations
-Area_6/office_17/Annotations
-Area_6/office_18/Annotations
-Area_6/office_19/Annotations
-Area_6/office_1/Annotations
-Area_6/office_20/Annotations
-Area_6/office_21/Annotations
-Area_6/office_22/Annotations
-Area_6/office_23/Annotations
-Area_6/office_24/Annotations
-Area_6/office_25/Annotations
-Area_6/office_26/Annotations
-Area_6/office_27/Annotations
-Area_6/office_28/Annotations
-Area_6/office_29/Annotations
-Area_6/office_2/Annotations
-Area_6/office_30/Annotations
-Area_6/office_31/Annotations
-Area_6/office_32/Annotations
-Area_6/office_33/Annotations
-Area_6/office_34/Annotations
-Area_6/office_35/Annotations
-Area_6/office_36/Annotations
-Area_6/office_37/Annotations
-Area_6/office_3/Annotations
-Area_6/office_4/Annotations
-Area_6/office_5/Annotations
-Area_6/office_6/Annotations
-Area_6/office_7/Annotations
-Area_6/office_8/Annotations
-Area_6/office_9/Annotations
-Area_6/openspace_1/Annotations
-Area_6/pantry_1/Annotations
+crimp-1/rebuild/Annotations
+crimp-2/rebuild/Annotations
+crimp-3/rebuild/Annotations
diff --git a/data/s3dis/meta_data/class_names.txt b/data/s3dis/meta_data/class_names.txt
index ca1d1788..811f8a42 100644
--- a/data/s3dis/meta_data/class_names.txt
+++ b/data/s3dis/meta_data/class_names.txt
@@ -1,13 +1,3 @@
-ceiling
-floor
-wall
-beam
-column
-window
-door
-table
-chair
-sofa
-bookcase
-board
-clutter
+empty
+hold
+volume
diff --git a/mmdet3d/datasets/s3dis_dataset.py b/mmdet3d/datasets/s3dis_dataset.py
index ed289cab..a5f41428 100644
--- a/mmdet3d/datasets/s3dis_dataset.py
+++ b/mmdet3d/datasets/s3dis_dataset.py
@@ -49,13 +49,12 @@ class S3DISDataset(Det3DDataset):
             Defaults to False.
     """
     METAINFO = {
-        'classes': ('table', 'chair', 'sofa', 'bookcase', 'board'),
+        'classes': ('empty', 'hold', 'volume'),
         # the valid ids of segmentation annotations
-        'seg_valid_class_ids': (7, 8, 9, 10, 11),
+        'seg_valid_class_ids': (1, 2),
         'seg_all_class_ids':
         tuple(range(1, 14)),  # possibly with 'stair' class
-        'palette': [(170, 120, 200), (255, 0, 0), (200, 100, 100),
-                    (10, 200, 100), (200, 200, 200)]
+        'palette': [(0, 0, 0), (255, 0, 0), (0, 255, 0)]
     }
 
     def __init__(self,
diff --git a/tools/create_data.py b/tools/create_data.py
index 406f1b06..80f69ab5 100644
--- a/tools/create_data.py
+++ b/tools/create_data.py
@@ -143,7 +143,7 @@ def s3dis_data_prep(root_path, info_prefix, out_dir, workers):
     """
     indoor.create_indoor_info_file(
         root_path, info_prefix, out_dir, workers=workers)
-    splits = [f'Area_{i}' for i in [1, 2, 3, 4, 5, 6]]
+    splits = [f'crimp-{i}' for i in [1, 2, 3]]
     for split in splits:
         filename = osp.join(out_dir, f'{info_prefix}_infos_{split}.pkl')
         update_pkl_infos('s3dis', out_dir=out_dir, pkl_path=filename)
diff --git a/tools/dataset_converters/indoor_converter.py b/tools/dataset_converters/indoor_converter.py
index 90922856..cb962fb6 100644
--- a/tools/dataset_converters/indoor_converter.py
+++ b/tools/dataset_converters/indoor_converter.py
@@ -94,7 +94,7 @@ def create_indoor_info_file(data_path,
         # S3DIS doesn't have a fixed train-val split
         # it has 6 areas instead, so we generate info file for each of them
         # in training, we will use dataset to wrap different areas
-        splits = [f'Area_{i}' for i in [1, 2, 3, 4, 5, 6]]
+        splits = [f'crimp-{i}' for i in [1, 2, 3]]
         for split in splits:
             dataset = S3DISData(root_path=data_path, split=split)
             info = dataset.get_infos(num_workers=workers, has_label=True)
diff --git a/tools/dataset_converters/s3dis_data_utils.py b/tools/dataset_converters/s3dis_data_utils.py
index d7a76a97..34684d0f 100644
--- a/tools/dataset_converters/s3dis_data_utils.py
+++ b/tools/dataset_converters/s3dis_data_utils.py
@@ -20,20 +20,17 @@ class S3DISData(object):
     def __init__(self, root_path, split='Area_1'):
         self.root_dir = root_path
         self.split = split
-        self.data_dir = osp.join(root_path,
-                                 'Stanford3dDataset_v1.2_Aligned_Version')
+        self.data_dir = osp.join(root_path, 'Stanford3dDataset_v1.2_Aligned_Version')
 
         # Following `GSDN <https://arxiv.org/abs/2006.12356>`_, use 5 furniture
         # classes for detection: table, chair, sofa, bookcase, board.
-        self.cat_ids = np.array([7, 8, 9, 10, 11])
+        self.cat_ids = np.array([0, 1, 2])
         self.cat_ids2class = {
             cat_id: i
             for i, cat_id in enumerate(list(self.cat_ids))
         }
 
-        assert split in [
-            'Area_1', 'Area_2', 'Area_3', 'Area_4', 'Area_5', 'Area_6'
-        ]
+        assert split in [ 'crimp-1', 'crimp-2', 'crimp-3' ]
         self.sample_id_list = os.listdir(osp.join(self.data_dir,
                                                   split))  # conferenceRoom_1
         for sample_id in self.sample_id_list:
@@ -169,7 +166,7 @@ class S3DISSegData(object):
     def __init__(self,
                  data_root,
                  ann_file,
-                 split='Area_1',
+                 split=None,
                  num_points=4096,
                  label_weight_func=None):
         self.data_root = data_root
@@ -177,9 +174,8 @@ class S3DISSegData(object):
         self.split = split
         self.num_points = num_points
 
-        self.all_ids = np.arange(13)  # all possible ids
-        self.cat_ids = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
-                                 12])  # used for seg task
+        self.all_ids = np.arange(3)  # all possible ids
+        self.cat_ids = np.array([0, 1, 2])  # used for seg task
         self.ignore_index = len(self.cat_ids)
 
         self.cat_id2class = np.ones(
diff --git a/tools/dataset_converters/update_infos_to_v2.py b/tools/dataset_converters/update_infos_to_v2.py
index a2ddbd06..e2f23231 100644
--- a/tools/dataset_converters/update_infos_to_v2.py
+++ b/tools/dataset_converters/update_infos_to_v2.py
@@ -535,7 +535,7 @@ def update_s3dis_infos(pkl_path, out_dir):
         print(f'Warning, you may overwriting '
               f'the original data {pkl_path}.')
         time.sleep(5)
-    METAINFO = {'classes': ('table', 'chair', 'sofa', 'bookcase', 'board')}
+    METAINFO = {'classes': ('empty', 'hold', 'volume')}
     print(f'Reading from input file: {pkl_path}.')
     data_list = mmengine.load(pkl_path)
     print('Start updating:')

oneformer3d

diff --git a/configs/oneformer3d_1xb2_s3dis-area-5.py b/configs/oneformer3d_1xb2_s3dis-area-5.py
index 7cef879..7c93350 100644
--- a/configs/oneformer3d_1xb2_s3dis-area-5.py
+++ b/configs/oneformer3d_1xb2_s3dis-area-5.py
@@ -59,7 +59,7 @@ model = dict(
             fix_mean_loss=True)),
     train_cfg=dict(),
     test_cfg=dict(
-        topk_insts=450,
+        topk_insts=200,
         inst_score_thr=0.0,
         pan_score_thr=0.4,
         npoint_thr=300,
@@ -69,8 +69,8 @@ model = dict(
         nms=True,
         matrix_nms_kernel='linear',
         num_sem_cls=num_semantic_classes,
-        stuff_cls=[0, 1, 2, 3, 4, 5, 6, 12],
-        thing_cls=[7, 8, 9, 10, 11]))
+        stuff_cls=[0],
+        thing_cls=[1, 2]))
 
 # dataset settings
 dataset_type = 'S3DISSegDataset_'
@@ -80,8 +80,8 @@ data_prefix = dict(
     pts_instance_mask='instance_mask',
     pts_semantic_mask='semantic_mask')
 
-train_area = [1, 2, 3, 4, 6]
-test_area = 5
+train_area = [1]
+test_area = [2, 3]
 
 train_pipeline = [
     dict(
@@ -99,7 +99,7 @@ train_pipeline = [
         with_seg_3d=True),
     dict(
         type='PointSample_',
-        num_points=180000),
+        num_points=100000),
     dict(type='PointInstClassMapping_',
         num_classes=num_instance_classes),
     dict(
@@ -161,7 +161,7 @@ train_dataloader = dict(
                 dict(
                     type=dataset_type,
                     data_root=data_root,
-                    ann_file=f's3dis_infos_Area_{i}.pkl',
+                    ann_file=f's3dis_infos_crimp-{i}.pkl',
                     pipeline=train_pipeline,
                     filter_empty_gt=True,
                     data_prefix=data_prefix,
@@ -174,31 +174,34 @@ val_dataloader = dict(
     persistent_workers=True,
     sampler=dict(type='DefaultSampler', shuffle=False),
     dataset=dict(
+        type='ConcatDataset',
+        datasets=([
+            dict(
                 type=dataset_type,
                 data_root=data_root,
-        ann_file=f's3dis_infos_Area_{test_area}.pkl',
+                ann_file=f's3dis_infos_crimp-{i}.pkl',
                 pipeline=test_pipeline,
                 test_mode=True,
                 data_prefix=data_prefix,
                 box_type_3d='Depth',
-        backend_args=None))
+                backend_args=None) for i in test_area])))
 test_dataloader = val_dataloader
 
 class_names = [
-    'ceiling', 'floor', 'wall', 'beam', 'column', 'window', 'door',
-    'table', 'chair', 'sofa', 'bookcase', 'board', 'clutter', 'unlabeled']
+    'empty', 'hold', 'volume'
+]
 label2cat = {i: name for i, name in enumerate(class_names)}
 metric_meta = dict(
     label2cat=label2cat,
-    ignore_index=[num_semantic_classes],
+    ignore_index=[0],
     classes=class_names,
     dataset_name='S3DIS')
-sem_mapping = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
+sem_mapping = [0, 1, 2]
 
 val_evaluator = dict(
     type='UnifiedSegMetric',
-    stuff_class_inds=[0, 1, 2, 3, 4, 5, 6, 12],
-    thing_class_inds=[7, 8, 9, 10, 11],
+    stuff_class_inds=[0],
+    thing_class_inds=[1, 2],
     min_num_points=1,
     id_offset=2**16,
     sem_mapping=sem_mapping,
@@ -208,11 +211,13 @@ val_evaluator = dict(
     metric_meta=metric_meta)
 test_evaluator = val_evaluator
 
+EPOCHS = 4096
+
 optim_wrapper = dict(
     type='OptimWrapper',
     optimizer=dict(type='AdamW', lr=0.0001, weight_decay=0.05),
     clip_grad=dict(max_norm=10, norm_type=2))
-param_scheduler = dict(type='PolyLR', begin=0, end=512, power=0.9)
+param_scheduler = dict(type='PolyLR', begin=0, end=4096, power=0.9)
 
 custom_hooks = [dict(type='EmptyCacheHook', after_iter=True)]
 default_hooks = dict(
@@ -224,6 +229,7 @@ default_hooks = dict(
 
 load_from = 'work_dirs/tmp/instance-only-oneformer3d_1xb2_scannet-and-structured3d.pth'
 
-train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=512, val_interval=16)
+
+train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=4096, val_interval=64)
 val_cfg = dict(type='ValLoop')
 test_cfg = dict(type='TestLoop')
diff --git a/oneformer3d/instance_seg_eval.py b/oneformer3d/instance_seg_eval.py
index 748a4e0..005311b 100644
--- a/oneformer3d/instance_seg_eval.py
+++ b/oneformer3d/instance_seg_eval.py
@@ -2,6 +2,8 @@
 # We fix instance seg metric to accept boolean instance seg mask of
 # shape (n_points, n_instances) instead of integer mask of shape
 # (n_points, ).
+from collections import defaultdict
+
 import numpy as np
 from mmengine.logging import print_log
 from terminaltables import AsciiTable
@@ -30,13 +32,24 @@ def aggregate_predictions(masks, labels, scores, valid_class_ids):
         label = label.numpy()
         score = score.numpy()
         info = dict()
+
+        categories = defaultdict(int)
+
         for i in range(mask.shape[0]):
+            categories[label[i]] += 1
+
+            if not 0 <= label[i] < len(valid_class_ids):
+                print("Incorrect label:", label[i])
+                continue
+
             # match pred_instance['filename'] from assign_instances_for_scan
             file_name = f'{id}_{i}'
             info[file_name] = dict()
             info[file_name]['mask'] = mask[i]
             info[file_name]['label_id'] = valid_class_ids[label[i]]
             info[file_name]['conf'] = score[i]
+        print(categories)
+
         infos.append(info)
     return infos
 
diff --git a/oneformer3d/s3dis_dataset.py b/oneformer3d/s3dis_dataset.py
index 7cc1ea4..cb7b076 100644
--- a/oneformer3d/s3dis_dataset.py
+++ b/oneformer3d/s3dis_dataset.py
@@ -6,14 +6,10 @@ from mmdet3d.datasets.s3dis_dataset import S3DISDataset
 class S3DISSegDataset_(S3DISDataset):
     METAINFO = {
         'classes':
-        ('ceiling', 'floor', 'wall', 'beam', 'column', 'window', 'door',
-         'table', 'chair', 'sofa', 'bookcase', 'board', 'clutter'),
-        'palette': [[0, 255, 0], [0, 0, 255], [0, 255, 255], [255, 255, 0],
-                    [255, 0, 255], [100, 100, 255], [200, 200, 100],
-                    [170, 120, 200], [255, 0, 0], [200, 100, 100],
-                    [10, 200, 100], [200, 200, 200], [50, 50, 50]],
+        ('empty', 'hold', 'volume'),
+        'palette': [[0, 0, 0], [255, 0, 0], [0, 255, 0]],
         'seg_valid_class_ids':
-        tuple(range(13)),
+        tuple(range(1, 3)),
         'seg_all_class_ids':
-        tuple(range(14))  # possibly with 'stair' class
+        tuple(range(3))
     }

@shaonianG
Copy link

I had the same issue. And solved it by taking care of the good format of the input data.

can you give me any advices to fix this problem?which step i should to debug,appreciate for any suggestions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants