CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input

[sean-wade]

When I started to train cascade-RCNN, I got the error above... Is something I set is wrong ?

Here is my config setting:

(Base-RCNN-FPN.yaml):
MODEL:
META_ARCHITECTURE: "GeneralizedRCNN"
BACKBONE:
NAME: "build_resnet_fpn_backbone"
RESNETS:
OUT_FEATURES: ["res2", "res3", "res4", "res5"]
FPN:
IN_FEATURES: ["res2", "res3", "res4", "res5"]
ANCHOR_GENERATOR:
SIZES: [[32], [64], [128], [256], [512]] # One size for each in feature map
ASPECT_RATIOS: [[0.5, 1.0, 2.0]] # Three aspect ratios (same for all in feature maps)
RPN:
IN_FEATURES: ["p2", "p3", "p4", "p5", "p6"]
PRE_NMS_TOPK_TRAIN: 2000 # Per FPN level
PRE_NMS_TOPK_TEST: 1000 # Per FPN level
# Detectron1 uses 2000 proposals per-batch,
# (See "modeling/rpn/rpn_outputs.py" for details of this legacy issue)
# which is approximately 1000 proposals per-image since the default batch size for FPN is 2.
POST_NMS_TOPK_TRAIN: 1000
POST_NMS_TOPK_TEST: 1000
ROI_HEADS:
NAME: "StandardROIHeads"
IN_FEATURES: ["p2", "p3", "p4", "p5"]
ROI_BOX_HEAD:
NAME: "FastRCNNConvFCHead"
NUM_FC: 2
POOLER_RESOLUTION: 7
ROI_MASK_HEAD:
NAME: "MaskRCNNConvUpsampleHead"
NUM_CONV: 4
POOLER_RESOLUTION: 14
DATASETS:
TRAIN: ("voc_2007_trainval",)
TEST: ("voc_2007_minitest",)
SOLVER:
IMS_PER_BATCH: 4 #16
BASE_LR: 0.005 #0.02
STEPS: (60000, 80000)
MAX_ITER: 90000
INPUT:
MIN_SIZE_TRAIN: (640, 672, 704, 736, 768, 800)

(cascade_mask_rcnn_X_152_32x8d_FPN_IN5k_gn_dconv.yaml):
BASE: "../Base-RCNN-FPN.yaml"
VERSION: 2
MODEL:
MASK_ON: False #True
WEIGHTS: "catalog://ImageNetPretrained/FAIR/X-152-32x8d-IN5k"
RESNETS:
STRIDE_IN_1X1: False # this is a C2 model
NUM_GROUPS: 32
WIDTH_PER_GROUP: 8
DEPTH: 152
DEFORM_ON_PER_STAGE: [False, True, True, True]
ROI_HEADS:
NAME: "CascadeROIHeads"
NUM_CLASSES: 26
ROI_BOX_HEAD:
NAME: "FastRCNNConvFCHead"
NUM_CONV: 4
NUM_FC: 1
NORM: "GN"
CLS_AGNOSTIC_BBOX_REG: True
ROI_MASK_HEAD:
NUM_CONV: 8
NORM: "GN"
RPN:
POST_NMS_TOPK_TRAIN: 2000
SOLVER:
IMS_PER_BATCH: 128
STEPS: (35000, 45000)
MAX_ITER: 50000
BASE_LR: 0.16
INPUT:
MIN_SIZE_TRAIN: (640, 864)
MIN_SIZE_TRAIN_SAMPLING: "range"
MAX_SIZE_TRAIN: 1440
CROP:
ENABLED: True
TEST:
EVAL_PERIOD: 2500
OUTPUT_DIR: "YJH/cascade_mask_rcnn_X_152_32x8d_FPN_IN5k_gn_dconv"

[sean-wade]

I changed my INPUT config into the cascade_mask_rcnn_X_152_32x8d_FPN_IN5k_gn_dconv.yaml. And it works...