Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TorchScript inference in Python #2056

Closed
SkalskiP opened this issue Sep 21, 2020 · 2 comments
Closed

TorchScript inference in Python #2056

SkalskiP opened this issue Sep 21, 2020 · 2 comments
Labels
duplicate This issue or pull request already exists

Comments

@SkalskiP
Copy link

If you do not know the root cause of the problem, and wish someone to help you, please
post according to this template:

Instructions To Reproduce the Issue:

  • With Detectron2, I have trained the R-CNN Mask model, which is based on the following architecture: link to yaml file.
  • I converted my model to TorchScript format using script provided by Detectron2 team: link to script, so it is now in .pt format.
  • Loaded and executed model with:
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
x = torch.jit.load(MODEL_PATH).to(device)

image = np.zeros((1, 3, 800, 800)).astype(np.float32)
im_info = np.float32((800, 800, 1))
im_info = np.reshape(im_info, (1, -1))

t1 = torch.tensor(image).to(device)
t2 = torch.tensor(im_info).to(device)
x.forward((t1, t2))
  • Go t RuntimeError:
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/detectron2/export/caffe2_modeling.py", line 16, in forward
    _5, _6, _7, _8, _9, _10, _11, = (_2).forward(input, )
    _12 = (_1).forward(_5, _6, _7, _8, _9, im_info, )
    _13 = (_0).forward(_12, _5, _6, _7, _10, im_info, _11, )
           ~~~~~~~~~~~ <--- HERE
    _14, _15, _16, _17, = _13
    return (_14, _15, _16, _17)
  File "code/__torch__/detectron2/modeling/roi_heads/roi_heads.py", line 42, in forward
    _16 = torch.select(_15, 1, 0)
    _17 = (_1).forward(tensor0, argument_2, argument_3, argument_4, argument_7, )
    _18 = (tensor0, _10, _11, (_0).forward(_17, _16, ))
                               ~~~~~~~~~~~ <--- HERE
    return _18
  File "code/__torch__/detectron2/modeling/roi_heads/roi_heads.py", line 41, in forward
    _15 = torch.slice(roi_batch_ids, 0, 0, 9223372036854775807, 1)
    _16 = torch.select(_15, 1, 0)
    _17 = (_1).forward(tensor0, argument_2, argument_3, argument_4, argument_7, )
           ~~~~~~~~~~~ <--- HERE
    _18 = (tensor0, _10, _11, (_0).forward(_17, _16, ))
    return _18
  File "code/__torch__/detectron2/export/c10/___torch_mangle_268.py", line 23, in forward
    _4 = ops._caffe2.RoIAlign(argument_5, roi_fpn2, "NCHW", 0.03125, 14, 14, 0, True, None)
    roi_feat_shuffled = torch.cat([_1, _2, _3, _4], 0)
    input = ops._caffe2.BatchPermutation(roi_feat_shuffled, rois_idx_restore_int32, None)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    return input

Traceback of TorchScript, original code (most recent call last):
/opt/conda/lib/python3.6/site-packages/detectron2/export/c10.py(341): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/opt/conda/lib/python3.6/site-packages/detectron2/modeling/roi_heads/roi_heads.py(776): _forward_mask
/opt/conda/lib/python3.6/site-packages/detectron2/modeling/roi_heads/roi_heads.py(702): forward_with_given_boxes
/opt/conda/lib/python3.6/site-packages/detectron2/modeling/roi_heads/roi_heads.py(676): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/opt/conda/lib/python3.6/site-packages/detectron2/export/caffe2_modeling.py(271): forward
/opt/conda/lib/python3.6/contextlib.py(52): inner
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(534): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(548): __call__
/opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(1027): trace_module
/opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(875): trace
/opt/conda/lib/python3.6/site-packages/detectron2/export/api.py(135): export_torchscript
<ipython-input-22-98020f8308b8>(14): convert
<ipython-input-28-bf0a4ab6c258>(6): <module>
/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3326): run_code
/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3249): run_ast_nodes
/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3058): run_cell_async
/opt/conda/lib/python3.6/site-packages/IPython/core/async_helpers.py(68): _pseudo_sync_runner
/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2881): _run_cell
/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2855): run_cell
/opt/conda/lib/python3.6/site-packages/ipykernel/zmqshell.py(536): run_cell
/opt/conda/lib/python3.6/site-packages/ipykernel/ipkernel.py(294): do_execute
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(209): wrapper
/opt/conda/lib/python3.6/site-packages/ipykernel/kernelbase.py(542): execute_request
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(209): wrapper
/opt/conda/lib/python3.6/site-packages/ipykernel/kernelbase.py(272): dispatch_shell
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(209): wrapper
/opt/conda/lib/python3.6/site-packages/ipykernel/kernelbase.py(365): process_one
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(748): run
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(714): __init__
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(225): wrapper
/opt/conda/lib/python3.6/site-packages/ipykernel/kernelbase.py(378): dispatch_queue
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(748): run
/opt/conda/lib/python3.6/site-packages/tornado/gen.py(787): inner
/opt/conda/lib/python3.6/site-packages/tornado/ioloop.py(743): _run_callback
/opt/conda/lib/python3.6/site-packages/tornado/ioloop.py(690): <lambda>
/opt/conda/lib/python3.6/asyncio/events.py(145): _run
/opt/conda/lib/python3.6/asyncio/base_events.py(1451): _run_once
/opt/conda/lib/python3.6/asyncio/base_events.py(438): run_forever
/opt/conda/lib/python3.6/site-packages/tornado/platform/asyncio.py(148): start
/opt/conda/lib/python3.6/site-packages/ipykernel/kernelapp.py(563): start
/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py(658): launch_instance
/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py(16): <module>
/opt/conda/lib/python3.6/runpy.py(85): _run_code
/opt/conda/lib/python3.6/runpy.py(193): _run_module_as_main
RuntimeError: [enforce fail at batch_permutation_op.cu:66] X.dim32(0) > 0. 0 vs 0

Environment:

Provide your environment information using the following command:

wget -nc -q https://github.com/facebookresearch/detectron2/raw/master/detectron2/utils/collect_env.py && python collect_env.py

------------------------  -------------------------------------------------------------------
sys.platform              linux
Python                    3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) [GCC 7.3.0]
numpy                     1.16.4
detectron2                0.1.3 @/opt/conda/lib/python3.6/site-packages/detectron2
detectron2 compiler       GCC 7.3
detectron2 CUDA compiler  10.1
detectron2 arch flags     sm_35, sm_37, sm_50, sm_52, sm_60, sm_61, sm_70, sm_75
DETECTRON2_ENV_MODULE     <not set>
PyTorch                   1.5.0+cu101 @/opt/conda/lib/python3.6/site-packages/torch
PyTorch debug build       False
CUDA available            True
GPU 0                     Tesla V100-PCIE-16GB
CUDA_HOME                 /usr/local/cuda
NVCC                      Cuda compilation tools, release 10.1, V10.1.243
Pillow                    7.2.0
torchvision               0.6.0+cu101 @/opt/conda/lib/python3.6/site-packages/torchvision
torchvision arch flags    sm_35, sm_50, sm_60, sm_70, sm_75
fvcore                    0.1.1.post20200716
cv2                       3.4.1
------------------------  -------------------------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2019.0.5 Product Build 20190808 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.3
  - Magma 2.5.2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_INTERNAL_THREADPOOL_IMPL -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,
@ppwwyyxx
Copy link
Contributor

Duplicate of #1895 (comment)

@ppwwyyxx ppwwyyxx marked this as a duplicate of #1895 Sep 22, 2020
@ppwwyyxx ppwwyyxx added the duplicate This issue or pull request already exists label Sep 22, 2020
@SkalskiP
Copy link
Author

Thank you.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 21, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ppwwyyxx @SkalskiP