Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gpu voxels distance voxelmap issue on old gpu (GeForce GT 520MX) #121

Open
wendwosen opened this issue Jan 22, 2021 · 1 comment
Open

gpu voxels distance voxelmap issue on old gpu (GeForce GT 520MX) #121

wendwosen opened this issue Jan 22, 2021 · 1 comment

Comments

@wendwosen
Copy link

Hi @cjue

I was trying evaluate the performance of PBA distance transform in gpu voxels with AN OLD GPU (GeForce GT 520MX) which has only 2.0 compute capability. (assuming gpu voxels is compatible). Everything compiled perfectly. however, examples only related to distancemap (distance_ros_demo, distance_voxel_test) have run time error. would you please look in to the log file for the two? thank you.

wendwosen@wendwosen:~/gpu-voxels/build/bin$ ./distance_ros_demo
<2021-01-22 19:59:32.379> Gpu_voxels(Info)::main: DistanceROSDemo start. Point-cloud topic: /camera/depth/points
<2021-01-22 19:59:32.379> CudaLog(Info)::cuTestAndInitDevice: Running on GPU 0 (GeForce GT 520MX)
Remove voxels containing less points than: 0
Erode voxels with neighborhood occupancy ratio less or equal to: 0
<2021-01-22 19:59:32.405> Gpu_voxels(Info)::main: start visualizing maps
<2021-01-22 19:59:32.713> Gpu_voxels(Info)::callback: DistanceROSDemo camera callback. PointCloud size: 307200
Name of shared buffer swapped: voxel_list_buffer_swapped_6.
Name of shared buffer swapped: voxel_list_buffer_swapped_7.
<2021-01-22 19:59:32.753> Gpu_voxels(Info)::main: erode voxels into pbaDistanceVoxmap
<2021-01-22 19:59:32.766> Gpu_voxels(Info)::main: calculate distance map for 17 occupied voxels
<2021-01-22 19:59:32.768> CudaLog(Error)::cuCheckForError: unknown error(30) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 536.
<2021-01-22 19:59:44.518> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 634.
terminate called after throwing an instance of 'thrust::system::system_error'
what(): function_attributes(): after cudaFuncGetAttributes: the launch timed out and was terminated
Aborted (core dumped)

//***********

also for the distance_voxel_test

wendwosen@wendwosen:~/gpu-voxels/build/bin$ ./distance_voxel_test
<2021-01-22 19:48:39.144> Gpu_voxels(Info)::main: DistanceVoxelTest start
<2021-01-22 19:48:39.151> CudaLog(Info)::cuTestAndInitDevice: Running on GPU 0 (GeForce GT 520MX)
<2021-01-22 19:48:39.967> Gpu_voxels(Info)::main: GpuVoxels created, size: 128x128x64=1048576 voxel
<2021-01-22 19:48:39.967> Gpu_voxels(Info)::main: loading obstacles:
<2021-01-22 19:48:39.967> Gpu_voxels(Info)::main: pointcloud scaling factor: 1.000000
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: loading pointcloud from file pelican.binvox succeeded!
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 0: 0.000000/1.673473/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 1: 0.000000/1.678469/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 2: 0.000000/1.683464/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 3: 0.000000/1.688460/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 4: 0.000000/1.693455/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 5: 0.000000/1.698450/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 6: 0.000000/1.703446/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 7: 0.000000/1.708441/0.419617
<2021-01-22 19:48:40.820> Gpu_voxels(Info)::main: obstacle 8: 0.000000/1.673473/0.424613
<2021-01-22 19:48:40.821> Gpu_voxels(Info)::main: obstacle 9: 0.000000/1.678469/0.424613
<2021-01-22 19:48:40.821> Gpu_voxels(Info)::main: obstacle count before filtering: 396812
<2021-01-22 19:48:40.828> Gpu_voxels(Info)::main: obstacle count after bounds filtering: 396812
<2021-01-22 19:48:40.836> Gpu_voxels(Info)::main: obstacle count after voxel duplicate filtering: 2542
<2021-01-22 19:48:40.836> Gpu_voxels(Info)::main: ================ 3D distances ================
<2021-01-22 19:48:40.838> Gpu_voxels(Info)::main: jumpFlood 3D start
<2021-01-22 19:48:40.839> Performance(Info)::print: outsideprefix::jfa clearMap done: 0.388 ms
<2021-01-22 19:48:40.840> Performance(Info)::print: outsideprefix::jfa clearMap and insertPointloud done: 1.683 ms
<2021-01-22 19:48:40.841> VoxelmapLog(Info)::jumpFlood3D: grid: 1024, block: 1024
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.841> VoxelmapLog(Info)::jumpFlood3D: jumpFlood: memcpy temp_buffer
<2021-01-22 19:48:40.842> Performance(Info)::print: ::jumpFlood3D done: 1.468 ms
<2021-01-22 19:48:40.842> Performance(Info)::print: outsideprefix::jfa done: 3.225 ms
<2021-01-22 19:48:40.842> Gpu_voxels(Info)::main: jumpFlood 3D done
<2021-01-22 19:48:40.842> Gpu_voxels(Info)::main: starting 1 JFA runs;
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.843> CudaLog(Error)::cuCheckForError: too many resources requested for launch(7) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 187.
<2021-01-22 19:48:40.844> Performance(Info)::print: computeprefix::jfa compute done: 1.403 ms
<2021-01-22 19:48:40.844> Performance(Info)::print: outsideprefix::jfa total done: 1.897 ms
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: collected 1 median compute times
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: jfa_block_size; compute_time_ms; num_voxels; NX; NY; NZ; num_obstacles; compute_voxel_throughput; compute DV throughput byte/s; compute_framerate; sizeof(DistanceVoxel);
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: 1024; 1.403;1048576;128;128;64;2542;747381325.731;2989525302.92;712.758;4;
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: collected 1 minimum compute times
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: jfa_block_size; compute_time_ms; num_voxels; NX; NY; NZ; num_obstacles; compute_voxel_throughput; compute DV throughput byte/s; compute_framerate; sizeof(DistanceVoxel);
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: 1024; 1.403;1048576;128;128;64;2542;747381325.731;2989525302.92;712.758;4;
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: collected 1 total times (clear + insert + init + compute)
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: jfa_block_size; total_time_ms; num_voxels; NX; NY; NZ; num_obstacles; total_voxel_throughput; total DV throughput byte/s; total_framerate; sizeof(DistanceVoxel);
<2021-01-22 19:48:40.844> Gpu_voxels(Info)::main: 1024; 1.897;1048576;128;128;64;2542;552754876.12;2211019504.48;527.148;4;
<2021-01-22 19:48:40.844> Performance(Info)::print:
########## Begin of Summary for prefix ##########

<2021-01-22 19:48:40.845> Performance(Info)::print: ::parallelBanding3D warmup sync: 0.007 ms
<2021-01-22 19:48:40.846> Performance(Info)::print: ::parallelBanding3D initialisation device_vector created: 0.867 ms
<2021-01-22 19:48:40.846> Performance(Info)::print: ::parallelBanding3D initialisation done: 0.026 ms
<2021-01-22 19:48:40.846> Performance(Info)::print: pbaprefix::parallelBanding3D init done: 0.879 ms
<2021-01-22 19:48:40.848> Performance(Info)::print: ::parallelBanding3D first phase 1 floodZ done: 2.145 ms
<2021-01-22 19:48:40.851> Performance(Info)::print: ::parallelBanding3D first phase 2 backpointer done: 2.596 ms
<2021-01-22 19:48:40.851> Performance(Info)::print: ::parallelBanding3D first phase 2 forward done: 0.621 ms
<2021-01-22 19:48:40.851> VoxelmapLog(Info)::parallelBanding3D: kernelPBAphase2MergeBands finished merging with band_size 32
<2021-01-22 19:48:40.852> Performance(Info)::print: ::parallelBanding3D first phase 2 merge iteration done: 0.294 ms
<2021-01-22 19:48:40.852> VoxelmapLog(Info)::parallelBanding3D: kernelPBAphase2MergeBands finished merging with band_size 64
<2021-01-22 19:48:40.852> Performance(Info)::print: ::parallelBanding3D first phase 2 merge iteration done: 0.185 ms
<2021-01-22 19:48:40.852> CudaLog(Error)::cuCheckForError: unknown error(30) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 536.
<2021-01-22 19:48:49.157> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 541.
<2021-01-22 19:48:49.157> Performance(Info)::print: ::parallelBanding3D first phase 3 done: 8305.67 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 553.
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 556.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D first transpose done: 0.078 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 567.
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 570.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second phase 2 backpointers done: 0.041 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 579.
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 584.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second phase 2 forwardpointers done: 0.036 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 596.
<2021-01-22 19:48:49.158> VoxelmapLog(Info)::parallelBanding3D: kernelPBAphase2MergeBands finished merging with band_size 32
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 603.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second phase2 merge iteration done: 0.048 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 596.
<2021-01-22 19:48:49.158> VoxelmapLog(Info)::parallelBanding3D: kernelPBAphase2MergeBands finished merging with band_size 64
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 603.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second phase2 merge iteration done: 0.04 ms
<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 615.
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 621.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second phase3 done: 0.032 ms
terminate called after throwing an instance of '<2021-01-22 19:48:49.158> CudaLog(Error)::cuCheckForError: the launch timed out and was terminated(6) in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 627.
thrust::system::system_error'
<2021-01-22 19:48:49.158> CudaLog(Error)::cuHandleError: the launch timed out and was terminated in /home/wendwosen/gpu-voxels/packages/gpu_voxels/src/gpu_voxels/voxelmap/DistanceVoxelMap.hpp on line 634.
<2021-01-22 19:48:49.158> Performance(Info)::print: ::parallelBanding3D second transpose done: 0.04 ms
<2021-01-22 19:48:49.158> Performance(Info)::print: pbaprefix::parallelBanding3D compute done: 8311.83 ms
what(): function_attributes(): after cudaFuncGetAttributes: the launch timed out and was terminated
Aborted (core dumped)

//************

information regarding the gpu memory

01:00.0 3D controller: NVIDIA Corporation GF119M [GeForce GT 520MX] (rev a1)
Subsystem: Samsung Electronics Co Ltd GF119M [GeForce GT 520MX]
Flags: bus master, fast devsel, latency 0, IRQ 30
Memory at e0000000 (32-bit, non-prefetchable) [size=16M]
Memory at b0000000 (64-bit, prefetchable) [size=256M]
Memory at c0000000 (64-bit, prefetchable) [size=32M]
I/O ports at 3000 [size=128]
Capabilities:
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_390, nvidia_390_drm
@wendwosen
Copy link
Author

information regarding nvidia driver

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.87 Driver Version: 390.87 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GT 520MX Off | 00000000:01:00.0 N/A | N/A |
| N/A 47C P8 N/A / N/A | 241MiB / 964MiB | N/A Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
+-----------------------------------------------------------------------------+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wendwosen