Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[depthMap] New option for multi-resolution similarity estimation and optimizations #1351

Merged
merged 77 commits into from
Jun 13, 2023
Merged

[depthMap] New option for multi-resolution similarity estimation and optimizations #1351

merged 77 commits into from
Jun 13, 2023

Conversation

gregoire-dl
Copy link
Member

@gregoire-dl gregoire-dl commented Feb 13, 2023 •
edited
Loading

Description

This PR has six purposes:

  • Compute the Refine step with a more meaningful resolution (using SGM thickness map).
  • More types of intermediate results export for better understanding / testing and debugging.
  • Use mipmap images instead of only two resolutions (SGM & Refine)
  • Use consistant scale for similarity volume patch comparison (SGM & Refine) [user option]
  • Use user custom multi-resolution patch pattern for similarity volume patch comparison (SGM & Refine) [user option]
  • Various CUDA code optimizations / Code simplification / More documentation

Features list

  • Generalization of tile map I/O for other map types (previously only depth/similarity map)
  • Add SGM max similarity thresold option
  • Add export intermediate normal maps option
  • Add export similarity volume topographic cut option
  • Add Refine upscale middle depth with bilinear interpolation option
  • Use pre-computed middle depth pixSize in Refine & ColorOptimization
  • Use SGM thikness map to compute Refine pixSize map
  • Smooth SGM thikness map using adjacent pixels
  • Use CUDA Mipmapped array to store image at multiple resolutions.
  • Use DeviceMipmapImage and DeviceCameraParams in DeviceCache
  • Use DeviceMipmapImage and DeviceCameraParams in all SGM and Refine CUDA functions
  • Remove DeviceCamera objects
  • Use planeSweeping CUDA function for normal map estimation and remove normal mapping module
  • Remove depthMapEstimation property tree usage
  • Use consistant scale for similarity volume patch comparison (SGM & Refine)
  • Use user custom multi-resolution patch pattern for similarity volume patch comparison (SGM & Refine)
  • CUDA code optimizations
  • Fix depth maps alpha masking

Related Meshroom branch: alicevision/Meshroom#1984

@gregoire-dl gregoire-dl force-pushed the dev/depthMapMultiResolution branch from 143acb7 to 8bd62d4 Compare April 14, 2023 08:55
@gregoire-dl gregoire-dl mentioned this pull request Apr 14, 2023
Merged
@gregoire-dl gregoire-dl added review and removed wip work in progress labels Apr 14, 2023
@gregoire-dl gregoire-dl marked this pull request as ready for review April 14, 2023 17:48
@gregoire-dl gregoire-dl self-assigned this Apr 14, 2023
@cbentejac cbentejac self-requested a review June 5, 2023 09:07
@fabiencastan fabiencastan added this to the 3.1.0 milestone Jun 5, 2023
gregoire-dl added 17 commits June 8, 2023 09:30
@gregoire-dl @cbentejac
[depthMap] SgmParams: Add maxSimilarity parameter
e35406e
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceSimilarityVolume: Add maxSimilarity threshold
51b9fc3
@gregoire-dl @cbentejac
[software] depthMapEstimation: Add sgmMaxSimilarity parameter
9d91ad6
@gregoire-dl @cbentejac
[mvsUtils] Rename depthMapIO to mapIO
3a34a14
@gregoire-dl @cbentejac
[mvsUtils] mapIO: Refactor and simplification
3acf5ae 
Function read/write map now support float/rgbf maps.
Tiling can be used on other maps (i.e. normal map) than depth/similarity maps.
All maps now use the same read/write map functions (which support tiling) in the MVS part of AliceVision.
@gregoire-dl @cbentejac
[mvsUtils] mapIO: Add asserts / Use default channels on pixelRoi
fde842a
@gregoire-dl @cbentejac
[mvsUtils] fileIO: Add depth map estimation forlder for normal maps o…
ae6d916 
…utput
@gregoire-dl @cbentejac
[depthMap] Sgm & Refine: Add intermediate normal map exports
750c53b
@gregoire-dl @cbentejac
[software] depthMapEstimation: Add exportIntermediateNormalMaps param…
2c44b09 
…eter
@gregoire-dl @cbentejac
[depthMap] depthMapUtils: Use name instead of customSuffix for better…
49de300 
… readability
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceSimilarityVolume: Use middle depth pixSize
aed2c81
@gregoire-dl @cbentejac
[depthMap] volumeIO: Export cross volume with middle depth pixSize
daf04f8
@gregoire-dl @cbentejac
[depthMap] Refine: Add upscale with bilinear interpolation option
6f6815d
@gregoire-dl @cbentejac
[software] depthMapEstimation: Add option interpolateMiddleDepth
2ce201e
@gregoire-dl @cbentejac
[depthMap] Sgm/Refine: Compute and use Sgm depth thikness as Refine p…
2557d91 
…ixSize
@gregoire-dl @cbentejac
[software] depthMapEstimation: Add option depthThiknessInflate
1c1105c
@gregoire-dl @cbentejac
[mvsUtils] fileIO: Add volumeTopographicCut file type
52f7e75
gregoire-dl added 17 commits June 8, 2023 09:31
@gregoire-dl @cbentejac
[depthMap] Cuda: Matrix: Small optimizations
478be63
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceDepthSimilarityMap: Unroll normal map computation
09d85ff
@gregoire-dl @cbentejac
[depthMap] cuda: deviceSimilarityVolume: get max block size from cuda…
96cd9ba 
… api
@gregoire-dl @cbentejac
[fuseCut] Fuser: Fix wrong depth map type in computeAveragePixelSizeI…
3a6aba8 
…nHexahedron
@gregoire-dl @cbentejac
[mvsUtils] mapIO: Fix filtered depth map storage type
7e0410e
@gregoire-dl @cbentejac
[depthMap] Cuda: Color: Define Rc min alpha / Tc min alpha
6421430
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceSimilarityVolume: Fix comments
f3b4e03
@gregoire-dl @cbentejac
[depthMap] Cuda: use Rc min alpha / Tc min alpha
70ca4bd
@gregoire-dl @cbentejac
[depthMap] Cuda: SimStat: computeWSim optimization
e61644b
@gregoire-dl @cbentejac
[depthMap] Cuda: Memory: Remove unused CudaArray class
9d3e7ff
@gregoire-dl @cbentejac
[depthMap] Cuda: DeviceMipmapImage: Rename method getDimensions
78be730
@gregoire-dl @cbentejac
[depthMap] Cuda: DevicePatchPattern: pre-compute subpart downscale
33753dd
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceDepthSimilarityMap: Remove max unroll
f2cecf4
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceMipmappedArray: Add unroll
3b62ebf
@gregoire-dl @cbentejac
[depthMap] Cuda: deviceColorConversion: minor optimization
345e9ba
@gregoire-dl @cbentejac
[mvsUtils] Fix Thickness spelling error
67fa68c
@gregoire-dl @cbentejac
[depthMap] Cuda: Patch: inline instead of static
9ce16af
@cbentejac cbentejac force-pushed the dev/depthMapMultiResolution branch from 5b9de73 to 9ce16af Compare June 8, 2023 10:48
cbentejac
cbentejac previously approved these changes Jun 8, 2023
View reviewed changes
@fabiencastan fabiencastan changed the title [depthMap] Multi-Resolution similarity estimation [depthMap] NEw option for multi-resolution similarity estimation Jun 13, 2023
@fabiencastan fabiencastan changed the title [depthMap] NEw option for multi-resolution similarity estimation [depthMap] New option for multi-resolution similarity estimation Jun 13, 2023
@fabiencastan fabiencastan changed the title [depthMap] New option for multi-resolution similarity estimation [depthMap] New option for multi-resolution similarity estimation and optimizations Jun 13, 2023
@fabiencastan fabiencastan merged commit a2b9e03 into develop Jun 13, 2023
@fabiencastan fabiencastan deleted the dev/depthMapMultiResolution branch June 13, 2023 09:45
@almarouk almarouk mentioned this pull request Jun 14, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cbentejac cbentejac cbentejac approved these changes

@fabiencastan fabiencastan fabiencastan approved these changes

Assignees

@gregoire-dl gregoire-dl

Labels
CUDA majorFeature review type:enhancement type:pr
Projects
None yet
Milestone
3.1.0
Development

Successfully merging this pull request may close these issues.
3 participants
@gregoire-dl @fabiencastan @cbentejac