Multispectral data from Mavic 3 Multispectral

[alduxvm]

How did you install WebODM (docker, installer, etc.)?

docker

What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/)

Chrome 116 on macOS (Monterey)

What is the problem?

Cannot process dataset error. Dataset is composed from RBG images (jpg) and multispectral (4 extra files in tiff format), task output error:

[INFO]    Export reconstruction stats
[INFO]    running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" compute_statistics --diagram_max_points 100000 "/var/www/data/30eecbf2-340e-4c2e-8a1a-696a52d9da38/opensfm"
2023-08-28 21:00:42,963 DEBUG: locator: <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f126c5ad2e0>
2023-08-28 21:00:42,963 DEBUG: Using auto colorbar locator <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f126c5ad2e0> on colorbar
2023-08-28 21:00:42,963 DEBUG: Setting pcolormesh
2023-08-28 21:00:43,643 DEBUG: locator: <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f12705c1ac0>
2023-08-28 21:00:43,643 DEBUG: Using auto colorbar locator <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f12705c1ac0> on colorbar
2023-08-28 21:00:43,643 DEBUG: Setting pcolormesh
/code/SuperBuild/install/bin/opensfm/opensfm/stats.py:1012: RuntimeWarning: invalid value encountered in divide
plt.imshow((camera_heatmap - lowest) / (highest - lowest) * 255)
[INFO]    running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" export_geocoords --reconstruction --proj "+proj=utm +zone=50 +datum=WGS84 +units=m +no_defs +type=crs" --offset-x 683515.0 --offset-y 3752428.0 "/var/www/data/30eecbf2-340e-4c2e-8a1a-696a52d9da38/opensfm"
[INFO]    Updating /var/www/data/30eecbf2-340e-4c2e-8a1a-696a52d9da38/opensfm/config.yaml
[INFO]    undistorted_image_max_size: 2592
[WARNING] Unreliable UUID/capture time detected (no primary band match), will use filenames instead
Traceback (most recent call last):
File "/code/opendm/multispectral.py", line 239, in compute_band_maps
raise Exception("Unreliable UUID/capture time detected (no primary band match)")
Exception: Unreliable UUID/capture time detected (no primary band match)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/code/run.py", line 59, in <module>
retcode = app.execute()
File "/code/stages/odm_app.py", line 117, in execute
raise e
File "/code/stages/odm_app.py", line 81, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 398, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 398, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 398, in run
self.next_stage.run(outputs)
File "/code/opendm/types.py", line 377, in run
self.process(self.args, outputs)
File "/code/stages/run_opensfm.py", line 165, in process
s2p, p2s = multispectral.compute_band_maps(reconstruction.multi_camera, primary_band_name)
File "/code/opendm/multispectral.py", line 275, in compute_band_maps
s2p[p.filename] = filename_map[filename_without_band]
KeyError: 'DJI_20230215104308_0101_MS.TIF'

How can we reproduce this? (What steps trigger the problem? What parameters are you using for processing? Include screenshots. If you are having issues processing a dataset, you must include a copy of your dataset uploaded on Dropbox, Google Drive or https://dronedb.app)

Parameters: dsm: true, radiometric-calibration: camera+sun

Dataset coming from DJI: https://terra-1-g.djicdn.com/851d20f7b9f64838a34cd02351370894/DJI%20Terra/%E5%A4%9A%E5%85%89%E8%B0%B1-Multispectral.zip

[pierotofy]

Hey @alduxvm 👋 thanks for the report and dataset.

Did you by any chance use WebODM's image resize function when uploading the images? If so, does it work if you skip the image resize when creating a new task?

[alduxvm]

Hi @pierotofy,

I just did the test of not resizing, but in my server (64GB RAM) I'm getting a Not enough memory error...

2023-09-08 15:03:43,539 INFO: -------------------------------------------------------
2023-09-08 15:03:43,539 INFO: Shots and/or GCPs are well-conditioned. Using naive 3D-3D alignment.
2023-09-08 15:03:49,807 DEBUG: Ceres Solver Report: Iterations: 25, Initial cost: 1.742312e+03, Final cost: 1.124387e+03, Termination: CONVERGENCE
2023-09-08 15:03:50,353 INFO: Removed outliers: 547
2023-09-08 15:03:50,678 INFO: Reconstruction 0: 23 images, 53020 points
2023-09-08 15:03:50,678 INFO: 1 partial reconstructions in total.
[INFO]    Export reconstruction stats
[INFO]    running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" compute_statistics --diagram_max_points 100000 "/var/www/data/978f7670-a605-46fa-b945-4458a1f5dc88/opensfm"
2023-09-08 15:03:58,778 DEBUG: locator: <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f9ce6c1eac0>
2023-09-08 15:03:58,778 DEBUG: Using auto colorbar locator <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f9ce6c1eac0> on colorbar
2023-09-08 15:03:58,779 DEBUG: Setting pcolormesh
2023-09-08 15:03:59,487 DEBUG: locator: <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f9ced4a93d0>
2023-09-08 15:03:59,487 DEBUG: Using auto colorbar locator <matplotlib.colorbar._ColorbarAutoLocator object at 0x7f9ced4a93d0> on colorbar
2023-09-08 15:03:59,487 DEBUG: Setting pcolormesh
/code/SuperBuild/install/bin/opensfm/opensfm/stats.py:1012: RuntimeWarning: invalid value encountered in divide
plt.imshow((camera_heatmap - lowest) / (highest - lowest) * 255)
[INFO]    running "/code/SuperBuild/install/bin/opensfm/bin/opensfm" export_geocoords --reconstruction --proj "+proj=utm +zone=50 +datum=WGS84 +units=m +no_defs +type=crs" --offset-x 683515.0 --offset-y 3752428.0 "/var/www/data/978f7670-a605-46fa-b945-4458a1f5dc88/opensfm"
[INFO]    Updating /var/www/data/978f7670-a605-46fa-b945-4458a1f5dc88/opensfm/config.yaml
[INFO]    undistorted_image_max_size: 5280
[INFO]    Computing band alignment
/code/run.sh: line 9: 102367 Killed                  python3 $RUNPATH/run.py "$@"

[pierotofy]

Try with a smaller subset of images.

That might be an issue due to the very large size of the multispectral images. Mm.

[alduxvm]

I increased the swap memory (from 8GB to 64GB) in my server and its currently processing! I will post the results! no errors so far!!

[pierotofy]

There's three issues here:

1. WebODM's resize function is definitely stripping important XMP/Exif tags from the JPEGs. We should fix that and skip resizing if any TIFF file is present (or long-term find a way to resize while preserving all meta)
2. ODM is running out of memory (or using a really large amount of memory) while calling find_features_homography. I have a fix for this coming up.
3. I'm doubtful ODM will be able to process this combination of RGB+multispectral images. The RGB sensor has severe lens distortion and does not seem to closely match the lens of the multispectral sensor. You'll get results, but they will most likely be distorted/unaligned.

[alduxvm]

The processing was done!

1. Yes, the resizing was making the process fail
2. I had to add 64GB of swap, and it peaked at using 64GB ram and around 40GB of swap, but it manage to do it at the end!
3. You can see the RGB ortho here the only issue is that I cannot see the multispectral channels, I'm just able to see the RGB, how can I know if the multispectral was processed??

[pierotofy]

Open it with QGIS. https://qgis.org/