New thermal camera test failure for Edifice

[adlarkin]

Environment

• OS Version: Ubuntu 18.04
• Source or binary build? Source, commit 6c91ff9

Description

• Expected behavior: The INTEGRATION_thermal_sensor_system test (ThermalSensorTest.ThermalSensorSystemInvalidConfig) should pass
• Actual behavior: The test fails

Steps to reproduce

1. Clone the repo: git clone https://github.com/ignitionrobotics/ign-gazebo.git
2. Complete the build so that tests are available:

cd ign-gazebo
mkdir build
cd build
cmake ..
make

3. Run the thermal sensor integration test:

# the following steps assume you are already in the ign-rendering/build directory
cd bin
./INTEGRATION_thermal_sensor_system

Output

[Msg] Setting ambient temperature to 310 Kelvin and gradient to -0.0065 K/m. The resulting temperature range is: 0.750384 Kelvin.
[Wrn] [RenderUtil.cc:1048] Unable to set thermal camera temperature linear resolution. Value must be greater than 0. Using the default value: 3.
[Wrn] [RenderUtil.cc:1061] Unable to set thermal camera temperature range.Max temperature must be greater or equal to min. Using the default values : [-inf, inf].
/home/addisu/ws/edifice/src/ign-gazebo/test/integration/thermal_sensor_system.cc:197: Failure
Value of: received
  Actual: false
Expected: true
/home/addisu/ws/edifice/src/ign-gazebo/test/integration/thermal_sensor_system.cc:198: Failure
Expected: (nullptr) != (g_image), actual: (nullptr) vs NULL
[Dbg] [Sensors.cc:309] SensorsPrivate::Stop
[Dbg] [Sensors.cc:295] SensorsPrivate::RenderThread stopped
[Dbg] [SimulationRunner.cc:499] Exiting postupdate worker thread (1)
[Dbg] [SimulationRunner.cc:499] Exiting postupdate worker thread (2)
[Dbg] [SimulationRunner.cc:499] Exiting postupdate worker thread (0)
[Dbg] [Sensors.cc:309] SensorsPrivate::Stop
[  FAILED  ] ThermalSensorTest.ThermalSensorSystemInvalidConfig (3176 ms)

[adlarkin]

A similar issue appeared when forward porting ign-rendering to Edifice as well: gazebosim/gz-rendering#253

[chapulina]

That test is passing on CI:

• Jenkins Ubuntu Edifice
• Jenkins Ubuntu Dome

And it seems to have passed on the latest GitHub actions too. Where is it failing, locally? Does it always fail or is it flaky?

[adlarkin]

@azeey, you had local failures, right? Can you share a little more information about what you're seeing?

[azeey]

Sorry for the delay in responding. The error I get is the same as the one shown in the description, but it doesn't happen every time. I think it might be a flaky test due to transport issues. Locally, you can try running ctest -VV -I 173,173 --repeat-until-fail 1000 where 173 is the index of the INTEGRATION_thermal_sensor_system test.

[jacobperron]

Another instance of this test failure: https://build.osrfoundation.org/job/ignition_rendering-ci-ign-rendering5-homebrew-amd64/17/

I noticed many instances of the following logged error (I'm not sure if it's relevant):

[Err] [Ogre2RenderEngine.cc:467] Unable to find Ogre Plugin[/usr/local/opt/ogre2.1/lib/OGRE-2.1/RenderSystem_GL3Plus]. Rendering will not be possible.Make sure you have installed OGRE properly.

Edit: I meant to post this over here instead gazebosim/gz-rendering#253

[jacobperron]

Looks like the same test failing for Fortress: https://build.osrfoundation.org/job/ignition_gazebo-ci-main-bionic-amd64/59/testReport/junit/(root)/ThermalSensorTest/ThermalSensorSystemInvalidConfig/

[chapulina]

I think I figured out what the problem is on Edifice, see #994