Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Camera Stops Sending Image Messages After World Reset #1917

Closed
osrf-migration opened this issue Apr 6, 2016 · 16 comments
Closed

Camera Stops Sending Image Messages After World Reset #1917

osrf-migration opened this issue Apr 6, 2016 · 16 comments
Labels
6.0 bug Something isn't working major msgs

Comments

@osrf-migration
Copy link

Original report (archived issue) by Xander Dunn (Bitbucket: xanderdunn).


6.5.1

We have a Gazebo plugin that listens for certain messages, such as joint and camera image messages. It's also capable of resetting the Gazebo world.

It appears after doing a world reset, we're no longer able to get messages from the camera. Other messages, such as joint position, continue to be received as usual.

Here is a screen recording of what I see:

To reproduce (Time in my video where this starts):

  • Start an environment (0:00)
  • Send messages to Gazebo to step x 50 (0:05)
  • Notice that both the Gazebo GUI Image View and our plugin receive the camera image message successfully
  • Send message to Gazebo to reset the world (1:05)
  • Notice that the world correctly goes back to initial state.
  • Send a message to Gazebo to step (1:09)
  • Notice that the plugin is stuck endlessly waiting for a camera image.
  • Step through Gazebo using the GUI (since the plugin is blocked) (1:31)
  • Notice that the GUI camera Image View doesn't update either

Choice of physics engine doesn't seem to be important. We've reproduced this in both ODE and Simbody. It's also not specific to this .world. The behavior is identical in other environments we've built.

Do you have any thoughts on why doing a world reset would cause problems with the camera? We're just using Gazebo's built-in camera model.

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


  • Edited issue description

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


We have had issues in the past with sensors stopping updates after world resets (see #236). The cause was generally a variable called lastUpdateTime that wouldn't get reset during the world reset. So the sensors wouldn't update until enough time had passed. For example, if you called reset world after 1 minute, the sensors wouldn't update for the first minute after the reset, but then they would start updating again.

This could be a different issue though, I'll try to reproduce it.

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


I just tested with gazebo 7.0.0, and the built-in camera doesn't seem to be affected by #236.

How does your plugin wait for a camera image?

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


Are you running gazebo with --verbose? Are there any unusual console messages?

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


The bug you described above sounds consistent with what I'm seeing. If I run the world for 50 iterations before the reset, it will take 50 iterations after the reset before I start seeing camera messages again. I also ran an experiment once with 100 iterations, then reset, and again I started to see messages 100 iterations after the reset.

I'm on 6.5.1, though. The bug should be independent of my plugin. I see it in the built-in camera image view in Gazebo, as you can see in the screen recording.

Yeah, I do run with --verbose. While I have sometimes seen some Queue limit reached warnings, they don't seem correlated. I've also gone through entire experiments showing this bug without any of these warnings showing up.

I'll run some experiments entirely without my plugin to show that it's unrelated, as well as try the same on Gazebo 7.

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


The queue limit warnings occur when the plugin is blocked (thus unable to receive messages) when I'm stepping the simulation via the Gazebo client GUI.

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


Did not reproduce for me with or without my plugin in Gazebo 7. Appears as though it may be fixed in 7.

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


Yes, I just reproduced it in 6.5.1, I'll see if I can track down what the fix was.

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


Looks like it was fixed in d0f39a7 from pull request #2085

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


Would you like this back-ported?

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


That would be superb! If you can do that, thanks a lot!

We'd simply use Gazebo 7, however it doesn't work well with our large scale camera use in virtual machines. In 6 we're just fixing the Z ordering issue (#1837), whereas in 7 it causes crashes.

@osrf-migration
Copy link
Author

Original comment by Xander Dunn (Bitbucket: xanderdunn).


In light of this fix, it's less critical to backport this fix to 6. Cameras don't crash when run in a virtual machine anymore, so we'll likely be able to use Gazebo 7.

@osrf-migration
Copy link
Author

Original comment by Ian Chen (Bitbucket: Ian Chen, GitHub: iche033).


I think the issue is critical enough to be backported, so backported to gazebo5 in pull request #2272 and will be merged forward from there.

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


  • changed state from "new" to "resolved"

issue has been resolved in gazebo5 and up and fix will be included in next releases

@osrf-migration
Copy link
Author

Original comment by Nate Koenig (Bitbucket: Nathan Koenig).


  • changed state from "resolved" to "closed"

@osrf-migration
Copy link
Author

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).


The fix for this may have caused #2048.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.0 bug Something isn't working major msgs
Projects
None yet
Development

No branches or pull requests

1 participant