gazebo.insert_world_from_sdf fails to download models from fuel #296
Comments
|
We have partial Fuel support and we currently use and test the programmatic loading of models from fuel: A use-case we never yet explored is loading an SDF file that has models specified as fuel resources. I suspect this could be related to #168 that got stale. Please note that the world file you posted has a lot of plugins in it. You can find the default empty world that we use here. The idea behind gym-ignition is to interact with the simulator programmatically (for instance by populating it from code). This is also the logic we followed for plugins, as reported in the For initial testing, I would suggest to start from the default empty world (that is loaded by default if you don't specify any world sdf). Then, you can 1) insert the models from Fuel and 2) and loading the plugins you need through the Python APIs. Then, if there's the need to support loading structured world by default, we can discuss of the actions to take to add the support. I don't see major problems, it should be doable with minor modifications. |
I really like this idea, and it is the main reason I chose to work with I will try adding the models step-by-step in python and see how far this gets me.
Looking at the bigger picture, I think full At least to me, this seems like a quite clean solution to the problem; though it may not actually add any unique new features. I checked the existing stale PR that you mentioned, and I hope to get the opportunity to pick it up some time next week. Also, this issue does seem like a dubplicate of #168 at first glance. |
|
I explored a little further, and it seems that the function from scenario import gazebo as scenario_gazebo
import gym_ignition_models
import time
gazebo = scenario_gazebo.GazeboSimulator(
step_size=0.001,
rtf=1.0,
steps_per_run=1
)
gazebo.initialize()
world = gazebo.get_world()
world.insert_model(gym_ignition_models.get_model_file("ground_plane"))
gazebo.gui()
gazebo.run(paused=True)
time.sleep(60)
gazebo.close()which opens the simulator showing an empty world with ground plane (as expected). I then saved the created world using the GUI both with tags expanded and not exapanded. generated_world.sdf<sdf version='1.7'>
<world name='default'>
<physics default='1' type='dart'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>-1</real_time_update_rate>
</physics>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>1 1 1 1</diffuse>
<specular>0.5 0.5 0.5 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
<spot>
<inner_angle>0</inner_angle>
<outer_angle>0</outer_angle>
<falloff>0</falloff>
</spot>
</light>
<gravity>0 0 -9.8</gravity>
<magnetic_field>6e-06 2.3e-05 -4.2e-05</magnetic_field>
<atmosphere type='adiabatic'/>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<plugin name='__default__' filename='__default__'/>
<include>
<uri>file:///home/sebastian/.local/lib/python3.8/site-packages/gym_ignition_models/ground_plane</uri>
<name>ground_plane</name>
<pose>0 0 0 0 -0 0</pose>
</include>
</world>
</sdf>generated_world_expanded.sdf<sdf version='1.7'>
<world name='default'>
<physics default='1' type='dart'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>-1</real_time_update_rate>
</physics>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>1 1 1 1</diffuse>
<specular>0.5 0.5 0.5 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
<spot>
<inner_angle>0</inner_angle>
<outer_angle>0</outer_angle>
<falloff>0</falloff>
</spot>
</light>
<gravity>0 0 -9.8</gravity>
<magnetic_field>6e-06 2.3e-05 -4.2e-05</magnetic_field>
<atmosphere type='adiabatic'/>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<plugin name='__default__' filename='__default__'/>
<model name='ground_plane'>
<static>1</static>
<link name='link'>
<collision name='collision'>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>1 1</size>
</plane>
</geometry>
<surface>
<friction>
<ode>
<mu>100</mu>
</ode>
</friction>
<bounce>
<restitution_coefficient>1</restitution_coefficient>
</bounce>
</surface>
</collision>
<visual name='visual'>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>100 100</size>
</plane>
</geometry>
<material>
<ambient>0.8 0.8 0.8 1</ambient>
<diffuse>0.8 0.8 0.8 1</diffuse>
<specular>0.8 0.8 0.8 1</specular>
</material>
<plugin name='__default__' filename='__default__'/>
</visual>
</link>
<plugin name='__default__' filename='__default__'/>
<pose>0 0 0 0 -0 0</pose>
</model>
</world>
</sdf>Finally, I tried to load the generated worlds, expecting to see the same world as the one created procedually via from scenario import gazebo as scenario_gazebo
import gym_ignition_models
import time
scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_info)
gazebo = scenario_gazebo.GazeboSimulator(
step_size=0.001,
rtf=1.0,
steps_per_run=1
)
gazebo.initialize()
gazebo.insert_world_from_sdf("./generated_world.sdf")
# gazebo.insert_world_from_sdf("./generated_world_expanded.sdf")
gazebo.gui()
gazebo.run(paused=True)
time.sleep(3)
gazebo.close()However, in both cases the result was an empty world. result_world.sdf<sdf version='1.7'>
<world name='default'>
<physics default='1' type='dart'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>-1</real_time_update_rate>
</physics>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>1 1 1 1</diffuse>
<specular>0.5 0.5 0.5 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
<spot>
<inner_angle>0</inner_angle>
<outer_angle>0</outer_angle>
<falloff>0</falloff>
</spot>
</light>
<gravity>0 0 -9.8</gravity>
<magnetic_field>6e-06 2.3e-05 -4.2e-05</magnetic_field>
<atmosphere type='adiabatic'/>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<plugin name='__default__' filename='__default__'/>
</world>
</sdf>@diegoferigo would you happen to have a working example of |
|
This is expected, the world must be set before initializing the simulator. During initialization, the default world is loaded if a custom world is not specified. We have some CI test for these core features. I was pretty sure there should be an error printed to the console in this case, but apparently the log level we used is wrong: We should have scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_info)For how the C++ APIs are structured, these methods return # Exceptions
if not gazebo.initialize():
raise RuntimeError
# Assertions: be careful that in python asserts are disabled when running with -O
assert gazebo.initialize()
# Safer assertions
ok_initialize = gazebo.initialize()
assert ok_initialize
I don't how what happened here. I tried on my setup and the exported world is the following, and it seems correct. Is your output really the content of your file? Do you have the same output also if you export the world opening the simulator with exported.sdf<sdf version='1.7'>
<world name='default'>
<physics default='1' type='dart'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>-1</real_time_update_rate>
</physics>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>1 1 1 1</diffuse>
<specular>0.5 0.5 0.5 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
<spot>
<inner_angle>0</inner_angle>
<outer_angle>0</outer_angle>
<falloff>0</falloff>
</spot>
</light>
<gravity>0 0 -9.8</gravity>
<magnetic_field>6e-06 2.3e-05 -4.2e-05</magnetic_field>
<atmosphere type='adiabatic'/>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<plugin name='__default__' filename='__default__'/>
<model name='ground_plane'>
<static>1</static>
<link name='link'>
<collision name='collision'>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>1 1</size>
</plane>
</geometry>
<surface>
<friction>
<ode>
<mu>100</mu>
</ode>
</friction>
<bounce>
<restitution_coefficient>1</restitution_coefficient>
</bounce>
</surface>
</collision>
<visual name='visual'>
<geometry>
<plane>
<normal>0 0 1</normal>
<size>100 100</size>
</plane>
</geometry>
<material>
<ambient>0.8 0.8 0.8 1</ambient>
<diffuse>0.8 0.8 0.8 1</diffuse>
<specular>0.8 0.8 0.8 1</specular>
</material>
<plugin name='__default__' filename='__default__'/>
</visual>
</link>
<plugin name='__default__' filename='__default__'/>
<pose>0 0 0 0 -0 0</pose>
</model>
</world>
</sdf>
I totally agree with you, it would be nice to have full SDF support. I could also think to add an option Beyond the sdf templating that you proposed, that is an interesting practice (and quite easy to do in Python), another great reason to fully support complex worlds with Fuel resources is the new model composition logic of SDF. Now it is possible to insert models specifying poses relative to other models, and this greatly simplifies the creation of dynamic worlds that adapt automatically if any of the |
Yes! I remember having seen a comment about this in the source code when skimming it to figure out how to use Do you think we should update the documentation with a comment or note in the form of I don't know how the current python bindings look like, but an additional step could be to raise an appropriate exception (perhaps
I moved from scenario import gazebo as scenario_gazebo
import time
scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_debug)
gazebo = scenario_gazebo.GazeboSimulator(
step_size=0.001,
rtf=1.0,
steps_per_run=1
)
# Note: I also tried the absolute path
assert gazebo.insert_world_from_sdf("./generated_world.sdf")
gazebo.initialize()
gazebo.gui()
gazebo.run(paused=True)
time.sleep(3)
gazebo.close()The
I'm not sure I follow here. What would be the danger of having a plugin loaded that isn't explicitly loaded by the script? In either case, this does seem like a useful feature to be able to turn this on/off. On top of the benefits you mentioned, the |
Yes, beyond fixing the verbosity of the error message, also the documentation could be improved.
We discussed this long time ago, and we decided to keep a 1:1 APIs compatibility with C++. ScenarI/O was born as C++ library, and we are used with the return-bool pattern among our projects. Then, the Python bindings inherited that, even though we are aware that using exceptions is more idiomatic to Python. However, mapping the C++ return-bool to Python exception is not directly doable with SWIG without duplicating all the methods (big work to implement it, huge maintenance effort, complete breaking of APIs). We recently explored in other projects the usage of pybind11, and discovered that they can be also used from Matlab that is the only other language that we could interests us, but we have no short-term plan to switch.
I suspect that you have to pass the absolute location of the file. The relative
I see your point. The main problem is that we aim to provide APIs that substitute many features that are typically implemented with plugins. One example is the
Levels are among those features that make Ignition Gazebo stand out from the crowd, they're very cool. It should not require much work to support them. I think they're kind of transparent from our point of view, maybe they already work, I haven't had the chance to try them. Something that could be challenging to handle is how to treat models that are not part of the level from our APIs. I guess that we can return a |
I'll check if I can submit a PR to update the documentation after I fixed the loading on my side.
I thought this could have been an issue, too. Unfortunately, using an absolute path doesn't change anything. Similarly, using the environment variable
I wonder if an
This is an interesting note; is there some documentation/information on this that I can read up on? Is the simulation loop of Ignition (or the physics engine) itself non-deterministic? In my mind, as long as you control the simulator's main loop, you control the simulation. An approach like # pseudo-code
while sim_not_done:
# step the simulator
simulator.step()
# handle the generated messages/events indesired order
camera_observation = camera_topic.recv()
imu_observation = imu_topic.recv()
# ...
# compute and apply controls
# ...
control_topic.send(control_msg)should be quite deterministic, at least in the current version of Ignition. Messages are sent over the local loopback device and via TCP, which should guarantee order and reception of messages. In addition, all callbacks are handled sequentially in the same thread (hello, GIL) and control messages (responses) are queued and not handled by the simulator until the next spin of the main loop. I'm wondering if I am missing something in my thinking. |
|
Update: This does not appear to be a file reading problem. I found that which can be enabled via the environment variable |
Great, thanks!
I was about to suggest checking that :) Can you post the sdf file and the python script you're using? I'll try reproduce it locally.
I didn't dive in the code, I suspect that happens when sdformat tries to open the file (maybe using ign-common?). If you have a Debug version of Ignition (you can use colcon), I think that their logs also include the line that originated the message.
Yep, simple addition as well. As you noticed, sdformat already has all the APIs. The new method signature could be the following: bool insertModelFromString(const std::string& modelString,
const core::Pose& pose = core::Pose::Identity(),
const std::string& overrideModelName = {});
TCP guarantees that the message is sent and received. However, the process is asynchronous with the caller script. You cannot be sure that all messages have been received remotely when you call The workaround is substituting TCP messages with RPC messages, and Ignition Transport supports them. I'm not expert, but they are a different type of message, you cannot "switch" backend from TCP to RPC. The only real way to ensure determinism under any load is interfacing the simulator from shared memory as we do in gym-ignition. |
Thanks! simulator.pyfrom scenario import gazebo as scenario_gazebo
import time
scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_debug)
gazebo = scenario_gazebo.GazeboSimulator(
step_size=0.001,
rtf=1.0,
steps_per_run=1
)
assert gazebo.insert_world_from_sdf("./generated_world.sdf", "generated_world")
gazebo.initialize()
gazebo.gui()
gazebo.run(paused=True)
time.sleep(3)
gazebo.close()generated_world.sdf<sdf version='1.7'>
<world name='default'>
<physics default='1' type='dart'>
<max_step_size>0.001</max_step_size>
<real_time_factor>1</real_time_factor>
<real_time_update_rate>-1</real_time_update_rate>
</physics>
<light name='sun' type='directional'>
<cast_shadows>1</cast_shadows>
<pose>0 0 10 0 -0 0</pose>
<diffuse>1 1 1 1</diffuse>
<specular>0.5 0.5 0.5 1</specular>
<attenuation>
<range>1000</range>
<constant>0.9</constant>
<linear>0.01</linear>
<quadratic>0.001</quadratic>
</attenuation>
<direction>-0.5 0.1 -0.9</direction>
<spot>
<inner_angle>0</inner_angle>
<outer_angle>0</outer_angle>
<falloff>0</falloff>
</spot>
</light>
<gravity>0 0 -9.8</gravity>
<magnetic_field>6e-06 2.3e-05 -4.2e-05</magnetic_field>
<atmosphere type='adiabatic'/>
<scene>
<ambient>0.4 0.4 0.4 1</ambient>
<background>0.7 0.7 0.7 1</background>
<shadows>1</shadows>
</scene>
<plugin name='__default__' filename='__default__'/>
<include>
<uri>file:///home/sebastian/.local/lib/python3.8/site-packages/gym_ignition_models/ground_plane</uri>
<name>ground_plane</name>
<pose>0 0 0 0 -0 0</pose>
</include>
</world>
</sdf>
and after calling it via console output
Actually,
This could make the PR to add fuel support redundant (maybe?).
Actually, For sending, you are right, it is indeed safer to assume that sending control commands to existing plugins is non-deterministic, as |
|
Ok I think I know what's happening. Try to add In order to draw the scene, the server has to send data to the GUI. This happens only when the ECM (that is the database containing all the simulated resources like world, models, ...) gets invalidated. This happens for instance if a new model is added, or a link pose is changed due to the physics running. In your case, the world is built from the SDF, and when the GUI opens, this first message to draw the world is not sent. If you edit your code as follows, you'll see that the GUI will show the scene: simulator.pyfrom scenario import gazebo as scenario_gazebo
import time
scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_debug)
gazebo = scenario_gazebo.GazeboSimulator(
step_size=0.001,
rtf=1.0,
steps_per_run=1
)
assert gazebo.insert_world_from_sdf("./generated_world.sdf", "generated_world")
gazebo.initialize()
print(gazebo.world_names())
world = gazebo.get_world()
# Add any model, e.g. the panda robot
import gym_ignition_models
world.insert_model(gym_ignition_models.get_model_file(robot_name="panda"))
# You'll see both the ground_plane from the sdf and the panda from APIs
gazebo.gui(True)
gazebo.run(paused=True)
time.sleep(3)
gazebo.close()There's no easy workaround to it. I remember last year I spent quite some time to enhance the interaction between server and gui, without much luck. I ended up implementing
I fear that manipulating the SDF is still necessary. We have to pass through the SDF to set the simulator parameters.
Interesting. I have zero experience with ROS2 (what I assume you're using, guessing from zmq). If those messages are blocking and not asynchronous, yes, you get determinism. |
|
Yes! That indeed fixes the problem 🚀 It's a bit odd that the GUI can't query the full state when it first starts; it sounds suspiciously like a classic late subscriber problem forcing the GUI to sit there waiting for the first message instead of being able to request the current state when it first starts (service call). Camera sensors are hopefully unaffected and - since I will run the simulation GUI-less once it works - this is a good enough solution for me for now. I will give this more thought once I get around to the fuel-support PR because it would be a tad annoying to template the world as
That's unfortunate. I have no specific idea yet, but if it is "only" physics - and more specifically step size - that prevents us from delegating I actually have no ROS dependency (yet). A problem that I will likely have to solve down the line with this is differing publication rates of sensors. If the sensor doesn't send a message at every simulation step, the loop will get stuck waiting for sensor data that was never sent (it's a blocking call). It also messes with the If I do manage to get it working within gym-ignition, I will leave a comment in #287 and #199. |
Thanks @diegoferigo and @FirefoxMetzger for the interesting discussion. As @diegoferigo already mentioned, it is perfectly possible to have network communication and deterministic simulation: a whole part of co-simulation field (https://en.wikipedia.org/wiki/Co-simulation, Sectio C.1.6 of https://arxiv.org/pdf/1702.00686.pdf) is exactly about that, and standard such as the Distributed Co-Simulation Protocol has been developed for that. Unfortunately most Pub/Sub middlewares used in robotics have been designed with other use cases in mind, and so achieving determinism with them is not trivial, but @FirefoxMetzger feel free to keep us posted on your effort! |
|
For future reference:
This will merge all the (downloaded) include tags into one file and save that. Downloading meshes and textures from fuel seems to work, and This workaround doesn't cover all use-cases, but I thought I'll add it here in case it may be useful for others. |
We use the simulator in a way that differs from upstream. Even when the simulation is paused, when you open
One of the clean ways would be to add the missing value in the
Ok now I understood my confusion and this is clear, especially after reading #287 (comment). The workaround is cool and offer a nice flexibility. As reported in #296 (comment), only receiving data could maintain determinism, but it's a good alternative that could be used in absence of sensor API in ScenarIO. As you noticed there, if the data rate is lower than the rate of which the simulation is stepped in the code, things could get stuck if the rate differ. A possible solution would be performing all the receiving operation in a slower thread with proper synchronization mechanism (a simple lock on the received image object would suffice). Being and I/O operation, I think it does not affect performance due to the GIL. |
It will block; however if one does the math correctly this can be accounted for. I'm doing this in the simulation I'm building now for camera images, but it should work with any sensor. If determinism is the end-goal I think getting stuck is a desirable feature, because it fails noisily. Potentially, this could be improved via a long (>1 s) timeout + an exception to produce a stack trace pointing to the sensor that was not in sync. This is the the relevant snippet # get synced camera images (published at 30Hz)
if (int(round(sim_time/gazebo.step_size()))) % steps_per_frame == 0:
zmq_msg = camera_topic.recv() # blocking call
image_msg = Image()
image_msg.ParseFromString(zmq_msg[2])
im = np.frombuffer(image_msg.data, dtype=np.uint8)
im = im.reshape((image_msg.height, image_msg.width, 3))
img_time = image_msg.header.stamp.sec + image_msg.header.stamp.nsec*1e-9
# ensure image message and simulator are in sync
assert sim_time == img_timeFull source: https://github.com/FirefoxMetzger/panda-ignition-sim/blob/master/simulator.py |
|
Closed via #309 |
I am trying to load an existing world from an
.sdffile into gazebo usinggym-ignition. Here is my codeand the coresponding
world.sdfworld.sdf
The world opens fine in gazebo itself (
ign gazebo world.sdf), but when I run it via the script I get the following error messages:I'm still learning about both
ignitionandgym-ignition, so this could very well be user-error on my part. Any ideas where things fall apart and where I should look to fix it? I'm happy to do a PR (if necessary) once a solution has been identified.Edit: I should add that I tried changing to the suggested
model://https://...URI, but this results in an error stating that the resource could not be found.The text was updated successfully, but these errors were encountered: