Perception Module Defective in AutowareAI with SVL

[marcusvinicius178]

Hi @EricBoiseLGSVL @heeen @lemketron would you know how to diagnose the origin of the failure in the perception module of AutowareAI while using the LGSVL simulator?

The whole description of my issue is here: https://github.com/Autoware-AI/autoware.ai/issues/2379

I am not expert on Lidar, and its respective filters that feed the occupancy grid map on AutowareAI to determine if there is an obstacle or not in front of the car in rviz.

As the AWF team still didn't answer me this question and always close the issues when I ask this specific question, I believe that maybe the issue can be a kind of bad alignment with the SVL sensors? I mean set the frequency correctly to send/receive messages or something similar?

It is just a guess, I have no idea why the peception is not working...I tried numerous times on numerous different powerful machines....and this module rarely works.

A recent attempt is here on youtube video below in CubeTown map. Starts from minute 38 to minute 41:
https://www.youtube.com/watch?v=w6_beZtlLBg

The Ego vehicle hits the NPC car. and the occupancy grid map last a LOT of time! to be generated (you can check I tried about 40 minutes of video to do it!

So it is my last attempt, maybe you can have some solution for me.
For Apollo there is the modular option on perception, so maybe for Autoware_AI it could exist other solution?

On my master thesis I am comparing the ADS (Apollo vs Autoware) to accomplish evasive maneuvers on different scenarios: https://youtu.be/mSIJ8c_oeQo

Actually the Apollo will avoid the obstacles from video above and I would like Autoware also recognized them to avoid them...

However I cannot finish the research, if Autoware does not recognize the obstacle and therefore the planner will never work to avoid it, as it is a subsequent step.

[EricBoiseLGSVL]

Sorry that you are not getting a response from AWF. I think there is the issue of AutowareAI being older but not sure. I'm guessing you would need to be able to edit the code. Have you tried Autoware.Auto?

[marcusvinicius178]

Yes previously the perception on Autoware.Auto is better however as I described in previous issue there was having issues to launch Autoware.Auto with the new SVL releases. It was working just with the LGSVL 2020.06 however I have checked that this Version is not more available

Do you know if now the Autoware.Auto is already working with the new SVL releases? Is there new tutorial for this synchronazation? I was trying and it was not working (some issues with the bridge receiving and sending data) ... I mean Autoware.Auto AVP demo was not working

[EricBoiseLGSVL]

We are working with autoware to make sure all features are working. I am not sure on what demo they have but our documentation should be used for AA integration. Please post any issues you have and we will try to resolve. Adding @lemketron

[marcusvinicius178]

OK I am following here: https://gitlab.com/autowarefoundation/autoware.auto/AutowareAuto/-/issues/977
It seems they are working to make Autoware.Auto to work with the LGSVL 2021.1 release.

However I does not know if is it going to work with the newer SVL releases (2021.1.1, 2021.1.2, 2021.2.1, 2021.2.2) ??

I really need to understand which SVL version, the AutowareAuto will be compatible, because I need to standardize my computers and my simulations and the ADS frameworks (AutowareAI, AutowareAuto and APolloAUTO). Otherwise I will never finish my simulations, because every time I have an issue with one ADS framework vs SVL release communication :(

What do you suggest me @EricBoiseLGSVL use the 2021.1 SVL release for all my ADS frameworks described above? Is it the more stable and will be maintained at least for next month?

I tried to use the new SVL 2021.2.1 release on a computer that had previously installed the old release: 2021.1 . Unfortunately I had similar issues as we discussed previously: #1646

I repeated the steps of the issue we discussed about, however now it is not working. Some cache memory in
.local/lib/python3.8/site-package/environs/pycache/init_.cpython-38pyc is kept

Also issues with the lgsvl-simulator-scenarios-runner-simulator-build_2021.2.1 tar no such file or directory

I have already remove the LGEletronics folder inside .config/unity3D folder however the issue persists.

How can I fix this definetly? I mean there must be a way to clean all cache and re-start from scratch using a new SVL simulator without having these issues every time when switching to another SVL release?

[marcusvinicius178]

@EricBoiseLGSVL in addition:

I made it work again, but I still have one doubt
(I am running the SVL on Desktop and the Autoware/Apollo on Notebook)

With PythonAPI template is easy to configure the communication between the simulator host and the ADS automatically. I just add on Autopilot the address:

187.95.100.99: 9090

However to run this with Python Only template, how should I configure the lines below @lemketron on the script ?

LGSVL__SIMULATOR_HOST = env.str("LGSVL__SIMULATOR_HOST", "127.0.0.1")#Address IP from simulator on Desktop
LGSVL__SIMULATOR_PORT = env.int("LGSVL__SIMULATOR_PORT", 8181)
LGSVL__AUTOPILOT_0_HOST = env.str("LGSVL__AUTOPILOT_0_HOST", "187.95.100.99")#Address IP from Desktop
LGSVL__AUTOPILOT_0_PORT = env.int("LGSVL__AUTOPILOT_0_PORT", 9090) #Bridge: ROS or CYBER


sim = lgsvl.Simulator(os.environ.get("SIMULATOR_HOST", "187.95.100.99"),8181)

When I was running both simulator and ADS (Apollo or Autoware) on the same Notebook I was just using the HOST: 127.0.0.1"

Now I changed to 187.95.100.99 in AutopilotHost as depicted:

The script starts however the simulator window does not display the environment

And I get these errors:

If I keep in the last line the address 127.0.0.1, however, the simulator opens the environment (CubeTown), but closes and raise the error: Bridge was unexpectdely closed

The bridge is running on notebook side, I did not forget to activate it...

[marcusvinicius178]

I have already ping ip from notebook and from Desktop and they are communicating with each other....
Now the simulator does not close if I use PythonAPI template
However it does not get/send image through the bridge

How must I configure these lines

API Template

and autopilot

The public ip is: 187.95.110.170
The notebook private ip where Cyber/ROS bridge run is: 10.42.0.1
The Desktop private ip where SVL 2021.2.1 run is: 10.42.0.110

I just wish to know how to configure these lines, that by default (when running both simulator and ADS on same pc) are:

LGSVL__SIMULATOR_HOST = env.str("LGSVL__SIMULATOR_HOST", "127.0.0.1")
LGSVL__SIMULATOR_PORT = env.int("LGSVL__SIMULATOR_PORT", 8181)
LGSVL__AUTOPILOT_0_HOST = env.str("LGSVL__AUTOPILOT_0_HOST", "127.0.0.1")
LGSVL__AUTOPILOT_0_PORT = env.int("LGSVL__AUTOPILOT_0_PORT", 9090)

sim = lgsvl.Simulator(os.environ.get("SIMULATOR_HOST", "127.0.0.1"),8181)

I have aleady tried lot of configuration but the ego does not connect to the bridge for some reason...

Some hint please @cyberphysicalrobotics @lemketron @EricBoiseLGSVL @heeen

[marcusvinicius178]

Hi I did it!

For PythonAPI template the solution is set everything to the private ip address

And also the autopilot used the private ip:

However for the PYTHON ONLY TEMPLATE it still does not work.....why??

What is wrong in this code?? What must be changed ??

The only difference is that PythonAPI template uses the Autopilot bridge interface, shouldn't this work also for the OnlyPython template scripts ???

[EricBoiseLGSVL]

can you screen shot the bridge you are using in wise? What is the guid of the vehicle sensor config?

[marcusvinicius178]

Hi @EricBoiseLGSVL sure. I will copy and paste all the bridge related code below:

ego = sim.add_agent(env.str("LGSVL__VEHICLE_0"), lgsvl.AgentType.EGO, ego_state)
ego.connect_bridge(env.str("LGSVL__AUTOPILOT_0_HOST", lgsvl.wise.SimulatorSettings.bridge_host), env.int("LGSVL__AUTOPILOT_0_PORT", lgsvl.wise.SimulatorSettings.bridge_port))
print("Waiting for connection...")
while not ego.bridge_connected:
    time.sleep(1)

print("Bridge connected:", ego.bridge_connected)

I believe the line you wish is this: **ego.connect_bridge(env.str("LGSVL__AUTOPILOT_0_HOST", lgsvl.wise.SimulatorSettings.bridge_host), **

Should I change something just here to work directly with PythonOnly scripts template?

Thanks in advance

[EricBoiseLGSVL]

I feel like there is an issue with the PythonOnly runner and some default assets. Pinging @revati-naik and @eagafonovlge on this

[marcusvinicius178]

Hi again @EricBoiseLGSVL
Just to return the solution for the perception module if some other user needs:

The tutorials that I have followed for m AutowareAi in are WRONG! If you launch the costmap_generator through the runtime manager or terminal, the occupancy grid map will not be generated or very very rarely....
To fix this there are 2 options:
1). Using Run-time Manager: on Computing Tab click over app option in costmap_generator, scroll down until you find the Lanelet2 checkbox option at the left bottom. CHECK this box ✔️
2) Inside the docker, using a new terminal launch:
roslaunch costmap_generator costmap_generator_lanelet2.launch
Ok now the costmap_generator will generate the occupancy_grid_map
3) Just Pay attention that you must check the ray_ground filter or compare_map_filter on Sensing Tab

However this will not help as the A* keeps not avoiding and crashing the Obstacle, as described in my issue: https://github.com/Autoware-AI/autoware.ai/issues/2387

If someone fix this A* please let me know

[EricBoiseLGSVL]

Thanks @marcusvinicius178 pinging @cyberphysicalrobotics and @revati-naik on this solution.