Solution sharing withing RL (all tiers) ,

[ugurkanates]

Hey I assume competition over at this point(21th november 2019) was last entry date. If not gonna close this issue until its over. I'm sure most people like me did try and develop somethings within challenge but couldn't complete due work - time balance.

If anyone was completed whole track even if its for tier 1 and training qualifications could you share some solutions so we can try and modify etc..

Or maybe actual race team/competition team can share a base solution as well ?

[yannbouteiller]

Well, I have been trying very hard the pure RL approach and I think I still need at least one month before my drones even start completing the track on tier 3, I would be astonished if other people qualified with the same approach, although I'm still convinced it is good one. However, from this point, what I miss is mainly computation power, and I am still working on deploying my framework efficiently on a HPC. I think it would have been relatively easy to make it to the live competition for teams with a good experience in the field.

[ugurkanates]

Were you able to complete tracks on Tier 1 ? if so maybe you can share that approach and people will contribute on your work (assuming you have permitted of course)

or maybe if AirSim dev guys maybe have baseline RL versions as well to share. I have PPO based version for Tier 1 but yet to complete track

[yannbouteiller]

We didn't try much on tier 1, just used it to give us ideas for tier 3. Also, we used a simple MLP on tier 1, and even with batchnorm it ends up getting nan weights at some point during training. Could probably have been avoided by clipping gradients the right way but we didn't investigate that because we are not really interested in tier 1. What we do have however is an advanced RL framework adapted to the competition for all tiers, we will continue working on that, make it useful for more general AirSim tasks (while not re-doing the same thing as AirLearning), open-source it and maybe write a small publication in a while when we get results that are worth showing.

[ugurkanates]

Interesting thanks for results yann , I also wish I had more time since we formed a team from r/ReinforcementLearning channel and had 20 or more people about RL who joined/and out about project. Project management is hard stuff in end we were up to 2 developers though this more depends on how hard was to setup project in early days . Thinks like Amazon deepracer which handles all hard stuff about enviroment etc is more useful for RL research. This project requires lots of hardware related setup and before docker which was super useful but kinda late it was hard to do it for most people(well i just did instructions but for different hardware it can give errors etc)

So that's why i was wondered in Tier 1 which seems to be easiest of all
I'm also curious about @madratman comments on this
(and personal thanks for docker setup which was pain in the ass for me like beginner in Docker)

[ugurkanates]

Any news on this @madratman

[ugurkanates]

I turned my repository to public (it contains a PPO based solution though its not perfect or anything)
Maybe would give people a head start and interest

https://github.com/ugurkanates/NeurIRS2019DroneChallengeRL

[madratman]

I added the workshop talks / slides on the website https://microsoft.github.io/AirSim-NeurIPS2019-Drone-Racing/

I'll upload the reports as soon as I've consent from all teams.

[madratman]

In addition, please feel free to check out https://github.com/microsoft/AirSim-Drone-Racing-VAE-Imitation

[Sohojoe]

@ugurkanates - thanks for sharing your work - I was trying to go the pure RL approach, but it was a real pain just setting it up as a gym environment