CCB: Fix potential deadlock in PolicyHandler

[LitvinenkoIra]

Bug Report

Potential deadlock in PolicyHandler

Detailed analysis:

There is a vulnerability in the PolicyHandler which causes a mutex deadlock. For example - MessageLoop thread of RpcService handles incoming messages. In case when SDL receives AllowSDLFunctionality notification, this thread calls OnAllowSDLFunctionalityNotification inside PolicyHandler. At some point of time this function captures accessor from AM which holds applications_list_lock_ there. At this moment thread AM Pool 0 of RequestController processes some RPC from queue and captures policy_manager_lock_ in PolicyHandler. After that at
some moment thread AM Pool 0 tries to get application shared pointer from AM and locks itself as this mutex are already locked with thread MessageLoop. Also, MessageLoop thread at some moment tries to acquire policy_manager_lock_ and locks itself as this mutex are already locked with thread AM Pool 0, which is waiting for applications_list_lock_ to unlock. As a result we have a
classical thread deadlock after which SDL stuck forever.

To avoid such situations, all bottlenecks related to
applications_list_lock_ and its accessors should be analyzed . Accessors should be scoped in several places to avoid similar deadlocks in the future.

Reproduction Steps

1. Start SDL, HMI, connect Mobile
2. Trigger PTU
3. Send VR.ChangeLanguage

Expected Behavior

SDL should not crash

Observed Behavior

SDL crashed

OS & Version Information

• OS/Version : SYNC4
• SDL Core Version: https://github.com/smartdevicelink/sdl_core/releases/tag/5.0.1
• Testing Against: SYNC4 Applink

[E-SAITO-TMC]

@joeygrover -san @theresalech -san This issue is a top priority for Toyota. Can the reproduction steps described actually reproduce this issue? Also, does this issue occur in the latest version?

[theresalech]

Hi @E-SAITO-TMC, we have documented that this is priority for Toyota, and will include for Steering Committee review and vote when planning the contents for the Q4 Core release. This planning will take place in June.
@LitvinenkoIra, can you please advise if the Luxoft team is seeing this issue in Core 6.1?

[LuxoftAKutsan]

@theresalech
Issue reproduced on SYNC version of SDL.
On open source version of SDL issue may be non reproducible.
Script will be created.
Fix from SYNC ported to open #3383

[jacobkeeler]

Closing with the merge of #3383