Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix session allocation loop when shutting down with an open commissioning window. #20715

Conversation

bzbarsky-apple
Copy link
Contributor

After #20487 if we shut down with a commissioning window open we end up in a
loop where the session manager shutdown marks the tentative PASE session for
eviction, we treat that as a commissioning error and start listening for PASE
again, creating a new session, etc. With a heap pool this ends up happening to
work in that we keep evicting the new sessions until we hit the 20-attempt limit
and close the commissioning window. With a non-heap pool, I sort of wonder what
happens, exactly.

The fix here is in two parts, with either part enough on its own to fix the
behavior described above:

  1. Shut down the commissioning window manager earlier, before we shut down the
    session manager. And correspondingly move its initialization during server
    init later.

  2. Once session manager starts shutdown, refuse to create any new sessions.

Problem

Stacks like this if I start all-clusters-app in uncommissioned state and then Ctrl-C it:
  * frame #0: 0x00000001005c8060 chip-all-clusters-app`chip::Transport::SecureSession::SecureSession(this=0x0000611000032040, table=0x00000001008883d0, secureSessionType=kPASE, localSessionId=15119) at SecureSession.h:92:5
    frame #1: 0x00000001005c800b chip-all-clusters-app`chip::Transport::SecureSession* chip::Platform::New<chip::Transport::SecureSession, chip::Transport::SecureSessionTable&, chip::Transport::SecureSession::Type&, unsigned short&>(args=0x00000001008883d0, args=0x00007ff7bfefd5c0, args=0x00007ff7bfefd5f2) at CHIPMem.h:148:24
    frame #2: 0x00000001005c50a3 chip-all-clusters-app`chip::Transport::SecureSession* chip::HeapObjectPool<chip::Transport::SecureSession>::CreateObject<chip::Transport::SecureSessionTable&, chip::Transport::SecureSession::Type&, unsigned short&>(this=0x00000001008883d8, args=0x00000001008883d0, args=0x00007ff7bfefd5c0, args=0x00007ff7bfefd5f2) at Pool.h:324:22
    frame #3: 0x00000001005c45b1 chip-all-clusters-app`chip::Transport::SecureSessionTable::CreateNewSecureSession(this=0x00000001008883d0, secureSessionType=kPASE, sessionEvictionHint=(mNodeId = 0, mFabricIndex = '\0')) at SecureSessionTable.cpp:76:30
    frame #4: 0x00000001005d4dde chip-all-clusters-app`chip::SessionManager::AllocateSession(this=0x0000000100888378, secureSessionType=kPASE, sessionEvictionHint=0x00007ff7bfefdac0) at SessionManager.cpp:389:28
    frame #5: 0x00000001006d35d0 chip-all-clusters-app`chip::PairingSession::AllocateSecureSession(this=0x000000010088a730, sessionManager=0x0000000100888378, sessionEvictionHint=0x00007ff7bfefdac0) at PairingSession.cpp:28:34
    frame #6: 0x00000001006bec54 chip-all-clusters-app`chip::PASESession::Init(this=0x000000010088a730, sessionManager=0x0000000100888378, setupCode=0, delegate=0x000000010088a700) at PASESession.cpp:118:5
    frame #7: 0x00000001006bf78b chip-all-clusters-app`chip::PASESession::WaitForPairing(this=0x000000010088a730, sessionManager=0x0000000100888378, verifier=0x00007ff7bfefe280, pbkdf2IterCount=1000, salt=0x00007ff7bfefe470, mrpLocalConfig=Optional<chip::ReliableMessageProtocolConfig> @ 0x00007ff7bfefe490, delegate=0x000000010088a700) at PASESession.cpp:166:22
    frame #8: 0x00000001006ec891 chip-all-clusters-app`chip::CommissioningWindowManager::AdvertiseAndListenForPASE(this=0x000000010088a700) at CommissioningWindowManager.cpp:244:9
    frame #9: 0x00000001006ea318 chip-all-clusters-app`chip::CommissioningWindowManager::HandleFailedAttempt(this=0x000000010088a700, err=(mError = 2, mFile = "../../src/protocols/secure_channel/PASESession.cpp", mLine = 75)) at CommissioningWindowManager.cpp:126:15
    frame #10: 0x00000001006eb06e chip-all-clusters-app`chip::CommissioningWindowManager::OnSessionEstablishmentError(this=0x000000010088a700, err=(mError = 2, mFile = "../../src/protocols/secure_channel/PASESession.cpp", mLine = 75)) at CommissioningWindowManager.cpp:112:5
    frame #11: 0x00000001006be55b chip-all-clusters-app`chip::PASESession::OnSessionReleased(this=0x000000010088a730) at PASESession.cpp:75:16
    frame #12: 0x00000001007067dc chip-all-clusters-app`chip::SessionHolderWithDelegate::SessionReleased(this=0x000000010088a750) at SessionHolder.h:98:19
    frame #13: 0x00000001005e4a79 chip-all-clusters-app`chip::Transport::Session::NotifySessionReleased(this=0x000061100003f9c0) at Session.h:120:31
    frame #14: 0x00000001005bfe9f chip-all-clusters-app`chip::Transport::SecureSession::MarkForEviction(this=0x000061100003f9c0) at SecureSession.cpp:140:9
    frame #15: 0x00000001005e5809 chip-all-clusters-app`auto chip::SessionManager::Shutdown(this=0x00007ff7bfeff100, session=0x000061100003f9c0)::$_0::operator()<chip::Transport::SecureSession*>(chip::Transport::SecureSession*) const at SessionManager.cpp:116:18
    frame #16: 0x00000001005e57ad chip-all-clusters-app`chip::internal::LambdaProxy<chip::Transport::SecureSession, chip::SessionManager::Shutdown()::$_0>::Call(context=0x00007ff7bfeff100, target=0x000061100003f9c0) at Pool.h:126:16
    frame #17: 0x00000001003f92e8 chip-all-clusters-app`chip::internal::HeapObjectList::ForEachNode(this=0x00000001008883e8, context=0x00007ff7bfeff100, lambda=(chip-all-clusters-app`chip::internal::LambdaProxy<chip::Transport::SecureSession, chip::SessionManager::Shutdown()::$_0>::Call(void*, void*) at Pool.h:125))(void*, void*)) at Pool.cpp:127:17
    frame #18: 0x00000001005e56c4 chip-all-clusters-app`chip::Loop chip::HeapObjectPool<chip::Transport::SecureSession>::ForEachActiveObject<chip::SessionManager::Shutdown(this=0x00000001008883d8, function=0x00007ff7bfeff220)::$_0>(chip::SessionManager::Shutdown()::$_0&&) at Pool.h:396:25
    frame #19: 0x00000001005cfb91 chip-all-clusters-app`chip::Loop chip::Transport::SecureSessionTable::ForEachSession<chip::SessionManager::Shutdown()::$_0>(this=0x00000001008883d0, function=0x00007ff7bfeff220)::$_0&&) at SecureSessionTable.h:84:25
    frame #20: 0x00000001005cef72 chip-all-clusters-app`chip::SessionManager::Shutdown(this=0x0000000100888378) at SessionManager.cpp:115:21

Change overview

See above.

Testing

See above about hitting Ctrl-C. With this PR there is no repeated listening for PASE, no

[1657772763786] [55269:28665263] CHIP: [SVR] Commissioning failed (attempt 15): ../../../examples/all-clusters-app/linux/third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:75: CHIP Error 0x00000002: Connection aborted

bits. Just clean shutdown.

…ning window.

After project-chip#20487 if we shut down with a commissioning window open we end up in a
loop where the session manager shutdown marks the tentative PASE session for
eviction, we treat that as a commissioning error and start listening for PASE
again, creating a new session, etc.  With a heap pool this ends up happening to
work in that we keep evicting the new sessions until we hit the 20-attempt limit
and close the commissioning window.  With a non-heap pool, I sort of wonder what
happens, exactly.

The fix here is in two parts, with either part enough on its own to fix the
behavior described above:

1) Shut down the commissioning window manager earlier, before we shut down the
   session manager.  And correspondingly move its initialization during server
   init later.

2) Once session manager starts shutdown, refuse to create any new sessions.
@github-actions
Copy link

github-actions bot commented Jul 14, 2022

PR #20715: Size comparison from 37bb2df to 78c92ec

Increases (21 builds for cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, telink)
platform target config section 37bb2df 78c92ec change % change
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read only) 666275 666291 16 0.0
.text 577820 577836 16 0.0
all-clusters-minimal-app LP_CC2652R7 (read only) 632083 632099 16 0.0
.text 554380 554396 16 0.0
lock-ftd LP_CC2652R7 (read only) 669183 669199 16 0.0
.text 592424 592440 16 0.0
lock-mtd LP_CC2652R7 (read only) 618591 618607 16 0.0
.text 541944 541960 16 0.0
pump-controller-app LP_CC2652R7 (read only) 664255 664263 8 0.0
.text 579208 579216 8 0.0
shell LP_CC2652R7 (read only) 658774 658790 16 0.0
.text 573540 573556 16 0.0
cyw30739 light cyw930739m2evb_01 (read/write) 579614 579638 24 0.0
.app_xip_area 458368 458392 24 0.0
lock cyw930739m2evb_01 (read/write) 585574 585590 16 0.0
.app_xip_area 459600 459616 16 0.0
ota-requestor-no-progress-logging cyw930739m2evb_01 (read/write) 583018 583034 16 0.0
.app_xip_area 462620 462636 16 0.0
efr32 lock-app BRD4161A+wf200 (read/write) 1129096 1129112 16 0.0
.text 982828 982844 16 0.0
esp32 all-clusters-app c3devkit (read only) 1020154 1020178 24 0.0
.flash.text 1020154 1020178 24 0.0
m5stack (read only) 1074031 1074047 16 0.0
.flash.text 1068647 1068663 16 0.0
k32w light k32w061+release (read/write) 658832 658848 16 0.0
.text 581524 581540 16 0.0
lock k32w061+release (read/write) 685684 685700 16 0.0
.text 607900 607916 16 0.0
linux chip-tool-ipv6only arm64 (read only) 10068812 10068908 96 0.0
.text 8026596 8026692 96 0.0
thermostat-no-ble arm64 (read only) 2595668 2595780 112 0.0
.text 2190224 2190336 112 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read/write) 2448312 2448376 64 0.0
.text 1410956 1411020 64 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 text 811544 811552 8 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1155607 1155623 16 0.0
text 800984 800996 12 0.0
telink light-switch-app tlsr9518adk80d (read/write) 797540 797564 24 0.0
text 565734 565758 24 0.0
lighting-app tlsr9518adk80d (read/write) 817364 817388 24 0.0
text 582056 582080 24 0.0
Decreases (14 builds for bl602, cc13x2_26x2, efr32, p6)
platform target config section 37bb2df 78c92ec change % change
bl602 lighting-app bl602 (read/write) 1397634 1397410 -224 -0.0
.text 1058640 1058412 -228 -0.0
bl602+rpc (read/write) 1443066 1442834 -232 -0.0
.text 1090324 1090096 -228 -0.0
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read/write) 184948 184932 -16 -0.0
lock-ftd LP_CC2652R7 (read/write) 172184 172168 -16 -0.0
pump-controller-app LP_CC2652R7 (read/write) 178080 178072 -8 -0.0
shell LP_CC2652R7 (read/write) 187952 187936 -16 -0.0
efr32 lighting-app BRD4161A (read/write) 1081700 1081492 -208 -0.0
.text 946636 946428 -208 -0.0
BRD4161A+rpc (read/write) 1136012 1135820 -192 -0.0
.text 984056 983864 -192 -0.0
BRD4161A+rs911x (read/write) 948172 947980 -192 -0.0
.text 805336 805144 -192 -0.0
window-app BRD4161A (read/write) 1075508 1075300 -208 -0.0
.text 938940 938732 -208 -0.0
p6 all-clusters-app default (read/write) 2566392 2566232 -160 -0.0
.text 1524656 1524496 -160 -0.0
all-clusters-minimal-app default (read/write) 2511688 2511528 -160 -0.0
.text 1469952 1469792 -160 -0.0
light-app default (read/write) 2441616 2441472 -144 -0.0
.text 1399880 1399736 -144 -0.0
lock-app default (read/write) 2468768 2468608 -160 -0.0
.text 1427032 1426872 -160 -0.0
Full report (32 builds for bl602, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)
platform target config section 37bb2df 78c92ec change % change
bl602 lighting-app bl602 (read/write) 1397634 1397410 -224 -0.0
.bss 116978 116978 0 0.0
.data 4480 4480 0 0.0
.text 1058640 1058412 -228 -0.0
bl602+rpc (read/write) 1443066 1442834 -232 -0.0
.bss 124418 124418 0 0.0
.data 4600 4600 0 0.0
.text 1090324 1090096 -228 -0.0
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read only) 666275 666291 16 0.0
(read/write) 184948 184932 -16 -0.0
.bss 74116 74116 0 0.0
.data 3356 3356 0 0.0
.rodata 88139 88139 0 0.0
.text 577820 577836 16 0.0
all-clusters-minimal-app LP_CC2652R7 (read only) 632083 632099 16 0.0
(read/write) 157684 157684 0 0.0
.bss 73412 73412 0 0.0
.data 3356 3356 0 0.0
.rodata 77379 77379 0 0.0
.text 554380 554396 16 0.0
lock-ftd LP_CC2652R7 (read only) 669183 669199 16 0.0
(read/write) 172184 172168 -16 -0.0
.bss 71148 71148 0 0.0
.data 3280 3280 0 0.0
.rodata 76279 76279 0 0.0
.text 592424 592440 16 0.0
lock-mtd LP_CC2652R7 (read only) 618591 618607 16 0.0
(read/write) 144264 144264 0 0.0
.bss 66868 66868 0 0.0
.data 3280 3280 0 0.0
.rodata 76159 76159 0 0.0
.text 541944 541960 16 0.0
pump-app LP_CC2652R7 (read only) 678431 678431 0 0.0
(read/write) 163784 163784 0 0.0
.bss 71228 71228 0 0.0
.data 3280 3280 0 0.0
.rodata 88703 88703 0 0.0
.text 589244 589244 0 0.0
pump-controller-app LP_CC2652R7 (read only) 664255 664263 8 0.0
(read/write) 178080 178072 -8 -0.0
.bss 71348 71348 0 0.0
.data 3276 3276 0 0.0
.rodata 84567 84567 0 0.0
.text 579208 579216 8 0.0
shell LP_CC2652R7 (read only) 658774 658790 16 0.0
(read/write) 187952 187936 -16 -0.0
.bss 76420 76420 0 0.0
.data 3360 3360 0 0.0
.rodata 84918 84918 0 0.0
.text 573540 573556 16 0.0
cyw30739 light cyw930739m2evb_01 (read/write) 579614 579638 24 0.0
.app_xip_area 458368 458392 24 0.0
.bss 64184 64184 0 0.0
.data 716 716 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
lock cyw930739m2evb_01 (read/write) 585574 585590 16 0.0
.app_xip_area 459600 459616 16 0.0
.bss 68912 68912 0 0.0
.data 720 720 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
ota-requestor-no-progress-logging cyw930739m2evb_01 (read/write) 583018 583034 16 0.0
.app_xip_area 462620 462636 16 0.0
.bss 63392 63392 0 0.0
.data 660 660 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read/write) 1081700 1081492 -208 -0.0
.bss 132996 132996 0 0.0
.data 2048 2048 0 0.0
.text 946636 946428 -208 -0.0
BRD4161A+rpc (read/write) 1136012 1135820 -192 -0.0
.bss 149676 149676 0 0.0
.data 2260 2260 0 0.0
.text 984056 983864 -192 -0.0
BRD4161A+rs911x (read/write) 948172 947980 -192 -0.0
.bss 140768 140768 0 0.0
.data 2048 2048 0 0.0
.text 805336 805144 -192 -0.0
lock-app BRD4161A+wf200 (read/write) 1129096 1129112 16 0.0
.bss 144184 144184 0 0.0
.data 2060 2060 0 0.0
.text 982828 982844 16 0.0
window-app BRD4161A (read/write) 1075508 1075300 -208 -0.0
.bss 134468 134468 0 0.0
.data 2076 2076 0 0.0
.text 938940 938732 -208 -0.0
esp32 all-clusters-app c3devkit (read only) 1020154 1020178 24 0.0
(read/write) 1485842 1485842 0 0.0
.dram0.bss 70080 70080 0 0.0
.dram0.data 14600 14600 0 0.0
.flash.rodata 215728 215728 0 0.0
.flash.text 1020154 1020178 24 0.0
.iram0.text 62902 62902 0 0.0
m5stack (read only) 1074031 1074047 16 0.0
(read/write) 487904 487904 0 0.0
.dram0.bss 75600 75600 0 0.0
.dram0.data 34144 34144 0 0.0
.flash.rodata 246164 246164 0 0.0
.flash.text 1068647 1068663 16 0.0
.iram0.text 123267 123267 0 0.0
k32w light k32w061+release (read/write) 658832 658848 16 0.0
.bss 69516 69516 0 0.0
.data 1992 1992 0 0.0
.text 581524 581540 16 0.0
lock k32w061+release (read/write) 685684 685700 16 0.0
.bss 69980 69980 0 0.0
.data 2004 2004 0 0.0
.text 607900 607916 16 0.0
linux chip-tool-ipv6only arm64 (read only) 10068812 10068908 96 0.0
(read/write) 686913 686913 0 0.0
.bss 42961 42961 0 0.0
.data 3304 3304 0 0.0
.data.rel.ro 622984 622984 0 0.0
.dynamic 528 528 0 0.0
.got 13736 13736 0 0.0
.init 24 24 0 0.0
.init_array 192 192 0 0.0
.rodata 479628 479628 0 0.0
.text 8026596 8026692 96 0.0
thermostat-no-ble arm64 (read only) 2595668 2595780 112 0.0
(read/write) 158289 158289 0 0.0
.bss 65249 65249 0 0.0
.data 1704 1704 0 0.0
.data.rel.ro 83240 83240 0 0.0
.dynamic 528 528 0 0.0
.got 5072 5072 0 0.0
.init 24 24 0 0.0
.init_array 400 400 0 0.0
.rodata 165668 165668 0 0.0
.text 2190224 2190336 112 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2448312 2448376 64 0.0
.bss 213940 213940 0 0.0
.data 5872 5872 0 0.0
.text 1410956 1411020 64 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1175411 1175411 0 0.0
bss 142900 142900 0 0.0
rodata 142076 142076 0 0.0
text 811544 811552 8 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1155607 1155623 16 0.0
bss 142136 142136 0 0.0
rodata 133608 133608 0 0.0
text 800984 800996 12 0.0
p6 all-clusters-app default (read/write) 2566392 2566232 -160 -0.0
.bss 149120 149120 0 0.0
.data 2776 2776 0 0.0
.text 1524656 1524496 -160 -0.0
all-clusters-minimal-app default (read/write) 2511688 2511528 -160 -0.0
.bss 148400 148400 0 0.0
.data 2776 2776 0 0.0
.text 1469952 1469792 -160 -0.0
light-app default (read/write) 2441616 2441472 -144 -0.0
.bss 140456 140456 0 0.0
.data 2592 2592 0 0.0
.text 1399880 1399736 -144 -0.0
lock-app default (read/write) 2468768 2468608 -160 -0.0
.bss 140304 140304 0 0.0
.data 2600 2600 0 0.0
.text 1427032 1426872 -160 -0.0
telink light-switch-app tlsr9518adk80d (read/write) 797540 797564 24 0.0
bss 70576 70576 0 0.0
noinit 40416 40416 0 0.0
text 565734 565758 24 0.0
lighting-app tlsr9518adk80d (read/write) 817364 817388 24 0.0
bss 71420 71420 0 0.0
noinit 40416 40416 0 0.0
text 582056 582080 24 0.0

@github-actions
Copy link

PR #20715: Size comparison from 37bb2df to 3df266b

Increases (5 builds for esp32, mbed, nrfconnect)
platform target config section 37bb2df 3df266b change % change
esp32 all-clusters-app c3devkit (read only) 1020154 1020240 86 0.0
(read/write) 1485842 1485874 32 0.0
.flash.rodata 215728 215760 32 0.0
.flash.text 1020154 1020240 86 0.0
m5stack (read only) 1074031 1074131 100 0.0
(read/write) 487904 487928 24 0.0
.flash.rodata 246164 246188 24 0.0
.flash.text 1068647 1068747 100 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read/write) 2448312 2448376 64 0.0
.text 1410956 1411020 64 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1175411 1175427 16 0.0
text 811544 811564 20 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1155607 1155623 16 0.0
text 800984 801008 24 0.0
Decreases (1 build for nrfconnect)
platform target config section 37bb2df 3df266b change % change
nrfconnect all-clusters-app nrf52840dk_nrf52840 rodata 142076 142060 -16 -0.0
Full report (5 builds for esp32, mbed, nrfconnect)
platform target config section 37bb2df 3df266b change % change
esp32 all-clusters-app c3devkit (read only) 1020154 1020240 86 0.0
(read/write) 1485842 1485874 32 0.0
.dram0.bss 70080 70080 0 0.0
.dram0.data 14600 14600 0 0.0
.flash.rodata 215728 215760 32 0.0
.flash.text 1020154 1020240 86 0.0
.iram0.text 62902 62902 0 0.0
m5stack (read only) 1074031 1074131 100 0.0
(read/write) 487904 487928 24 0.0
.dram0.bss 75600 75600 0 0.0
.dram0.data 34144 34144 0 0.0
.flash.rodata 246164 246188 24 0.0
.flash.text 1068647 1068747 100 0.0
.iram0.text 123267 123267 0 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2448312 2448376 64 0.0
.bss 213940 213940 0 0.0
.data 5872 5872 0 0.0
.text 1410956 1411020 64 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1175411 1175427 16 0.0
bss 142900 142900 0 0.0
rodata 142076 142060 -16 -0.0
text 811544 811564 20 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1155607 1155623 16 0.0
bss 142136 142136 0 0.0
rodata 133608 133608 0 0.0
text 800984 801008 24 0.0

@bzbarsky-apple bzbarsky-apple merged commit 39e5ed2 into project-chip:master Jul 14, 2022
@bzbarsky-apple bzbarsky-apple deleted the commissioning-window-shutdown branch July 14, 2022 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants