Internal error ONCHAIN: HTLC in 2053 invalid update SENT_ADD_ACK_COMMIT->SENT_REMOVE_HTLC

[mercurytoxic]

Crashes on restart on multiple HTLCs with failonion in testnet. No idea how to proceed.

Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11300 already failed with failonion b8c93e538f29633ba9e01283b012777729b4760e0d654dea96cdc66a31bcc07acdd7dd7892a9229eabf42ae5aac66dafd8ab6ccedd832af32941a404ced01c6fb4c3e110f0c850dc3f02f479e00471cab249bb8b0ce1dc3f77fab30c442d892158e5cb17cd1fa94ca5d777f4b271494cd02402089b733b8b36d5886ecc789369a33de1a7d3065f989f895944e48d2dcb3d2a650c52aee0ab535e8630ff0cfd2d5ed4d7791e72f72c0e51247eace05db79f204718a44a34503963fee52a2efebc502bcd5237ce8fb1f80fabba892a140335d171cc9db76edd2ae1259dc89b96b85d59f7051af4421211bfad681c0876d0b083aa9b798796cccca30d1a59b6df3cb0263a447ab321903e476a9bc9d141689ca737e06ccff3af1907e785c27a0274a90fc863
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11293 already failed with failonion f64257c433c0d00b75be821040a4288916353bd82cdc6b7fa25e5acb4fb10af23933effc088f62f411f13de1b38725c2350824c4965d6ec4eec417008a93186e1923481451466db85b7be4326ff11cb5ec3c924326614ecc5d1673a1d794e408e356bc008d6e3a36507b40b9636becfc9d667fd8744a16ab0611198b129e86bbd21c9abec3049e1cb47bd2ac04fa2a9090f7e8183a3980c42b162ade5a3d12875dd28c24577a286869bbb5f5f380f14795d549f24439f9db1c8e830844b88144e8879153f7abbcbc4736d2d222ea436cc1bdd168731a32dfda264c53553943fd7b1c38968ec1c6503add4c1641030f3637fbf980ff3ccc323f671dbfaabab45168e8ca63d064791a115fbc005c7fd63ba0f33f8933807bec95a879d5b1a7fcfd608088c0
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11287 already failed with failonion 89ccb405a18ac78f9b6c695f1c6cd60f69497fb01317cbbf095c9b47c5138b97a3c86692c859328e83f7add74dab782bda986edd5a4094346441592017a9a7bec10779ae97ab61a9f90d7b3aa10745f91889b0a25b6ad7112af0bf78a39f75c8efffa7dba71eb6dafafab6a3978957b4c019008997d587b77d6f1059770229955d48a7a960bd5ed93b640b30ca4512bd348133e45bc324c631ff479e400087fed92e89c40fa917582256f59e90df9c2a38576656c2fd1139fc734e449b9279d42582521b4069b287031af48906c196cae9277e668ff6ad6e48e307319deebf885dc585dc830983ff23608a9990ed11127c40d08a007be6e00b14662d0e5ccf9b78dec53c36e8e5ffdff1378316368032a58a4ea127c8a8f77a12be13a716f43764bb438a
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 025a7e38c2834dd843591a4d23d5f09cdeb77ddca85f673c2d944a14220ff14cf7-chan#206: htlc 2813 has matching hash: failing
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z DEBUG   025a7e38c2834dd843591a4d23d5f09cdeb77ddca85f673c2d944a14220ff14cf7-chan#206: HTLC in 2813 RCVD_ADD_ACK_REVOCATION->SENT_REMOVE_HTLC
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 025a7e38c2834dd843591a4d23d5f09cdeb77ddca85f673c2d944a14220ff14cf7-chan#206: htlc 2814 has matching hash: failing
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z DEBUG   025a7e38c2834dd843591a4d23d5f09cdeb77ddca85f673c2d944a14220ff14cf7-chan#206: HTLC in 2814 RCVD_ADD_ACK_REVOCATION->SENT_REMOVE_HTLC
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11301 already failed with failonion e7a21a25674d2de25d479f00fd5a3eaec6a94263f67aa61fd201de4aee935291365e0b36b7e77bf145405e63c12a3863258ed93ae064cb65d99b1c5064ecf5eeca799c8e77314017df16a5679ab6e5fc163e5b402339cf648fea8fff84b5c5297667cdc544955e4f5a44026d87a230e913f2f0e914274bc7a9123ecac8f025d6c35b5f477b5331cba0df5df09c68b0fe3a0210ec83019cf9e5c72cf5402392cd8ae355be74ed494dc981c6a030e99e8b79ae93a0b06f9779229b591802a8ac95171ef4ad839827e29ebdf1173955d246855108de93dde1d17d4b13d4a5d3ad794ae6579a464acf94d759b98ecabe8f08d6f2da3a920c09ad485ab33c2ed8388c8ed77c3165f3add0379d3ca473b17883c26054fad1dcf4afe8370bd28f21e31c3c9a62be
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11285 already failed with failonion 345cdedcb767c080b7ae6b15838b4b4abc23b949ca1a47a21709d34675bf80d628f9c95ddfe77bc90f70f1d9de6dbd86f91cae6d80eafb77f97bb41568e3a2b3a695dfbe1703d3cba6312328498395c2e918320f0a9d909a42e6eac06ff7a1a37bcd9a899f94d5268effe3a5195a237113adc2c02375b98f2a9ff3ab5d07b47fb36aa6bc765dfd386d7246da91a323cee48b4925459ccc37cc8e00dc1a703aa275c4d54d9d07aa817c799d6469bc0dfd17a619d1e8d247e508479607de872ecca7d322d18474afa545f49a77e89b1e44e9d82d8199c246c391cb1bbbf7518713834f71503d75686618a253340d9ec0e86fb51e68747ff554d5a0b42fcdf5d63e78367dc6e05681cc48d488fbb7ddcc15d8c5ed7e9d1e5600aa4fd151e8bd80e89deeff48
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11286 already failed with failonion d782385093acb5e8ca1cc3a2b849ce8cadff770251d73fecb6e145ba3a22f4f89b40c65b56f1fda6fe7745fd1f8581aad6ea9b402542a1e95806bc29c746ccad807821eb495cf5db6bf4954a98098dd3aab4bbe0d593b1d556cae50f8836e86254a40670044e88f7a3ab3e5117a24510649fb02ac04ee3b643ae53457842699d5e1d0f62951e9d077a89da81312053b567b1b917fa8b29d0058129446cf02d38fc8c1d7c69810b5d9abee057dc7d764c0ca01433b1e676c7c87bccddf0dac8bc82fffeb2b6ab12b6de6d413d7b0550322bcc19e4b478b4e1994bad11b00d4dea7d06639c49b2a64591b8cd78d4965823b5b6d6fb63dffc323db7a6e55fdb10a24194d6bdeb0632e42098a1a1d04d48ae540bc6717ec573f079338b6333f650dc39f2e050
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11295 already failed with failonion e103aa00b39645f362575c4cecc8c9d635a4458d95746b990834c67bb93be7c8d9816b700f0025bf9be8f09dab614f28520e2402b4649e916e13995d3304bf86c72f2063c3d008193c6fd742763b169ec0769c5d34242e20883ef9c4cd0c90e917cc7c744d145b63f1120493227c94f1ca73dae2c993acb1b5252c19ebf6c27abf47da220e547d4bcd8f4283c342e239d80ec6816a999395061822e036f9d969248b7c50986876083c8b57a343d3dd38cc883110fa60b41ee076eb917cf5b9ab9fd1ce466006f11880b17862f3caaf8251455840e31ade730e6515f6798ebcbe7e6034d1539cb5dd8656f8db290c1608ab338323571920aadb81d9fc59641a52d78119269b1a295b617d92a00cb326312fdb027d8be98b602612ad21670dda92b6cf8acf
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11291 already failed with failonion 77a2aded49c2ffb398ce0d608ac4c4352e2bf55b60e4c9f15eb65098592218f5788665d49eedb4222f63673f29eaad99b4089b0ed4c9aca9e040871c101be1263dccaaf2ccc2e9b2eda5c863e08dc3491a84e699c525ef610d3736dd821aa987fb0bef5dc67e2b2b06d1aae87ac9d8a98fc327e83d52e5326da1d404c322032e7467d811f63ad4a2aac864d813c9555779f709d6de52f9764e9f9ad4324f4ea8c33d10d6e06e063855842f95c7307d426effd18d018333349642bc2fe1e9e640508e7995d15dffa430663ddc5f8deb9096186c07c79cbdf024c9ae859be283b113e9e151934ed0ca29ee10e7ecf17416c7eb50e541ff4e054715555cf78553d3c1758e9dd71ea5f8afca35262d60cd9e62f664843c14724ef332299a236ba2ab62b49b97
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11302 already failed with failonion 2684128cc377420e0e870fd8d6b20e35608f4ba60c0466e699010653706db0fedc75da2bf31993ab25268a5ff6efb78920ee6e9e65154f64b4e87e5feed4fe09a90411b9d12a87d3f4b71b1966f499d84bcaaf5e0cfe16563955c06f9aff3a511f8e1c8fb8a4da15339d98672ede83ad2a3664d58d8f0f450e2ce52d23aa694f573d13a8d85a14c9d11b85bb09cf54e1323fe9a393922526c694d4dd53836b5b10bd2146d4d06f9f941a9bd89d1a6af1bff97613fae3973094f0031ab31799d60d8a33de70f7665eb9f000440898dc7b2f8d6babbd8b91559c61169df064e08d0257ede92a3e14ce5d4401aafdb868bc980aeb06c87a396ca846cafe1162f5ad2a234876e08f4ff5fbbd80559877e77e4bf2935794a385f617e62b2aa5fb4408c07d37dc
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11290 already failed with failonion 4bd643611b1621f71fb9fbcdee4f4cb36d6ab342909d3f9cfa8ec10cf66bbf8ab5c403737fa4358484d3da856bd9ad5f49949779390c975aeecd3fe1998b65ec0014496ae9db91edef49cfb6499b5396b4dcd89d9c28634f0047eb9f340a937aedd2d12f76febe3617b5c5f8cff634e26a9a63459812dd8c6001b717f079413a134674fe43e73a43ba7c2655a613074a5e3e9fa78f8e4d1209f95fba7db4df947f8c93b8adac150eedc7214cb5dd53e6a1f35ee39cbbb2cf327e96e6d22fc99bd6edb701f44df80be0aa86da9c9b3c63b741ff63cb5fb8239c99797b65511d3be3a06d00567d0a8db30c3be075a38ad5a55bc9fef9b664ec1e07138d863375e71aae182417a5630ca124cf862ace660cb650d4e843c8e2246ee52c0a44f7f7f65901bcc2
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11292 already failed with failonion 6c512cb43968b45683fa79b79541103a00787d74d73178ad3f0cbfad9bc51420e4dffcd6f00b3c7beb53a179d54303fa3110120f9cad7830433ed71f3d4b41b14f3c24a7ddb064e8fd8be2e0837e59d588eebc79ad17d56022756f5a92fab1421ceee3c1b95b8b4ae505751ee49207c5977c027627d330fb7ebc742aa97dfab1875af8ac7410fdfabba051ea06c8e538a1919c08b1ad1470d7458a742786611935713aa86287748afaad61698f12f677802b35ddf17e48effe4518b7d0f54b3d0774f669b40e11165a952d53875e859c547528f35558e1028fe8f210e1ca445d57b2a67bd879070c5d0c18e5b6184d71235570675aaefbee53497538ba96877ccbdba6dcff44e192bea6eeb78a6599f97adb18f1382858139cbb7b5c28a9df717e94ecb2
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11297 already failed with failonion b33e46b9edb1080234ae7e8ab9f405afcb311e54a5cbf05a36fbeccd6e4596bd052bcc6b3ae69a0051ca4e87a1a0dc6b925096c5cc949baaf9ff2dc8ab4b1e5d0f685211234f6d404819dc905dcb96a94e45dfd7fa471d5368cfd793e7a1d9b517c2d9ca050a3aadd3601f80362a6f768a3a38f744ae12d9f4549afa1025d4b8c588b3755ad0f76c022db5d6ca1e6ce60cf0adef5d57da74e14a498b5f95d9c2853d229679b2c08a464a8707c0cdb874999f99b15331e09be681b107f5b0afe14a2e78a3c4dab3098de92402c363f5d63fa61ed85bed9764b9328faf96ce8ba4cb2754dbfb4e590643660aac2c4fe35f3d9f76f42a2dc3011911326766d5d1f45eb045c801e5ada13edaf4351b4cada9e168c6902ac2dc3b8ff748a385f014ca0020f4de
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11298 already failed with failonion 8e19fbca0a0a7ec124b09fec764448003adbddee21c6f03c16bd5248d8fe3b3ea00174facebe6691f76f24c3d654998ed3ef84b7e64df55c6fa80020f6f47c2fae32db9964323c693da8b36fdbaa61daf5b609e5e38be7b4222d31c52d84267ca7375d73d327a2b9d8af83410e34e374ed7956e71cddcfa583ae248ea1766feeed6da15eb38c5e7aacff1af1da820625eed71990dc0e7a8149586d3506eaa99a2dac66130b92a48a5cb223a2eaecba281a8a9c3231f2259ef25cb358cb796d9d3bbd62afb76cdccd005c9db3bb18b5e535b786d19a4a4bf75e0427f97cf5d273d87d8a92ce75532fc53ff7e774a045c7bb92e5b7e384f49098cf2e7d89417d0b364bd281e513cff9057809bd8979201ba5851deec22ba30f1777b61160865d3ca60c1d1c
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 02346860ea1c432c4d449f8a1bc8ca7ebd0b38165ee99b43d2a820189171a4a922-chan#114: htlc 11299 already failed with failonion de18c4e3554ffbec92d47ea44b51aeffe97604aab7a8a99a46b2ae317193c822971ad440b7ca09e2641876dc92ee9f03e94aa68a5981302470bc3068da29c1222b30d11b9c18c1622ed531d92aa4743c5087d29459c0fce481e079ca3cac4e15a2d0986019024b9a037600f6ed684ceade632cdf9c5cdd5ed011ed58a24696bbf688e5801bc1f10107c77a6a29f889df1c215f03aedaeb6cf92f1edbb699af12a1f776ddd40c84fa4dd6b89ac2041b21407a12c7b36fad0e2b8fd20d2da3467a167f1907c7dd04918b2ba27f62485306bc84c0e6e387ce8b1e334e7623ba20fe848e803cb43c2cfbf16cf8b39feb00f0022b2583f3661438720a0f62199cdb60b18ac5f46c73db37139d5d5fa6aade7256191a9411212aa1bc6043344165bca5125a8ee5
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 032202ec98d976b0e928bd1d91924e8bd3eab07231fc39feb3737b010071073df8-chan#212: htlc 2053 has matching hash: failing
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z **BROKEN** 032202ec98d976b0e928bd1d91924e8bd3eab07231fc39feb3737b010071073df8-chan#212: Internal error ONCHAIN: HTLC in 2053 invalid update SENT_ADD_ACK_COMMIT->SENT_REMOVE_HTLC
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z UNUSUAL 032202ec98d976b0e928bd1d91924e8bd3eab07231fc39feb3737b010071073df8-chan#212: Peer permanent failure in ONCHAIN: Internal error (reason=local)
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.763Z DEBUG   032202ec98d976b0e928bd1d91924e8bd3eab07231fc39feb3737b010071073df8-onchaind-chan#212: Status closed, but not exited. Killing
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.820Z DEBUG   lightningd: Broadcasting txid 248b646df597f07fdf62ad891df90e67e84c60c090f5af70e307368127297445
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.820Z DEBUG   lightningd: sendrawtransaction: 0200000000010148e6d9bf79da9effd53367fb0c8cb1eeafd626639e038bb6467666348c8861500a000000008fe1458002d30f000000000000220020e4bb931bf1333dc7640beef57505c7a59fe855e748cb9621ad1641d8e831f8d47a51010000000000160014e273d8877b8313abcbbbc0a795732dbc2d25c606040047304402202b73381792255c7fc64db2a56cc602869dd41f9c351e1e3c4272db9cb9da83bb0220031716fe65d0f2d54df01eca734d046f10c5de0d06e576af1f3a2d0302b5202d01473044022011bf324fd9f9878c38c9b61e754f655a9521d6949dbdb352780b5bf0e19f8cfd02203e0cf96c1cdf0be2447c69c1a834350cae806356b0abd9dfe9ecdfa4bc45af560147522102ce7dc643a4214179e7a5fe500d6a0531fcfdd15c141db249db39bd967121c2552103833d2c0186a0af22dfe52e61003d54ef10be889b204720e6d4c2693afa955a8052ae2b648220
Mar 24 16:30:48 testnet lightningd[2000675]: fail_in_htlc:Still adding, has failmsg
Mar 24 16:30:48 testnet lightningd[2000675]: 2025-03-24T05:30:48.820Z **BROKEN** lightningd: fail_in_htlc:Still adding, has failmsg?
Mar 24 16:30:48 testnet lightningd[2000675]: lightningd: FATAL SIGNAL 6 (version v25.02-31-g08210da-modded)

Head

Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.356Z INFO    lightningd: v25.02-31-g08210da-modded
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.487Z DEBUG   plugin-manager: started(2000693) /usr/lib/c-lightning/plugins/autoclean
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.490Z DEBUG   plugin-manager: started(2000694) /usr/lib/c-lightning/plugins/chanbackup
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.492Z DEBUG   plugin-manager: started(2000695) /usr/lib/c-lightning/plugins/bcli
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.495Z DEBUG   plugin-manager: started(2000696) /usr/lib/c-lightning/plugins/commando
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.497Z DEBUG   plugin-manager: started(2000697) /usr/lib/c-lightning/plugins/funder
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.500Z DEBUG   plugin-manager: started(2000698) /usr/lib/c-lightning/plugins/topology
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.502Z DEBUG   plugin-manager: started(2000699) /usr/lib/c-lightning/plugins/exposesecret
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.505Z DEBUG   plugin-manager: started(2000700) /usr/lib/c-lightning/plugins/keysend
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.507Z DEBUG   plugin-manager: started(2000701) /usr/lib/c-lightning/plugins/offers
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.510Z DEBUG   plugin-manager: started(2000702) /usr/lib/c-lightning/plugins/pay
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.512Z DEBUG   plugin-manager: started(2000703) /usr/lib/c-lightning/plugins/recklessrpc
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.515Z DEBUG   plugin-manager: started(2000704) /usr/lib/c-lightning/plugins/recover
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.518Z DEBUG   plugin-manager: started(2000705) /usr/lib/c-lightning/plugins/txprepare
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.521Z DEBUG   plugin-manager: started(2000706) /usr/lib/c-lightning/plugins/cln-renepay
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.524Z DEBUG   plugin-manager: started(2000707) /usr/lib/c-lightning/plugins/cln-xpay
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.527Z DEBUG   plugin-manager: started(2000708) /usr/lib/c-lightning/plugins/spenderp
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.530Z DEBUG   plugin-manager: started(2000709) /usr/lib/c-lightning/plugins/cln-askrene
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.534Z DEBUG   plugin-manager: started(2000710) /usr/lib/c-lightning/plugins/sql
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.537Z DEBUG   plugin-manager: started(2000711) /usr/lib/c-lightning/plugins/cln-grpc
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.540Z DEBUG   plugin-manager: started(2000712) /usr/lib/c-lightning/plugins/clnrest
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.543Z DEBUG   plugin-manager: started(2000713) /usr/lib/c-lightning/plugins/bookkeeper
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.546Z DEBUG   plugin-manager: started(2000719) /usr/lib/c-lightning/plugins/wss-proxy/wss-proxy
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.703Z INFO    plugin-wss-proxy: Killing plugin: disabled itself: No module named 'pyln.spec'
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.703Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.703Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.703Z DEBUG   lightningd: io_break: check_plugins_manifests
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.703Z DEBUG   lightningd: io_loop_with_timers: plugins_init
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.707Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_channeld
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.711Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_closingd
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.714Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_connectd
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.717Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_gossipd
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.720Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_hsmd
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.723Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_onchaind
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.726Z DEBUG   lightningd: testing /usr/bin/../lib/c-lightning/lightning_openingd
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.728Z DEBUG   hsmd: pid 2000733, msgfd 57
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_CHECK_PUBKEY
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_ANY_DELAYED_PAYMENT_TO_US
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_ANCHORSPEND
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_HTLC_TX_MINGLE
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_SPLICE_TX
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_CHECK_OUTPOINT
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_FORGET_CHANNEL
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_REVOKE_COMMITMENT_TX
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_BOLT12_2
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_PREAPPROVE_INVOICE_CHECK
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.729Z DEBUG   hsmd: capability +WIRE_HSMD_PREAPPROVE_KEYSEND_CHECK
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.847Z DEBUG   connectd: pid 2000734, msgfd 61
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.847Z DEBUG   hsmd: Client: Received message 27 from client
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.847Z DEBUG   hsmd: Client: Received message 27 from client
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.847Z DEBUG   hsmd: Client: Received message 27 from client
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.847Z DEBUG   hsmd: new_client: 0
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.850Z DEBUG   0225ff2ae6a3d9722b625072503c2f64f6eddb78d739379d2ee55a16b3b0ed0a17-hsmd: Got WIRE_HSMD_ECDH_REQ
Mar 24 16:30:38 testnet lightningd[2000675]: 2025-03-24T05:30:38.850Z DEBUG   hsmd: Client: Received message 1 from client
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z DEBUG   connectd: Proxy address: 10.10.10.6:9050
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z DEBUG   connectd: Created listener on 0.0.0.0:19735
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z INFO    connectd: Static Tor service onion address: "aqihq326i6jlwzctgx46dmvu5xq4rpesyad2q7redtmivrjdcfxbyuad.onion:19735,0.0.0.0:19735" bound from extern port 19735
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z DEBUG   connectd: REPLY WIRE_CONNECTD_INIT_REPLY with 0 fds
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z DEBUG   lightningd: io_break: connect_init_done
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.016Z DEBUG   lightningd: io_loop: connectd_init
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.070Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.070Z DEBUG   lightningd: io_break: plugin_config_cb
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.070Z DEBUG   lightningd: io_loop_with_timers: config_plugin
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.070Z DEBUG   lightningd: All Bitcoin plugin commands registered
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.076Z DEBUG   lightningd: Unable to estimate any fees
Mar 24 16:30:46 testnet lightningd[2000675]: 2025-03-24T05:30:46.087Z DEBUG   lightningd: Adding block 3052235: 00000000000015b352240e9cb5094800dd3178b0bf994ee4c921cf6f1e1aa392