Spurious Heal overflows with Python 3.11

[chander]

With Python 3.11 we are getting some heap overflows/exceptions when working with python-xmlsec - it's possible that this is related to an underlying library, but seems perhaps related to python-xmlsec.

I'm not sure whether or not this might also constitute a medium or high CVE, since this does cause the process to crash. I don't have a specific way to reproduce this yet (like a block of code) - this is happening when processing a response from a sAML provider. This did/does not happen with Python 3.9

==1== Thread 9:
==1== Invalid read of size 4
==1==    at 0xBAA4797: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==  Address 0x2a3f9fb8 is 8 bytes before a block of size 1,048 alloc'd
==1==    at 0x48407B4: malloc (vg_replace_malloc.c:381)
==1==    by 0x2EFA2803: UnknownInlinedFun (dict.c:172)
==1==    by 0x2EFA2803: xmlDictLookupInternal.lto_priv.0 (dict.c:768)
==1==    by 0x2EFA2885: xmlDictLookup (dict.c:827)
==1==    by 0x2EFA2FD0: UnknownInlinedFun (hash.c:504)
==1==    by 0x2EFA2FD0: xmlHashAddEntry (hash.c:622)
==1==    by 0x2EFDF8B9: UnknownInlinedFun (valid.c:2571)
==1==    by 0x2EFDF8B9: xmlAddID (valid.c:2517)
==1==    by 0x2F0D4875: xmlSchemaValAtomicType.constprop.0 (xmlschemastypes.c:2981)
==1==    by 0x2F0458D3: xmlSchemaValPredefTypeNodeNoNorm (xmlschemastypes.c:3605)
==1==    by 0x2F03C914: xmlSchemaVCheckCVCSimpleType.lto_priv.0 (xmlschemas.c:24560)
==1==    by 0x2F0403AE: UnknownInlinedFun (xmlschemas.c:25614)
==1==    by 0x2F0403AE: xmlSchemaValidateElem (xmlschemas.c:26911)
==1==    by 0x2F04318F: UnknownInlinedFun (xmlschemas.c:27892)
==1==    by 0x2F04318F: xmlSchemaVStart (xmlschemas.c:28070)
==1==    by 0x2EF63D48: UnknownInlinedFun (etree.c:244248)
==1==    by 0x2EF63D48: __pyx_pw_4lxml_5etree_9XMLSchema_7__call__.lto_priv.0 (etree.c:244040)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==
==1== Invalid read of size 8
==1==    at 0xBAA47A8: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==  Address 0x2a3f9f98 is 24 bytes after a block of size 320 in arena "client"
==1==
==1== Invalid read of size 8
==1==    at 0xBAA47B0: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==  Address 0x2a3f9f90 is 16 bytes after a block of size 320 alloc'd
==1==    at 0x48407B4: malloc (vg_replace_malloc.c:381)
==1==    by 0x2EF9E799: xmlHashGrow (hash.c:362)
==1==    by 0x2EFA2E5C: UnknownInlinedFun (hash.c:474)
==1==    by 0x2EFA2E5C: xmlHashAddEntry (hash.c:622)
==1==    by 0x2EFDF8B9: UnknownInlinedFun (valid.c:2571)
==1==    by 0x2EFDF8B9: xmlAddID (valid.c:2517)
==1==    by 0x2F0D4875: xmlSchemaValAtomicType.constprop.0 (xmlschemastypes.c:2981)
==1==    by 0x2F0458D3: xmlSchemaValPredefTypeNodeNoNorm (xmlschemastypes.c:3605)
==1==    by 0x2F03C914: xmlSchemaVCheckCVCSimpleType.lto_priv.0 (xmlschemas.c:24560)
==1==    by 0x2F0403AE: UnknownInlinedFun (xmlschemas.c:25614)
==1==    by 0x2F0403AE: xmlSchemaValidateElem (xmlschemas.c:26911)
==1==    by 0x2F04318F: UnknownInlinedFun (xmlschemas.c:27892)
==1==    by 0x2F04318F: xmlSchemaVStart (xmlschemas.c:28070)
==1==    by 0x2EF63D48: UnknownInlinedFun (etree.c:244248)
==1==    by 0x2EF63D48: __pyx_pw_4lxml_5etree_9XMLSchema_7__call__.lto_priv.0 (etree.c:244040)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x2EF4E4FF: UnknownInlinedFun (etree.c:295173)
==1==    by 0x2EF4E4FF: UnknownInlinedFun (etree.c:270406)
==1==    by 0x2EF4E4FF: __pyx_pw_4lxml_5etree_10_Validator_3validate.lto_priv.0 (etree.c:207774)
==1==
==1== Invalid read of size 8
==1==    at 0xBAA47E8: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==  Address 0x2a3f9f98 is 24 bytes after a block of size 320 in arena "client"
==1==
==1== Invalid read of size 1
==1==    at 0xBB04D64: xmlStrEqual (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAA47F3: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==  Address 0x180 is not stack'd, malloc'd or (recently) free'd
==1==
==1==
==1== Process terminating with default action of signal 11 (SIGSEGV)
==1==  Access not within mapped region at address 0x180
==1==    at 0xBB04D64: xmlStrEqual (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAA47F3: xmlHashLookup3 (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0xBAB6DBB: xmlGetID (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14)
==1==    by 0x36426609: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36426585: xmlSecAddIDs (in /usr/lib/x86_64-linux-gnu/libxmlsec1.so.1.2.37)
==1==    by 0x36382D1C: PyXmlSec_TreeAddIds (tree.c:192)
==1==    by 0x4A613CB: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4759E: _PyObject_MakeTpCall (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4E8EA: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A4D479: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A6AD75: ??? (in /usr/local/lib/libpython3.11.so.1.0)
==1==    by 0x4A50A70: _PyEval_EvalFrameDefault (in /usr/local/lib/libpython3.11.so.1.0)
==1==  If you believe this happened as a result of a stack
==1==  overflow in your program's main thread (unlikely but
==1==  possible), you can try to increase the size of the
==1==  main thread stack using the --main-stacksize= flag.
==1==  The main thread stack size used in this run was 8388608.
==1==
==1== HEAP SUMMARY:
==1==     in use at exit: 54,206,283 bytes in 68,135 blocks
==1==   total heap usage: 965,787 allocs, 897,652 frees, 2,516,301,590 bytes allocated
==1==
==1== LEAK SUMMARY:
==1==    definitely lost: 1,626 bytes in 29 blocks
==1==    indirectly lost: 102 bytes in 2 blocks
==1==      possibly lost: 251,037 bytes in 170 blocks
==1==    still reachable: 53,953,518 bytes in 67,934 blocks
==1==                       of which reachable via heuristic:
==1==                         stdstring          : 23,733 bytes in 648 blocks
==1==                         newarray           : 16 bytes in 1 blocks
==1==         suppressed: 0 bytes in 0 blocks
==1== Rerun with --leak-check=full to see details of leaked memory
==1==
==1== For lists of detected and suppressed errors, rerun with: -s
==1== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)

valgrind: the 'impossible' happened:
   main(): signal was supposed to be fatal

host stacktrace:
==1==    at 0x580429AA: show_sched_status_wrk (m_libcassert.c:406)
==1==    by 0x58042AC7: report_and_quit (m_libcassert.c:477)
==1==    by 0x58042D1B: panic (m_libcassert.c:553)
==1==    by 0x58042D1B: vgPlain_core_panic_at (m_libcassert.c:558)
==1==    by 0x58042D3A: vgPlain_core_panic (m_libcassert.c:563)
==1==    by 0x580AFC30: shutdown_actions_NORETURN (m_main.c:2383)
==1==    by 0x580E1C99: run_a_thread_NORETURN (syswrap-linux.c:201)
==1==    by 0x580E1E4A: vgModuleLocal_start_thread_NORETURN (syswrap-linux.c:328)
==1==    by 0x580AA17D: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???

[chander]

Closing because I think this is the same issue as SAML-Toolkits/python3-saml#389