Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade OpenSAML version from 4.3.2 to 5.1.3 #90

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

danielcompton
Copy link

https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview

This fixes one deprecated method removal, but there are a few other test failures I'm not so sure about.

@camsaul camsaul requested a review from a team February 10, 2025 19:32
@camsaul
Copy link
Member

camsaul commented Feb 10, 2025

Seems like 5.x drops support for Java 11 which is why that job fails. Some legitimate failures in the Java 17 job tho

@danielcompton
Copy link
Author

danielcompton commented Feb 12, 2025

@camsaul how do you feel about dropping support for Java 11? https://www.metabase.com/docs/latest/installation-and-operation/running-the-metabase-jar-file#1-install-java-jre seems to suggest Java 21 is your minimum supported version of Metabase.

@danielcompton
Copy link
Author

danielcompton commented Feb 13, 2025

The other test failure appears to be coming from

<saml:EncryptedAssertion ID="pfx210ac9a7-b8ad-ebc7-1f24-5e1a13a50f6d">

the ID attribute seems to not be valid there? IntelliJ also complains about it being there, which suggests that maybe OpenSAML got stricter in validating the XML. I tried removing the ID attribute, but a new test fails now because the message signature was no longer valid.

  actual: org.opensaml.core.xml.io.UnmarshallingException: Saw invalid attribute 'ID' on element {urn:oasis:names:tc:SAML:2.0:assertion}EncryptedAssertion

I'm not quite sure how to fix this, I assume we might need a new test file here? It seems more like a validation issue, than a bug in saml20-clj.

Update: this looks like a similar issue, where OpenSAML 5 is stricter in its validation: https://shibboleth.atlassian.net/browse/OSJ-392.

@danielcompton
Copy link
Author

danielcompton commented Feb 13, 2025

Ah, yep, here's where the issue was introduced: https://shibboleth.atlassian.net/browse/OSJ-293. OpenSAML introduced a strict mode to reject unexpected content.

The tests all pass with:

clojure -J-Dopensaml.config.xml.unmarshall.strictMode=false -X:dev:test

I'm not too sure what the right thing to do is here. Probably to document the change and how to opt out of it? And then fix the test file so it is valid and signed.

The Shibboleth release notes say:

The XML processing code in OpenSAML has been enhanced to support a more strict form of processing that rejects unexpected/unknown XML Attributes, Elements, and even stray characters inside elements. Older versions tended to ignore them. In the vast majority of cases, rejecting such content is desirable but it is possible to turn off this processing mode by setting the property opensaml.config.xml.unmarshall.strictMode to “false”.

I reviewed https://shibboleth.atlassian.net/browse/OSJ-392 and am not 100% sure what the actual outcome of the ticket was. We need to test this, but I have to assume that ADFS works by default without disabling strict mode.

@camsaul
Copy link
Member

camsaul commented Feb 19, 2025

@metabase/core-backend-admin-webapp can you help shepherd this PR?

@edpaget
Copy link

edpaget commented Feb 19, 2025

@danielcompton I'm working on getting a new valid test file which I'll post here, so you can incorporate it.

Documenting the new strictness and how to opt out is a fine path forward. We should also accompany that by bumping library version, which you can set by updating the VERSION.txt file. A major bump seems appropriate here but not exactly sure the Metabase philosophy on lib version is since I'm relatively new here.

@edpaget
Copy link

edpaget commented Feb 19, 2025

@danielcompton Here's a clean test file

test/saml20_clj/test/response-with-signed-message-and-signed-and-encrypted-assertion.xml

<?xml version="1.0"?>
<samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" ID="pfx4adc6cb4-d390-77ee-6ae5-d5f798badfca" Version="2.0" IssueInstant="2014-07-17T01:01:48Z" Destination="http://sp.example.com/demo1/index.php?acs" InResponseTo="ONELOGIN_4fee3b046395c4e751011e97f8900b5273d56685">
  <saml:Issuer>idp.example.com</saml:Issuer><ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
  <ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
    <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
  <ds:Reference URI="#pfx4adc6cb4-d390-77ee-6ae5-d5f798badfca"><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>o+H48AO2W6gUqzeKMbkzGFEeiGw=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>XusPe0BSQiUsORxARxPzOP0u44i6eZgBAKdQfBz+G6Xe0sQ32FOy4R4bRIqei1lCqG7TlJ9S10H1zn5ZNKFLzAkbvMZV0eKutdoyTO7YnCMUwkmS1NFNU0gpu8H2bCpNOxsUmyKcsnhvN6XEoApJZb+E5deXi2fPqBoX2sc9K4U=</ds:SignatureValue>
<ds:KeyInfo><ds:X509Data><ds:X509Certificate>MIICajCCAdOgAwIBAgIBADANBgkqhkiG9w0BAQ0FADBSMQswCQYDVQQGEwJ1czETMBEGA1UECAwKQ2FsaWZvcm5pYTEUMBIGA1UECgwLRXhhbXBsZSBJZHAxGDAWBgNVBAMMD2lkcC5leGFtcGxlLmNvbTAeFw0yMDA5MjMxNzQyMjRaFw0zMDA5MjExNzQyMjRaMFIxCzAJBgNVBAYTAnVzMRMwEQYDVQQIDApDYWxpZm9ybmlhMRQwEgYDVQQKDAtFeGFtcGxlIElkcDEYMBYGA1UEAwwPaWRwLmV4YW1wbGUuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDOqpTbp9VJq0aXQbV6duEDGm8Def5F6LMtSgOkNb3GVw5nBrsQtxI2R6aBwgVpgNbkbG6WLQxRWEpEoSbKM0kUle3YN04+k/e7+LkWrzBx3dykdhEqF+gQyK7fWjfj35UJqReM8cSzxUHQKHBcL+D59VAyJC3sA1mvuKsS/7RIMwIDAQABo1AwTjAdBgNVHQ4EFgQUviPb7EZV3vBqOlKlyxI/Ps2Z/9AwHwYDVR0jBBgwFoAUviPb7EZV3vBqOlKlyxI/Ps2Z/9AwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQ0FAAOBgQBus/4hCWuEVWX882TaUXmIr3yMuICxm+5VEt4dmiQrj6miJoV6kNTPuiyskPe5SK/5VBJMSSm1eqbW7nTaUYrnwUaYT8UBAfkBgAEM/PhuI8cBSC3YoVPZwQbmywKMsprSMfE4K9eOyay9796lddrdMVkD8MgD8Z1i5+vvw7kpcg==</ds:X509Certificate></ds:X509Data></ds:KeyInfo></ds:Signature>
  <samlp:Status>
    <samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success"/>
  </samlp:Status>

<saml:EncryptedAssertion><xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Type="http://www.w3.org/2001/04/xmlenc#Element"><xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc"/><dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"><xenc:EncryptedKey><xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/><xenc:CipherData><xenc:CipherValue>jIbl+4Lppnx8ExjYQANOM3yY00xsmU9/9FOWpOB8mT0biAo+I6D33j3EVqFvfZqbfD+Eisk1msoTUaGQ9AR0k4QCgwoIwskafvh9mbLCsD3siMhnbdMiM3CPK5IAvGwUyc3yO97FFaSMV0JuQQ3cvNMo9OIPpdZu5VIvRrNsmfQ=</xenc:CipherValue></xenc:CipherData></xenc:EncryptedKey></dsig:KeyInfo>
   <xenc:CipherData>
      <xenc:CipherValue>fSopuN4yWlTD2KBceF6cUA8qHR4XV6Ywvw2Dq0DhAEvoKuVkarOK2rpMsGi3cdum1jFJimwd73igrW/EX7U3KM+Kg8VWOEDPWUhphin6I4TYhoq21IWIkwPJ7mxdvJDjImKKN9zGcjCxmBSLkNbwXcCcUwQRlMXg3tHSQ4gKXNpQSEw84sS64ojpgSevyZVxJeL4tuJtwibzAcg285W16ycva1AmEVqjxYSAVIEkbLVeoNzabFEsZ0siz33+qCJw9A+Rrcpoe/Jsihp0h0QiBWvyntvFxKL+ebEEV0pmocg4h/mLthNdD9uF9jituHXhuQQY/zUnDK7ycSXE9BhsNm7Aze2AI0DjQ5dFPYayuXoeqjRAOVVAJe1izoVGlDUKh2LRqjaHxyCS2rTrAN7en1mEPje9undU/UZ+alueydqCd6kF7Wxx3g1iHG8hYJaD8w8nnnBLH7mwEec1qjreuc1DUVbir9a/9oIjqlTj9PRPHOT7tpYsINj7Il9Y2TL9azz1X7y9Jwj4OZkoTder/nIV4R0V1PDXn101rBB519bz+5iKcu9LpOxnQIgL/jRWs628JZtBmh4SqA6PSgwKqZXRNoHSIWp6tCbpCrrPdOhdN8NUlQgc59ys4Rw+6bxlZ1b0LIwnsALS1biPWChDxdQQboSnftLKhGBEyH8RwkBP1QMTKTB9fy8GFSSpISqOTu2AtCq6DSDs+ZMjEeb9Uq0zkvRJ0Au4n7avcT6Asv6FJUEI0EExqkAyP/jcdOnIthPWEWHRDl9n+SqN9qilUeK1L3wqX/vb6itaxyGAX/ZhMuA2NnN2GLtcep3jmu6ZSB1lEGromGAkavoA1jc0V71Qg+1Zc355vgUdWCBtveub1QWLob6CwkTUW0o/XBXC4D8llnO1fjimElax2nINqzr7ED30ZBOWX2bZDHhwXGgjAOWAAetF2NryRb9IWPPntr+q8E93HBoiedNGlM/fXCmmGeZHUV5tq4Zv26L5YODSVPQnXzV6Ta66yMShaD3/i+v+0vr/+bq/C5ZZdUOXAb/sP6SiPolrn9XE+n1W6xXyEtVwJ/+eSEk4wPfxwDTFWA7Q4mV58ewwkCBoydsmvKEgcWqmAro81EYre5/cqSHfLc/5aBP7J8SOjGCSBoXMpwzuZoJwFRo6kjjf0nrM5rlG5PDOHw3rwOStJKq6e7th2tH6er+P1IKJya+NqUFDbnmNKFmFewHGg9M5fQ0VLLxAXAKjjpde/vp6r2Etl60mm4ZbsJkQkWuzOsqBfkj4BbBIouRh/20lO/Oj7sK3gv+rC1Hx6arFj4NrFJ3VhJf36Iwm+bUMzQwR3AEbFfov6UwBA2h0/T4KTn+4txMt4YEsQLWA5FufaPSRLvMqEqwXPQrL6IXNC1XPDK9s3PGOrsKPhAskJNVM//Bij2N0M8r3+Z49U1+vTEJZY07zm3Zo4FZomX99dYUaqlLVc84F/YO3vM0qNh3n7WHuTUT8CCDBwGhs69rKPv074dFN5RWgy8HuVCH3za7OSihM9zO7CE4AErhe/LPU5ySNKM0mrXYzLn9bDgKwDwD6Hz4YQDr+0JTpWY4NX/w3kpIINxHB9z1Ga+AlybcPQncqB6xc9FBTI5qHkavwgyb+ZW4O5NZ/A+gavOWDkalH56Syqpxp2MJivOD8prvGR/RImUBl4EZWnWXRfRoxAcuklRnPv8M9wjSLTMaqqWRYl79ZpCN5qBfbwHQ43jE71Af9M/0HxAMz6LGuwmCUo1qdva7XLl4dcm9V5FmMwxRwCcoEclNeAgtaZAGpu2UCgg6E8skagNvKqQltTtT+uRxrlWIWdEH4zyIwQ3R8/NnEWfdkVnxhDl3Ae0BfTI+TL+KoxkozeN/Fbm3NL9lyaKXWYHf+lJv581u/Iuri4jzyemp6ql8h6uzYAi1j+8JGxrGrJK+bRuvM9H8VZdG5LDeC90vGQnJ7rfGIbqJoS3InbJIQwIkASTylOvGG+vW5JLRM+SUYi6XsYFhvDB07D5GLnNYcj00lMgfQxdvebblRv6jEdP7DKsazoCfW+tPvKfyv7rJdQOTZC93PMMnOhHzWQYyqZ9z5lMZdt0JznOrlcXJkqd419j00KiZXseSQV4l+H+KzpGBh1YAjxWfGWaAgCRoLgACvJ7N/4bACy4jImDiqBMGlHmfnjS+c+hwWhIuajeCXDxj8vZ/kO+W6PScuiiYY6TcIGIsOFawmPiXDWgVMVqtm7hRVWDClHfhHaEsB8iSMhBRbdQs/jpy6r8DkRGfZU+ZbP7/IznW/FfBABc9Gz55hCQ/lqiDGWNX47LCy262Nhc1gqKpYR7BJnakfg/XCSENeiYL+0PPWlzzRX0N3NNLzdXSdhjtshfoSkMYPalTUxU3ooXvv4kRhG1UsFt1E4KwyQ8VTsF800y69jNsZk2jd1uYEUcQvjqNzjnSdxKjcINHvqaVqXc/v//Dhz58MDdQFrYsaSMMdTMTBvhtCWQlsNHA0kjwZAKO0dRtvFgXnD4anZbRxsOJ2sq/xtqXdXQ6WgpITEvYoC7Ik746n8S7V9nJeybVrZd+yTfjWQaLWXXdNpbFNTm9NO1Oq9498qdlPFScWp330aR7ZBg0XoOi6QAoaMj6ToQDSHPze2ag3V1Cb78QBBvfEEiRC6zDrX4EEwa9YXdvi76FiXKaYuAh1WuBxFyNMYoL20fP0PlKRY4rx9uInztAkOHlqPSOobdtApWhgj2LgpE5m84NzTaibOK8bNxe6rZe5y6GEHKTfWUBzJWTbqBhQvMqfAnuH739ZveWgNgv2eEPBmW5MGMKaYvrtp9BfvkRXBv1YkcWKC6wjglvvwMTgJ2u1uDddo3apTmU3DGVVBSTxnmMJxOcE1Tk9AW08kDQ79cIBAIWZImXCZsJMeOUBTyKcBruKWfJu7IPhkiYA4JUEW8wQtje6Gg6620qxKV5CGNSkWnRDTVSLcuEE4Il/YOtihxYSGGMfvHj7VxpP4n5+Ee/9G9iRMvnZtcoudHn+6l/tNp+e6bqNfDbaDvIffEXDOb+sbu4FUdwyO+/Ak3uGoiDRGigL6lmyKMwgHstud5sdOWD0XqraBTZfleYsrrwJbVzKjEW1nakDP+HvL3wGLCC2o6FLr/gFfGlEmQ1zItcJOOQWG+YCEnpiKIXz1DjVJoCIeTc1c2U300CHknENXOjtKSPQWmovubEGHSM52oa+0tGg02IQz7b1Qk9YFe2unUAt5goO96aYlISLyVO/P6MQSzK1Fkwte5KdTcj4BiHVZ2RkM/UzVvvUBydnkWaEtMre8mPypPO6uAKuM1sf1svzarwO9z9v5bLG0JfJzmSjQ6gtWG4aYXMPnZ3GmoXb5Hp9zbzfxg/54XEPrQKW3ogJAic+E0rD5bNascD/adbV1aUCd/9RTr+IvtYn8SU23W6z9EaXcyWEx0K04nxRwIFmQZIKDpxIPeGr19TSkXKWkqqwEbhGrb5tPskwpSYDaTSnHZKCkfxVCa5tWnGVpOcs+OFkqJUajWiwZL52MD6W06JZBFmyf9OkSE5Wg0LpFfWGzFJ9CWJCUGjzl60ERhTYVAAoJZDeQWmw7Sk9Z5h7pPyIWYdizBsXik/FSmST7MyhYrSgQ79idMx+ht3LEzmCy/YZ8zEbp/HShyDKOtMkKFPXG193Q8eGgEwms+U7M5ameFn7xzy81fK7Zq6Kl86ArxSTYZuTXi75PvL3UBO+4fBk/CQG4sQeDo6wIYKiZylMf33Jg1yitErDWCYGiTcxjdBbnapFK6cNudut2n+5DoqzADSxBDGLHaLH/XUx9N1+eCZvSBbkL+b03GeyHL6Tdd3R3Ob/iPrr+PrPAjDs63wj/iWAcYQcl0DXM3G2ZPs7UcbgRorrZ7FSbAxPgiRR6qlp7yGsV3IuddaRh+5psImy4ybw2V6KJYAnoyaEx2PJlIEJHj/b2BohSssODbTmgpP1oIrtdD3blOzPWj0qDUvPs/OWj2Zamqh8X4U0wARtaLEvpcvE3IoTnaXWgoj3cQSkYlFyWduRGlRewZCfojZ9JufkhM1X6nrqzgk/iSKJK27kDydwMzR49/etxZZRyai8cDGVx+SojeHP6x5sbfunvrX1Uh4CLIIOKUaK5nMJNrJvlyh2FJJhPq7hAz8/KA66cGFU6ADUTiZ6SqONy5eZyJP2vignbZbUsZM0d4Kx6bSsnqRVhGfTlRzbR3Ye75Wd7k47/ZSmeV1pAwSaQ46FauXJrq/zV+IqeNAhkinEm0UVaeh/q+6tiQe29N7A5L/rmsBXHpgjUXS0TmPppfNonbrV8odOwm9LwfSWUlXBe+zE/9fvmdLhHj6uJ6ZUpyY0ehDlbbeKkchYsL7YPlnhVd9vIwePsibXawZ54Gh5Iz9o4jAo0IWS0l0GL9O/qNrOaZuh/GHC3A2jQuYLc0i4uBfdPrdGT72DUTWz4sSxBsAY+UT2VxHsrvVOxv0crAdU6N10QaP/MLZktJA+O00TX/y2yYU0BzIK8+6KjqUD9XwMi5qw9USYcwoFj8FKcQrAYyQva6of9cjBbq5BK6YmiXiLuAlQl2tN+mYgb9TQB5Fea7McDDBqqfPmiKcmuqcaxtN2DDer1HgBd7rIyApLGk89ox39cA3OkBp7j0PrKdds1RQzPtf7zBFw9KIKAD/adnkvd1CoJbPTn1/plVqmZ5W/9z3bjEcIFkD4uod34h7N/oUftCdYfJ7SIc7rL0B7chqXaAl7r54Q7ZWxYOz4gFADwDwl8brK+pvYlFg4uiKaXpWT52cJY75rzVb+P4PfQzYlVyIjc92KWoZ6pOhhAXV1jPtqAKDU/S6dHS0tZJF9y7we5kON0w8FzcmeFxbkKihi6r2L788INapKcG0bf6bx9OzW8vJqRMYX/OFnSepJbhpt/VA3mFwMMJT56KcfvWebo/XbRHPrjC1UGDM4okJpbyEpmIXJDxh7k7ctWo3xqfQD9f2lDdMB0UWY28CpL2nCbkc7Pl+905DHNXx0HKS/dLf9ofVi4dzyw8oDLr1vFYHiiQN2AsMhnH800eQBqOe9zoAoUSa/ZJ9QQsURy6G5+Jz5IP7mIZGu5vMgjd6PDRv1TjQ5z8rPTkRo22y0Ggi3hFgrHCwDnsiBDAhbVYN5rc+x2Ppc4BKfq3mLk9L5knRvCqpkyR/asHK/21sdiaouwyn9O5thTcWLdmhCdh7WhXtF10pEXO0PRNKzMXDfBj3JQwKNTTsPLMuJlCbrp9N3Lp8ObKxrfj4bt3GI7174xLwXjaLqpudgWK1YJ7Y=</xenc:CipherValue>
   </xenc:CipherData>
</xenc:EncryptedData></saml:EncryptedAssertion></samlp:Response>

@danielcompton
Copy link
Author

@edpaget thanks for the file. Do I need to do something else with it? I'm getting a test failure with it:

ERROR in (assert-valid-signatures-test) (response.clj:55)
valid signed responses should pass
 Response with SIGNED message, not-malicious SIGNED ENCRYPTED assertion
(saml20-clj.test/response {:message-signed? true, :assertion-signed? true, :assertion-encrypted? true})
signature should be valid when checking against IdP cert
expected: (instance? Response (response/validate response test/idp-cert test/sp-private-key {:response-validators [:signature :require-signature], :assertion-validators [:signature]}))
  actual: clojure.lang.ExceptionInfo: Invalid <Response> signature

@edpaget
Copy link

edpaget commented Feb 21, 2025

@danielcompton it looks like the indentation changed a bit in the version of the file you checked in which makes the signature invalid. I'll send a PR to your branch

@edpaget edpaget self-assigned this Feb 21, 2025
@edpaget
Copy link

edpaget commented Feb 21, 2025

danielcompton#1

@edpaget
Copy link

edpaget commented Feb 21, 2025

Also there seems to be some issue with cloverage and xml parsing.

@danielcompton
Copy link
Author

danielcompton commented Feb 21, 2025

@edpaget I've resolved the merge conflict and squashed the commits, I think this is good to go now? CI: https://github.com/danielcompton/saml20-clj/actions/runs/13465086881

edpaget
edpaget previously approved these changes Feb 21, 2025
Copy link

@edpaget edpaget left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind also updating the VERSION.txt to 3.0.0 since the release will be run after master builds.

deps.edn Outdated
org.opensaml/opensaml-saml-impl {:mvn/version "4.3.2"}
org.opensaml/opensaml-xmlsec-api {:mvn/version "4.3.2"}
org.opensaml/opensaml-xmlsec-impl {:mvn/version "4.3.2"}
;org.apache.santuario/xmlsec {:mvn/version "4.0.2"} ; use latest version and override transient dep from OpenSAML
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mind deleting these commented out deps.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked, and opensaml 5.1.3 still hasn't bumped them past this version, and even includes a version of xmlsec with a CVE. So I uncommented them to keep them.

https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview

Updates signed and encrypted SAML assertion as it wasn't valid under
OpenSAML 5.

Also typehint getchildnodes for cloverage. cloverage fails with
a reflection issue without it.

Co-authored-by: Edward Paget <ed.paget@gmail.com>
@danielcompton
Copy link
Author

Do you mind also updating the VERSION.txt to 3.0.0 since the release will be run after master builds.

Yep, done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants