Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure:System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice #57187

Closed
Tracked by #64488
VincentBu opened this issue Aug 11, 2021 · 14 comments
Closed
Tracked by #64488

Test failure:System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice #57187

VincentBu opened this issue Aug 11, 2021 · 14 comments
Labels
area-System.Security test-failure
Milestone
Future

Comments

@VincentBu
Copy link
Contributor

Run: runtime-coreclr libraries-jitstress 20210810.1

Failed test:

net6.0-Linux-Release-arm-CoreCLR_checked-no_tiered_compilation-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440

- System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice

Error messags:

Chain is valid on execution 1
Expected: False
Actual:   True


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0& , <>c__DisplayClass28_1& , <>c__DisplayClass28_2& ) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1018
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1024
@VincentBu VincentBu added arch-arm32 area-System.Security os-linux Linux OS (any supported distro) labels Aug 11, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Aug 11, 2021
@ghost
Copy link

ghost commented Aug 11, 2021

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

Run: runtime-coreclr libraries-jitstress 20210810.1

Failed test:

net6.0-Linux-Release-arm-CoreCLR_checked-no_tiered_compilation-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440

- System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice

Error messags:

Chain is valid on execution 1
Expected: False
Actual:   True


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0& , <>c__DisplayClass28_1& , <>c__DisplayClass28_2& ) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1018
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1024
Author: VincentBu
Assignees: -
Labels:

arch-arm32, area-System.Security, os-linux
Milestone: -

@ViktorHofer
Copy link
Member

Failed again in https://dev.azure.com/dnceng/public/_build/results?buildId=1287580&view=ms.vss-test-web.build-test-results-tab&runId=38013546&resultId=178938&paneView=debug

Configuration: net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open

@ViktorHofer ViktorHofer mentioned this issue Aug 11, 2021
Merged
@bartonjs bartonjs removed the untriaged New issue has not been triaged by the area owner label Aug 18, 2021
@bartonjs bartonjs added this to the 7.0.0 milestone Aug 18, 2021
@bartonjs
Copy link
Member

The failure isn't exactly reproducible, but we should make a better special assert for successful chain builds that reports what went wrong when they fail.

@vcsjones
Copy link
Member

we should make a better special assert for successful chain builds that reports what went wrong when they fail.

Well, this particular case is nothing went wrong, but we expected it to (the chain built when it was not expected to).

@bartonjs
Copy link
Member

Oy. One of these days, I'll learn to read.

@bartonjs
Copy link
Member

OK, for "should have failed, but didn't", we probably want to figure out a sensible way of logging all of the generated data to see if it reproduces.

@ViktorHofer
Copy link
Member

Happened again:

  Discovering: System.Security.Cryptography.X509Certificates.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Security.Cryptography.X509Certificates.Tests (found 631 of 684 test cases)
  Starting:    System.Security.Cryptography.X509Certificates.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice [FAIL]
      Chain is valid on execution 1
      Expected: False
      Actual:   True
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(1019,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0& , <>c__DisplayClass28_1& , <>c__DisplayClass28_2& )
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(1025,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice()
  Finished:    System.Security.Cryptography.X509Certificates.Tests
=== TEST EXECUTION SUMMARY ===
   System.Security.Cryptography.X509Certificates.Tests  Total: 1105, Errors: 0, Failed: 1, Skipped: 0, Time: 38.748s

Configuration: net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

https://dev.azure.com/dnceng/public/_build/results?buildId=1341267&view=ms.vss-test-web.build-test-results-tab&runId=39292366&resultId=181986&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

@premun
Copy link
Member

premun commented Dec 7, 2021

Here it is again:

  Discovering: System.Security.Cryptography.X509Certificates.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Security.Cryptography.X509Certificates.Tests (found 640 of 693 test cases)
  Starting:    System.Security.Cryptography.X509Certificates.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice [FAIL]
      Chain is valid on execution 1
      Expected: False
      Actual:   True
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(1019,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0&, <>c__DisplayClass28_1&, <>c__DisplayClass28_2&)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(1025,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice()
  Finished:    System.Security.Cryptography.X509Certificates.Tests
=== TEST EXECUTION SUMMARY ===

Configuration: net7.0-windows-Debug-x86-CoreCLR_release-Windows.10.Amd64.Server19H1.Open

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-62392-merge-c64f3680e65943a2b9/System.Security.Cryptography.X509Certificates.Tests/1/console.f58f08d8.log?sv=2019-07-07&se=2021-12-27T11%3A20%3A09Z&sr=c&sp=rl&sig=GiuYJoqPkFEIYWnQySUeKU2xy6GD1mytEurFunxkZm4%3D

@VincentBu
Copy link
Contributor Author

Failed again in: runtime 20211228.1

Failed test:

net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

- System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice

Error message:

Chain is valid on execution 1
Expected: False
Actual:   True


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0&, <>c__DisplayClass28_1&, <>c__DisplayClass28_2&) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1019
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1025

@jeffhandley jeffhandley mentioned this issue Jan 29, 2022
Closed
37 tasks
@vcsjones
Copy link
Member

This seems to be mostly occurring on Windows 7 now, despite the original issue indicating Linux. Has it happened at all on Linux since the initial report?

@vcsjones vcsjones removed arch-arm32 os-linux Linux OS (any supported distro) labels Jan 29, 2022
@danmoseley
Copy link
Member

danmoseley commented Feb 14, 2022 •
edited
Loading

Happened on Windows 7 for me just now:

❌ System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice 
Exception Message
Chain is valid on execution 1
Expected: False
Actual:   True
CallStack
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0&, <>c__DisplayClass28_1&, <>c__DisplayClass28_2&) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1019
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 1025
Pass Rate
Test has failed 2 out of 42648 runs (0.00%)
Failing Configuration
net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

It's happening about 5x a month:

Execute: Web | Desktop | Web (Lens) | Desktop (SAW)

https://engsrvprod.kusto.windows.net/engineeringdata

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Started >= now(-60d)
| where Result == "Fail"
| where  Method  == "BuildInvalidSignatureTwice"
| where Message startswith "Chain is valid on execution 1"
| project QueueName, Started//, Message
| sort by Started desc
//| summarize count() by Status, QueueName, Message
QueueName Started
windows.7.amd64.open.rt 2022-02-14 03:30:41.4460000
redhat.7.amd64.open.rt 2022-02-09 01:03:28.9210000
windows.7.amd64.open.rt 2022-02-05 15:08:26.8680000
windows.7.amd64.open.rt 2022-02-04 03:50:20.3690000
ubuntu.1804.armarch.open 2022-02-01 22:17:40.0070000
windows.7.amd64.open.rt 2022-01-21 00:19:11.9610000
ubuntu.1804.amd64.open.rt 2022-01-13 20:01:49.3610000
ubuntu.1804.amd64.open.rt 2022-01-06 00:55:41.2290000
ubuntu.1804.armarch.open 2021-12-28 19:48:04.1940000
windows.7.amd64.open.rt 2021-12-28 09:27:07.4490000
windows.7.amd64.open.rt 2021-12-21 13:34:10.1740000

@vcsjones
Copy link
Member

@danmoseley for your kusto query... there are actually two tests with that method name...

runtime/src/libraries/System.Security.Cryptography.X509Certificates/tests/DynamicChainTests.cs

Line 71 in 380cfc9

public static void BuildInvalidSignatureTwice(

and

runtime/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs

Line 951 in 380cfc9

public static void BuildInvalidSignatureTwice()

Is your query lumping them together? If so, what do the results look like when broken up?

@jeffhandley jeffhandley modified the milestones: 7.0.0, Future Aug 9, 2022
@v-wenyuxu
Copy link

Failed again in: runtime-coreclr libraries-jitstressregs 20230226.1

Failed test:

net8.0-windows-Release-x86-CoreCLR_checked-jitstressregs8-Windows.10.Amd64.Open

- System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice

Error message:

Chain is valid on execution 1
Expected: False
Actual:   True


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0&, <>c__DisplayClass28_1&, <>c__DisplayClass28_2&) in /_/src/libraries/System.Security.Cryptography/tests/X509Certificates/ChainTests.cs:line 1024
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography/tests/X509Certificates/ChainTests.cs:line 1030
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 59
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Duration WorkItemFriendlyName Properties BuildReason
2023-01-25T07:32:59.806Z runtime-community Release ubuntu.1804.armarch.open armv6 System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice Failed 00:00:15.0070000 System.Security.Cryptography.Tests {"configuration":"Release","Project":"public","operatingSystem":"(Raspbian.10.Armv6.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:raspbian-10-helix-arm32v6","System.StageAttempt":"1","BuildNumber":"20230124.2","System.JobId":"db8e93a1-262e-5037-1aaf-a14621311ed2","System.JobName":"__default","DefinitionName":"runtime-community","System.PhaseAttempt":"1","DefinitionId":"148","Reason":"Schedule","System.StageName":"Build","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"build_linux_armv6_Release_AllSubsets_Mono","architecture":"armv6","BuildId":"147712","System.JobAttempt":"1","AzurePipelinesTestRunId":"3153993","runtimeFlavor":"mono"} Schedule

@jozkee
Copy link
Member

jozkee commented Mar 17, 2023

Duplicate of #82837

@jozkee jozkee marked this as a duplicate of #82837 Mar 17, 2023
@jozkee jozkee closed this as completed Mar 17, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Apr 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Security test-failure
Projects
None yet
Milestone
Future
Development

No branches or pull requests
9 participants
@vcsjones @jeffhandley @danmoseley @premun @ViktorHofer @bartonjs @jozkee @VincentBu @v-wenyuxu