Skip to content

Conversation

davidnguyen-tech
Copy link
Member

@davidnguyen-tech davidnguyen-tech commented Aug 14, 2025

Description

We agreed to disable failing test in System.Security.Cryptography.Tests.csproj for CoreCLR Android x64 (emulators) in #118603. At first glance, there were already project exclusions for the tests, but the tests were still being executed on the CI.

This PR finally disables ImportWithPasswordOrFileName_IterationCountLimitExceeded from System.Security.Cryptography.Tests.csproj and HTTP tests by removing them from Smoke Tests for CoreCLR Android.

Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

@davidnguyen-tech davidnguyen-tech changed the title [Android][CoreCLR] Remove System.Security.Cryptography.Tests.csproj from Smoke Tests [Android][CoreCLR] Disable System.Security.Cryptography.Tests.csproj for emulators Aug 14, 2025
@davidnguyen-tech davidnguyen-tech changed the title [Android][CoreCLR] Disable System.Security.Cryptography.Tests.csproj for emulators [Android][CoreCLR] Disable System.Security.Cryptography.Tests.csproj for emulators (x64) Aug 14, 2025
@davidnguyen-tech
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@davidnguyen-tech
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@davidnguyen-tech davidnguyen-tech changed the title [Android][CoreCLR] Disable System.Security.Cryptography.Tests.csproj for emulators (x64) [Android][CoreCLR][x64] Disable System.Security.Cryptography.Tests.csproj for Aug 14, 2025
@davidnguyen-tech davidnguyen-tech changed the title [Android][CoreCLR][x64] Disable System.Security.Cryptography.Tests.csproj for [Android][CoreCLR][x64] Disable System.Security.Cryptography.Tests.csproj Aug 14, 2025
@kotlarmilos kotlarmilos self-requested a review August 14, 2025 15:53
Copy link
Member

@kotlarmilos kotlarmilos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@PranavSenthilnathan
Copy link
Member

Do we need to exclude the entire project? Looking at the logs, the low memory killer seems to kick in when ImportWithPasswordOrFileName_IterationCountLimitExceeded is run, so can we just try skipping that test?

logs
08-14 12:18:15.536  5502  5556 I DOTNET  : Test collection for System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection
08-14 12:18:15.536  5502  5556 I DOTNET  : Test collection starting
08-14 12:18:15.536  5502  5556 I DOTNET  :    Test collection: Test collection for System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.Import_IterationCounLimitNotExceeded_Succeeds
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.Import_IterationCountLimitExceeded_Throws
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.Import_NonNullOrEmptyPasswordExpected_Throws
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ExportedPfxWithNullPassword_DecryptReturnsValidPaddingWithEmptyPassword
08-14 12:18:15.536  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.Import_BlobHasMoreThanOnePfx_LoadsOnlyOne
08-14 12:18:15.536  5502  5556 I DOTNET  : Test class starting
08-14 12:18:15.536  5502  5556 I DOTNET  :    Class name: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection
08-14 12:18:15.536  5502  5556 I DOTNET  :    Class assembly: System.Security.Cryptography.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
08-14 12:18:15.536  5502  5556 I DOTNET  :    Class assembly path: System.Security.Cryptography.Tests.dll
08-14 12:18:15.537  5502  5556 I DOTNET  : Test method starting
08-14 12:18:15.537  5502  5556 I DOTNET  :    Class name: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection
08-14 12:18:15.537  5502  5556 I DOTNET  :    Class assembly: System.Security.Cryptography.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
08-14 12:18:15.537  5502  5556 I DOTNET  :    Class assembly path: System.Security.Cryptography.Tests.dll
08-14 12:18:15.537  5502  5556 I DOTNET  :    Associated test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.537  5502  5556 I DOTNET  : Test case starting
08-14 12:18:15.537  5502  5556 I DOTNET  :    Starting test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.541  5502  5556 I DOTNET  : Test starting
08-14 12:18:15.541  5502  5556 I DOTNET  :    Test name: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded(name: "Pkcs12NoPassword600KPlusOneRoundsHex", password: null, usesPbes2: True, blob: [48, 130, 9, 210, 2, ···], iterationCount: 1800003, usesRC2: False)
08-14 12:18:15.541  5502  5556 I DOTNET  :    Test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.541  5502  5556 I DOTNET  : Test class construction starting
08-14 12:18:15.541  5502  5556 I DOTNET  :    Test name: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded(name: "Pkcs12NoPassword600KPlusOneRoundsHex", password: null, usesPbes2: True, blob: [48, 130, 9, 210, 2, ···], iterationCount: 1800003, usesRC2: False)
08-14 12:18:15.541  5502  5556 I DOTNET  :    Test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.542  5502  5556 I DOTNET  : Test class construction finished
08-14 12:18:15.542  5502  5556 I DOTNET  :    Test name: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded(name: "Pkcs12NoPassword600KPlusOneRoundsHex", password: null, usesPbes2: True, blob: [48, 130, 9, 210, 2, ···], iterationCount: 1800003, usesRC2: False)
08-14 12:18:15.542  5502  5556 I DOTNET  :    Test case: System.Security.Cryptography.X509Certificates.Tests.PfxIterationCountTests_X509Certificate2Collection.ImportWithPasswordOrFileName_IterationCountLimitExceeded
08-14 12:18:15.737  5502  5512 I ptography.Test: Background young concurrent copying GC freed 23987(1570KB) AllocSpace objects, 1(524KB) LOS objects, 3% free, 3315KB/3448KB, paused 621us total 156.990ms
08-14 12:18:16.001  5502  5512 I ptography.Test: Background concurrent copying GC freed 86596(4033KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 2802KB/5604KB, paused 782us total 254.263ms
08-14 12:18:16.275  5502  5512 I ptography.Test: Background young concurrent copying GC freed 83280(3935KB) AllocSpace objects, 0(0B) LOS objects, 47% free, 2914KB/5604KB, paused 919us total 127.812ms
08-14 12:18:16.566  5502  5512 I ptography.Test: Background concurrent copying GC freed 83347(3937KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 2912KB/5824KB, paused 629us total 153.371ms
08-14 12:18:16.870  5502  5512 I ptography.Test: Background young concurrent copying GC freed 88058(4159KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 2912KB/5824KB, paused 1.210ms total 143.184ms
08-14 12:18:17.717  5502  5512 I ptography.Test: Background concurrent copying GC freed 131782(6208KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 5088KB/10176KB, paused 731us total 696.797ms
08-14 12:18:18.019  2043  2057 I ndroid.systemu: Background young concurrent copying GC freed 141333(3545KB) AllocSpace objects, 0(0B) LOS objects, 40% free, 5464KB/9112KB, paused 657us total 140.832ms
08-14 12:18:18.131  5502  5512 I ptography.Test: Background young concurrent copying GC freed 180895(8511KB) AllocSpace objects, 0(0B) LOS objects, 70% free, 2624KB/8768KB, paused 645us total 130.487ms
08-14 12:18:18.619  5502  5512 I ptography.Test: Background young concurrent copying GC freed 150519(7087KB) AllocSpace objects, 0(0B) LOS objects, 64% free, 3120KB/8768KB, paused 716us total 150.589ms
08-14 12:18:19.114  5502  5512 I ptography.Test: Background young concurrent copying GC freed 150519(7087KB) AllocSpace objects, 0(0B) LOS objects, 67% free, 2864KB/8768KB, paused 960us total 146.122ms
08-14 12:18:20.044  5502  5512 I ptography.Test: Background young concurrent copying GC freed 131695(6207KB) AllocSpace objects, 0(0B) LOS objects, 61% free, 3057KB/7888KB, paused 665us total 138.188ms
08-14 12:18:20.430  5502  5512 I ptography.Test: Background young concurrent copying GC freed 131746(6207KB) AllocSpace objects, 0(0B) LOS objects, 61% free, 3073KB/7888KB, paused 903us total 141.208ms
08-14 12:18:20.821  5502  5512 I ptography.Test: Background young concurrent copying GC freed 170595(8015KB) AllocSpace objects, 0(0B) LOS objects, 84% free, 1169KB/7313KB, paused 937us total 132.686ms
08-14 12:18:22.063  5502  5512 I ptography.Test: Background young concurrent copying GC freed 127651(6015KB) AllocSpace objects, 0(0B) LOS objects, 40% free, 4353KB/7313KB, paused 1.067ms total 211.676ms
08-14 12:18:22.466  5502  5512 I ptography.Test: Background concurrent copying GC freed 196917(9249KB) AllocSpace objects, 0(0B) LOS objects, 55% free, 1248KB/2784KB, paused 710us total 245.296ms
08-14 12:18:23.029  5502  5512 I ptography.Test: Background young concurrent copying GC freed 31059(1487KB) AllocSpace objects, 0(0B) LOS objects, 0% free, 5200KB/5200KB, paused 609us total 299.635ms
08-14 12:18:23.247  5502  5512 I ptography.Test: Background concurrent copying GC freed 85990(4048KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 3952KB/7904KB, paused 1.620ms total 217.374ms
08-14 12:18:23.826  5502  5512 I ptography.Test: Background young concurrent copying GC freed 120641(5688KB) AllocSpace objects, 0(0B) LOS objects, 5% free, 7495KB/7904KB, paused 1.316ms total 355.164ms
08-14 12:18:24.577  5502  5512 I ptography.Test: Background concurrent copying GC freed 72146(3310KB) AllocSpace objects, 0(0B) LOS objects, 27% free, 16MB/22MB, paused 631us total 736.118ms
08-14 12:18:26.521  5502  5512 I ptography.Test: Background young concurrent copying GC freed 119151(5617KB) AllocSpace objects, 0(0B) LOS objects, 0% free, 40MB/40MB, paused 751us total 1.636s
08-14 12:18:27.043  1721  1721 E netmgr  : Failed to open QEMU pipe 'qemud:network': Invalid argument
08-14 12:18:27.413  1735  1735 E wifi_forwarder: RemoteConnection failed to initialize: RemoteConnection failed to open pipe
08-14 12:18:30.026  1719  1719 E lowmemorykiller: Kill 'com.android.email' (3323), uid 10107, oom_adj 995 to free 63752kB

@kotlarmilos kotlarmilos marked this pull request as ready for review August 14, 2025 16:44
@Copilot Copilot AI review requested due to automatic review settings August 14, 2025 16:44
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR disables the System.Security.Cryptography.Tests.csproj test project for CoreCLR Android x64 emulators by removing it from the smoke tests configuration. The change addresses issue #118603 where these tests were agreed to be disabled but were still executing in CI despite existing project exclusions.

Key changes:

  • Removes the System.Security.Cryptography test project from Android CoreCLR smoke tests
  • Also removes System.Net.Mail and System.Net.Http test projects from smoke tests (likely related cleanup)
Comments suppressed due to low confidence (1)

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

@kotlarmilos
Copy link
Member

/azp run runtime-extra-platforms

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos kotlarmilos changed the title [Android][CoreCLR][x64] Disable System.Security.Cryptography.Tests.csproj [Android] Disable ImportWithPasswordOrFileName_IterationCountLimitExceeded and HTTP tests Aug 14, 2025
@kotlarmilos
Copy link
Member

kotlarmilos commented Aug 14, 2025

Do we need to exclude the entire project? Looking at the logs, the low memory killer seems to kick in when ImportWithPasswordOrFileName_IterationCountLimitExceeded is run, so can we just try skipping that test?

Thanks, let's test it. /cc: @akoeplinger I remember we disabled a single test recently due to OOM issue.

@akoeplinger
Copy link
Member

yeah sounds good

@kotlarmilos
Copy link
Member

/ba-g Known issue reported #118766

@kotlarmilos kotlarmilos merged commit da322a2 into dotnet:main Aug 15, 2025
82 of 84 checks passed
@davidnguyen-tech davidnguyen-tech deleted the disable-test branch August 15, 2025 09:34
Assert.Contains(FwlinkId, ce.Message);
}

[SkipOnPlatform(TestPlatforms.Android, "Android emulators report OOM error")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reference the issue tracking this problem?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, here it is: #118836

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think longer term we should add some PlatformDetection.IsLowMemoryDevice helper or something like that, we've been using the Is64BitProcess in a couple places as a proxy and just assumed they'd have enough RAM.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants