-
Notifications
You must be signed in to change notification settings - Fork 537
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872)
What do we want? (with apologies to 48e3fc2) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
- Loading branch information
Showing
29 changed files
with
700 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| using System; | ||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.IO.Compression; | ||
| using System.Linq; | ||
| using System.Reflection; | ||
|
|
||
| using NUnit.Framework.Api; | ||
| using NUnit.Framework.Internal.Filters; | ||
|
|
||
| namespace Xamarin.Android.BclTests | ||
| { | ||
| static partial class App | ||
| { | ||
| internal static string[] ExcludeAssemblyNames = { | ||
| // Not needed; distributed only for "sanity"/consistency | ||
| "nunitlite.dll", | ||
| }; | ||
|
|
||
| internal static IEnumerable<Assembly> GetTestAssemblies () | ||
| { | ||
| yield return typeof (App).Assembly; | ||
|
|
||
| var names = TestAssemblyNames.Except (ExcludeAssemblyNames); | ||
| foreach (var name in names) { | ||
| var a = Assembly.Load (name); | ||
| if (a == null) { | ||
| Console.WriteLine ($"# WARNING: Unable to load assembly: {name}"); | ||
| continue; | ||
| } | ||
| yield return a; | ||
| } | ||
| } | ||
|
|
||
| internal static IEnumerable<string> GetExcludedCategories () | ||
| { | ||
| var excluded = new List<string> { | ||
| "AndroidNotWorking", | ||
| "CAS", | ||
| "InetAccess", | ||
| "MobileNotWorking", | ||
| "NotWorking", | ||
| }; | ||
|
|
||
| if (!System.Environment.Is64BitOperatingSystem) { | ||
| excluded.Add ("LargeFileSupport"); | ||
| } | ||
|
|
||
| return excluded; | ||
| } | ||
|
|
||
| internal static void ExtractBclTestFiles () | ||
| { | ||
| var cachePath = global::Android.App.Application.Context.CacheDir.AbsolutePath; | ||
| if (Directory.Exists (Path.Combine (cachePath, "Test"))) | ||
| return; | ||
|
|
||
| using (var files = typeof (App).Assembly.GetManifestResourceStream ("bcl-tests.zip")) | ||
| using (var zip = new ZipArchive (files, ZipArchiveMode.Read)) { | ||
| zip.ExtractToDirectory (cachePath); | ||
| } | ||
| } | ||
|
|
||
| static string[] ExcludeTestNames = new string[]{ | ||
| // https://jenkins.mono-project.com/job/xamarin-android-pr-builder/1720/testReport/(root)/AssemblyTest/GetReferencedAssemblies/ | ||
| // https://bugzilla.xamarin.com/show_bug.cgi?id=59908 | ||
| // AssemblyName.Flags == AssemblyNameFlags.PublicKey; expected AssemblyNameFlags.None | ||
| "MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies", | ||
| // https://jenkins.mono-project.com/job/xamarin-android-pr-builder/1720/testReport/(root)/WebInvokeAttributeTest/RejectTwoParametersWhenNotWrapped/ | ||
| // https://bugzilla.xamarin.com/show_bug.cgi?id=59909 | ||
| // InvalidOperationException wasn't thrown when it was expected | ||
| "MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped", | ||
| }; | ||
|
|
||
| internal static ITestFilter UpdateFilter (ITestFilter filter) | ||
| { | ||
| if (ExcludeTestNames?.Length == 0) | ||
| return filter; | ||
| var excludeTestNamesFilter = new SimpleNameFilter (ExcludeTestNames); | ||
| return new AndFilter (filter, new NotFilter (excludeTestNamesFilter)); | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| namespace Xamarin.Android.BclTests { | ||
| partial class App { | ||
| public static string[] TestAssemblyNames = { | ||
| @ASSEMBLIES@ | ||
| }; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| Any raw assets you want to be deployed with your application can be placed in | ||
| this directory (and child directories) and given a Build Action of "AndroidAsset". | ||
|
|
||
| These files will be deployed with your package and will be accessible using Android's | ||
| AssetManager, like this: | ||
|
|
||
| public class ReadAsset : Activity | ||
| { | ||
| protected override void OnCreate (Bundle bundle) | ||
| { | ||
| base.OnCreate (bundle); | ||
|
|
||
| InputStream input = Assets.Open ("my_asset.txt"); | ||
| } | ||
| } | ||
|
|
||
| Additionally, some Android functions will automatically load asset files: | ||
|
|
||
| Typeface tf = Typeface.CreateFromAsset (Context.Assets, "fonts/samplefont.ttf"); |
Oops, something went wrong.