GitHub: Onboard std tests

[MahmoudGSaleh]

The devcrt is a test suite used internally to validate new STL features and bug fixes. We need to migrate those tests to Github and onboard them as part of the CI process.

[StephanTLavavej]

Our current consensus is that tests/std would be the proper location/name for devcrt.

[stevemac321]

I scouted what google test does for windows. I don't think google test adds much value, its more about their approach that might be interesting. There cmake create a top level VS solution, with additional projects (gtest, gmock, etc.) and a separate vcxproj for each .cpp test. I added std\VSO_0961751_has_range_erase to the solution and after patching up bin, include, lib paths and preprocessor define _ENABLE_STL_INTERNAL_CHECK, all the projects built, and you could run the resulting executable and get your logging and success code (fail is good here). After setting VSO_0961751_has_range_erase to the active project, was able to debug.
If you don't care about running on other platforms, perhaps an msbuild solution (which you can run at the command line) is interesting. It eliminates runall and test.lst and makes for easy debugging if there is a failure.

[barcharcraz]

One problem with that approach is that our tests actually need to be compiled with several different compilers, including some that we don't support compiling the actual library with. CMake doesn't have great support for this.

Additionally since we want to share tests with llvm we'd like to keep a similar structure to them. Our current plan (as seen in #520 ) is to use llvm's lit test harness with custom code to parse and handle our env.lst files.

[stevemac321]

Glad to hear it. This is not up to date, but about 7 years ago we proved that the libcxx tests provided the same level of coverage as the industry standard conformance tests. Back then, the libcxx tests used a bash shell script to run the tests. I think Eric Feisler (google now) did the lit work, at least for the libs part, maybe for the entire llvm test suite.

[cbezault]

#520 Partially implemented adding this testsuite. The remaining features which need to be implemented are:

• Enable /BE (i.e. EDG) tests
• This can be achieved by enabling vcpkg env to specify you want to use the x86 native compiler or by removing our use of vcpkg env and just calling vcvars ourselves.
• Enable /clr:pure tests
• We need to actually build the clr:pure flavor of the stl in order to run these tests.