Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce expected test sets are included in CI and nightly lab runs #1304

Closed
2 tasks done
zhenlan opened this issue Jun 16, 2016 · 4 comments
Closed
2 tasks done

Enforce expected test sets are included in CI and nightly lab runs #1304

zhenlan opened this issue Jun 16, 2016 · 4 comments
Assignees
Labels
infrastructure Issues related to the build, packaging, testing or related areas.
Milestone

Comments

@zhenlan
Copy link
Member

zhenlan commented Jun 16, 2016

For each testing environment (eg. Windows, OS X, etc.), we will have expected set of tests to be included. We should enforce this by setting the specific conditions to true so those tests attributed with corresponding ConditionFact will always be included.

@zhenlan zhenlan added the infrastructure Issues related to the build, packaging, testing or related areas. label Jun 16, 2016
@roncain
Copy link
Contributor

roncain commented Jun 16, 2016

At the moment, the following conditions are used in ConditionalFact and will be auto-detected if left unspecified.

Root_Certificate_Installed
Client_Certificate_Installed
Server_Domain_Joined
Domain_Joined
Windows_Authentication_Available
Basic_Authentication_Available
Digest_Authentication_Available
NTLM_Available
SPN_Available
UPN_Available
Ambient_Credentials_Available
Explicit_Credentials_Available

Auto-detection can be avoided by explicitly setting these values 'true' or 'false' when build.cmd is called. Example:

    build.cmd /p:Root_Certificate_Installed=true

ConditionalFact tests for each value obey these rules:

  1. If unspecified, auto-detect and either run or skip the test based on the result
  2. If 'true', always run the test
  3. If 'false', always skip the test

The complete set of ConditionalFact conditions can be found here

@zhenlan zhenlan added this to the 2016.06 milestone Jun 21, 2016
@hongdai
Copy link
Contributor

hongdai commented Jun 28, 2016

Update on Helix lab: I'm researching on options to enable cert tests on non window machines.

@hongdai
Copy link
Contributor

hongdai commented Jul 7, 2016

Updates:

  1. Have made changes to VSO build to enable certs for Windows OS.
  2. We have identified 2 possible solutions for Linux cert installation.
    a. install it via process API in the test run time.
    b. install it via a set up script.

b) is preferable if it's not very complicate to implement. Investigating it with Helix owner.

@iamjasonp
Copy link
Member

Done! Closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Issues related to the build, packaging, testing or related areas.
Projects
None yet
Development

No branches or pull requests

4 participants