Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SharePoint functional test plan for the spo variant is failing/erroring for multiple policies #1093

Open
nanda-katikaneni opened this issue May 9, 2024 · 2 comments
Assignees
Labels
bug This issue or pull request addresses broken functionality Testing This issue or task involves testing the automation tool function
Milestone

Comments

@nanda-katikaneni
Copy link
Collaborator

nanda-katikaneni commented May 9, 2024

🐛 Summary

Ran SharePoint functional test plan for both spo and pnp variants on E5 test tenant. The spo version is failing or erroring for 1.2 and 3.2 policies. The test plan needs to be fixed for clean functional test execution.

To reproduce

Steps to reproduce the behavior:

  1. Run SharePoint functional test suite for spo variant on a E5 tenant
  2. Observe if all complaint and non-complaint test cases are running without error and passing all tests.

Expected behavior

SharePoint test plan in spo variant running without errors and successfully complete all complaint and non-complaint cases.

Any helpful log output or screenshots

Context Execute test, MS.SHAREPOINT.1.2v1 Non-compliant - SharingCapability = ExternalUserSharingOnly (New and existing guests)
VERBOSE: Download the right chromedriver from 'http://chromedriver.chromium.org/downloads'
    [-] Check test case results 10.17s (10.16s|9ms)
     Expected case sensitive like wildcard 'Unknown' to match 'N/A', because unexpected criticality. [Output folder: xxx; Expected: False; Result: N/A; Details: This product does not currently have the capability to check compliance for this policy. See Secure Con
figuration Baseline policy for instructions on manual check], but it did not match.
     at $RowData[2].text | Should -BeLikeExactly "Unknown" -Because "unexpected criticality. [$Msg]", 

```test execution summary
Tests completed in 325.32s
Tests Passed: 43, Failed: 8, Skipped: 0 NotRun: 0

@schrolla schrolla added this to the Iceberg milestone May 9, 2024
@schrolla schrolla modified the milestones: Iceberg, Jellyfish Jul 29, 2024
@schrolla schrolla added bug This issue or pull request addresses broken functionality Testing This issue or task involves testing the automation tool function labels Aug 21, 2024
@nanda-katikaneni nanda-katikaneni self-assigned this Aug 26, 2024
@rgbrow1949 rgbrow1949 modified the milestones: Jellyfish, Kraken Oct 7, 2024
@tkol2022
Copy link
Collaborator

tkol2022 commented Oct 9, 2024

Please drop a comment to describe what this issue is blocked on. Thanks.

@nanda-katikaneni
Copy link
Collaborator Author

nanda-katikaneni commented Oct 10, 2024

SharePoint.1.2.v1 is failing intermittently when setting the tenant with a non-compliant config (specifically for OneDrive external sharing config). Based on the debugging this boiled down to "Set-SPOTenant -DefaultSharingLinkType AnonymousAccess -SharingCapability ExternalUserAndGuestSharing". This test pre-condition is required to be set before the next command is executed; but is failing. Based on an earlier investigation by @mitchelbaker-cisa for a similar issue for pnp variant - issue is boiled down to Microsoft API response. Working with Microsoft team to resolve - until then its blocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue or pull request addresses broken functionality Testing This issue or task involves testing the automation tool function
Projects
None yet
Development

No branches or pull requests

5 participants