Feature Test Summary - Add option to process war manifest class path like WebSphere

[tjwatson]

Test Strategy

This the test summary for epic Add option to process war manifest class path like WebSphere

The feature is to ease migration from EE implementations that place Class-Path JARs from WARs onto the EAR classloader. The tests validate that various configurations of the WAR class-path behave like such EE implementations (like traditional webspere).

The following libraries and WARs are defined for a test EAR:

Test libraries:
lib1 - Class-Path-From: war1, war2
lib2 - Class-Path-To: lib3; Class-Path-From: war1, war2, war3
lib3 - Class-Path from lib2
lib4 - Class-Path-To: lib6; implicitly included as part of the EAR lib/ folder
lib6 - Class-Path-From: lib4
lib5 - Class-Path-To: ../lib11.jar; included in war3 WEB-INF/lib
lib7 - configured as a private library
lib8 - configured as a common library
lib9 - not referenced
lib10 - Class-Path-From: rar1
lib11 - Class-Path-From: lib5; included in war3 WEB-INF/
lib12 - nested in rar2 testlib/ folder
lib13 - Class-Path-From: war3
lib14 - Class-Path-From: eb1
lib15 - Class-Path-From: client1
lib16 - Class-Path-From: testResourceAdaptor
lib17 - included in testResourceAdaptor

Test WARs
war1 - Class-Path: lib1.jar lib2.jar
war2 - Class-Path: lib2.jar lib1.jar; includes WEB-INF/lib/ejb3.jar
war3 - Class-Path: lib2, ejb2, lib13; includes WEB-INF/lib/lib5.jar, WEB-INF/lib11.jar

Test EJBs
ejb1 - Class-Path-To: lib14; implicitly included as an EJB in the EAR
ejb2 - Class-Path-From: war3

test RARs
rar1 - Class-Path-To: lib10; includes testlib/lib12.jar, testResourceAdaptor.jar
testResourceAdaptor - Class-Path-To: testlib/lib12.jar, ../lib6.jar; includes lib17.jar

List of FAT projects affected

• com.ibm.ws.classloading_classpath_fat
• com.ibm.ws.cdi.visibility_fat

Test strategy

• What functionality is new or modified by this feature?

New fuctionality was added to the enterpriseApplication configuration for Open Liberty. A new attribute was added named webModuleClassPathLoader. This attribute as two valid values of "ear" and "war". The value of "war" is the default value which retains the current behavior of Liberty. Tests have been added to verify the new class loader behavior when using the webModuleClassPathLoader attribute for the default, "war", "ear" and an invalid value.

• What are the positive and negative tests for that functionality? (Tell me the specific scenarios you tested. What kind of tests do you have for when everything ends up working (positive tests)? What about tests that verify we fail gracefully when things go wrong (negative tests)? See the Positive and negative tests section of the Feature Test Summary Process wiki for more detail.)

com.ibm.ws.classloading_classpath_fat:
This is a new FAT developed to test the new class loader behavior for EARs

The order of the class-path for the EAR and WAR loaders are different between tWAS and Liberty with respect to where the referenced libraries (from Class-Path headers) get placed in the overall search order. Additionally the Class-Path references from the WAR manifest are added to the EAR loader for tWAS. When using the webModuleClassPathLoader value of "ear" the behavior in Liberty is to mimic how the class loaders behave in tWAS.

Positive Tests

• Verify the class path search order is preserved for the current Liberty behavior when using the "war" value or the default.
• Verify the expected classes included in the Class-Path specified by the WARs are loadable with the WAR class loader when using the "war" value
• Verify the class path search order changes to the tWAS order when using the "ear" value.
• Verify the expected classes included in the Class-Path specified by the WARs are loaded with the EAR class loader when using the "ear" value
• Verify the expected classes included in Class-Path specified by one WAR are reachable (from class loader delegation) for all WARs in the EAR when using the "ear" value

Negative Tests

• Verify the expected classes included in Class-Path specified by one WAR are NOT reachable (from class loader delegation) for other WARs in the EAR when using the "war" value
• Verify the expected error message CWWKG0032W is displayed when using an invalid value for webModuleClassPathLoader

com.ibm.ws.cdi.visibility_fat:
This is an existing FAT that tests visibility rules for CDI for the various modules contained in an EAR. For webModuleClassPathLoader using the "ear" value we wanted to be sure the CDI visibility rules did not change. To do that we repeated all the existing tests in com.ibm.ws.cdi.visibility.tests.vistest.VisTest in a new test class com.ibm.ws.cdi.visibility.tests.vistest.VisTestAddWarCPToEar that uses the "ear" value.

• What manual tests are there (if any)? (Note: Automated testing is expected for all features with manual testing considered an exception to the rule.)

None

Confidence Level

Collectively as a team you need to assess your confidence in the testing delivered based on the values below. This should be done as a team and not an individual to ensure more eyes are on it and that pressures to deliver quickly are absorbed by the team as a whole.

Please indicate your confidence in the testing (up to and including FAT) delivered with this feature by selecting one of these values:

0 - No automated testing delivered

1 - We have minimal automated coverage of the feature including golden paths. There is a relatively high risk that defects or issues could be found in this feature.

2 - We have delivered a reasonable automated coverage of the golden paths of this feature but are aware of gaps and extra testing that could be done here. Error/outlying scenarios are not really covered. There are likely risks that issues may exist in the golden paths

3 - We have delivered all automated testing we believe is needed for the golden paths of this feature and minimal coverage of the error/outlying scenarios. There is a risk when the feature is used outside the golden paths however we are confident on the golden path. Note: This may still be a valid end state for a feature... things like Beta features may well suffice at this level.

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.

5 - We have delivered all automated testing we believe is needed for this feature. The testing covers all golden path cases as well as all the error/outlying scenarios that make sense. We are not aware of any gaps in the testing at this time. No manual testing is required to verify this feature.

Based on your answer above, for any answer other than a 4 or 5 please provide details of what drove your answer. Please be aware, it may be perfectly reasonable in some scenarios to deliver with any value above. We may accept no automated testing is needed for some features, we may be happy with low levels of testing on samples for instance so please don't feel the need to drive to a 5. We need your honest assessment as a team and the reasoning for why you believe shipping at that level is valid. What are the gaps, what is the risk etc. Please also provide links to the follow on work that is needed to close the gaps (should you deem it needed)

Confidence Level is rated at 4.