Test Overview

JUnit Tests

Plugin:

org.eclipse.codewind.test

Goal:

• To test that the interface between the IDE and Codewind is working properly. It is not meant to cover full testing of the IDE.
• For this reason not all project types are covered by the tests since for the most part the interaction between the IDE and Codewind does not change based on the project type.

Utilities:

• CodewindUtil:
  • Utilities to wait for build states and application states
  • Utilities to make sure the state does not change over a period of time (if auto build is disabled and a file is changed then the state should not change)
  • Utilities to clean up at the end of a test - remove all of the applications, remove the connection if remote
• ImportUtil:
  • Utilities for importing projects into Eclipse
• TestUtil:
  • print methods to use instead of calling System.out.println (that way printing can be changed later)
  • print is used to log information as the test progresses (very useful when the tests are run as part of the build)
  • file utilities for updating source files (when testing auto build for example)
  • utilities for waiting for jobs to finish

BaseTest:

• All tests inherit from this class
• Includes all of the code for:
  • getting a connection
  • creating applications
  • changing the run mode
  • opening all of the log files
  • checking the application (state, mode, pinging the app, log files, etc.)

BaseAutoBuildTest/BaseAppsodyAutoBuildTest:

• Has all of the steps for the test case
• Creates an application and checks that it goes to running state and can be pinged, etc.
• Disables auto build (Codewind style projects only)
  • Modifies a file and makes sure no build starts and that the old content is there when the app is pinged
  • Does a manual build and makes sure the changes get picked up
  • Re-enables auto build
• Modifies a file and makes sure the changes get picked up automatically
• Extended version (only enabled for go project right now because otherwise takes too long):
  • Tests the app overview page
  • Disables the project and checks that the app is not available (can't ping, logs should be removed, etc. - though the app overview is still there)
  • Re-enables the project and checks that the app is available
  • Removes the project and makes sure the app is not available and the app overview page should be gone now
  • Adds the project back as an existing project and does a full check on the app

BaseDebugTest/BaseAppsodyDebugTest:

• Creates an application and checks that it goes to running state and can be pinged, etc.
• Switches to debug mode and checks the app is in debugging state, checks the mode, etc. as well as that there is a launch for the app with a debug target that has an active process
• Modifies a source file and makes sure that the app returns or stays in debugging state, makes sure the debugger is attached
• Modifies a file that requires a full build (such as Dockerfile) and makes sure the app returns to debugging state and the debugger is attached (Codewind style projects only)

Individual Test Cases:

• Most only set some the variables needed to be able to run the test for that particular project type such as:
  • The project template to use
  • The relative URL to ping the app
  • The location of the file to modify and the new text
• The variables that need to be set are identified in the base test case (BaseAutoBuildTest, BaseDebugTest, etc.)
• Some have special setup such as creating a new endpoint that can be used to ping the app and update it

Setup for Remote:

• Set the following system properties:

     -Dcwtest.connectionType=remote
     -Dcwtest.remoteURL=<url>
     -Dcwtest.remoteUser=<user>
     -Dcwtest.remotePassword=<password>

• The tests have been run with local kubernetes successfully but a remote cluster may require updating a lot of the timeouts

Test Lists:

• AllTests: includes all of the test cases
• BuildVerificationTests: A subset of the tests that covers most of the interaction between the IDE and Codewind but runs fairly quickly. Local tests only.