Cleanup PYSYS test devices also on startup

[haraldmeyer]

It looks like it is very common, that test devices are not properly cleaned up on shutdown especially if the test does not terminate properly. In my case, I have 1500 test devices after a couple of days of test development. It would be good if the test framework deleted all PYSYS devices on startup.

[skom-c8y]

https://github.com/SoftwareAG/apama-eplapps-tools/blob/c6ed29ebdb153ecde696d39a488c6a4e3e1d4ecf/testframework/apamax/eplapplications/basetest.py#L188 is supposed to achieve the exact same behavior, are you saying that its not working as expected ?

Also to note is that the DELETE of managedObjects is asynchronous, so it might take sometime for devices to be completely cleaned up.

[haraldmeyer]

I think I know what the issue is, we are using the base class "ApamaC8YBaseTest" for our test but _deleteTestDevices() is only called by the derived "SimpleTest" classes.

[haraldmeyer]

@skom-c8y Do you think this is a defect?

[skom-c8y]

well, I think we can consider it as one. Technically, we can do it in the BaseTest and make sure that cleanup is done only for simulated/test devices and doesn't impact real devices.

But, moving the cleanup to basetest can also be considered as a change of behavior. Perhaps we can control it via a flag and let customer's enable it if they need it. I will log an internal defect to address it.