[FEATURE] Run same tests for security enabled or disabled cluster

[owaiskazi19]

Is your feature request related to a problem?

Currently, we run different test suites for security enabled tests here, which is mostly checking for the roles and not actually testing the APIs with the response. Whereas our integ tests does that.

What solution would you like?

We should run the same integ tests suite for the plugin with security-enabled or disabled cluster with just changing the local cluster url to pick either https or http protocol based on the flag security-enabled=true or security-enabled=false.

What alternatives have you considered?

The current test suite which just checks for the role based access for security enabled cluster.

Do you have any additional context?

[minalsha]

I thought about this and i think it is alright to have different test for security enabled versus not security enabled. Depending on the test case and what we want to test. Security enabled test need to focus on the special setup brought up by the security such as roles/permission.

With Security enabled, what are we uniquely testing if it is the same test that run without security enabled? Wouldnt we duplicate the work?

[joshpalis]

I agree with your sentiment, but thinking about this again, overall I think having more tests is always good, especially as we add support for more plugins in the future. Running all of our integration tests with security enabled should at the very least help us catch issues early on that would otherwise only be caught during full distribution builds.

[owaiskazi19]

+1 to @joshpalis and the main reason to open this issue in the first place was to find the failure early be it with security enabled/disabled cluster before the distribution build during the release. For the same reason, all the tests need to be run in both type of cluster.

[minalsha]

this is a nice to have and I dont think this is a blocker for 2.13.