Test use cases

[TheChymera]

Perhaps it would be time to create a small test suite for Gebuilder itself.
But I can't come up with much which can be executed within the 60' time frame of the Travis CI.

Ideas?

[Doeme]

Hm... There are some™ self-tests distributed. At least one could execute the ./tests.sh script

Note: I'm not sure whether the tests are still up to date. Maybe they don't even run anymore.

[TheChymera]

Should the tests be run as root?

At any rate, I get the same thing executing as either root or user:

newbuilder /home/gentoo/gebuilder/gebuilder # ./tests.sh 
cat: '': No such file or directory
Exiting
Cleaning up
Cleaning up after error
Exiting
Cleaning up
Cleaning up after error
Exiting
Cleaning up
Cleaning up after error

[TheChymera]

Given that we have established that the user needs to be root, I'm starting to question in how far we can formulate meaningful integration tests. It seems we wouldn't be able to try and build a stemgentoo system as part of the test suite at all, since it is executed by the portage user.

[Doeme]

Well, the test-suite does not necessarily have to be executed by emerge (although it would be nice).

Having one is useful in any case.

[Doeme]

Completion of test suite for commands:

• initialize
• update
• openstack_image
• docker_image

How would we test the docker_image? Just assume docker is installed and running? Or should we not test it if the testing system does not have docker installed?

[TheChymera]

Installing it can be managed by listing the docker USE flag as a dep of the test FEATURE:

REQUIRED_USE="test? (docker)"

Does it need to be separately started? if so, do the permissions allow us to start it in the test suite?

[Doeme]

Does it need to be separately started?

Yes

if so, do the permissions allow us to start it in the test suite?

Right now: Yes, we still require the tests to be run as root (although not yet explicitly, which we should fix at some point), but in general I'd aim for no. I want to drop the requirement for being root at some point.

[TheChymera]

Well, we will still need root privileges for chroot. Maybe we should just treat being root as a necessary requirement for the tests at this point and use it to start docker as well. Even if we fix the chroot part, we can keep the root requirement for starting docker only, since there's no way around this, unless we somehow do the tests completely in a virtual system.

Also, starting a service as root isn't as big an issue as file manipulations as root.