Testing Meta Issue

[vsoch]

As we get #49 finished up I want to elaborate on needs for testing

• add support for flux option flags #49 handles basic headless tests - e.g., we can provide a headless job, and verify successful output (if deterministic) and if not, container exit codes == 0.
  • However, this doesn't yet handle jobs that require a certain number of tasks or flags. I know --test-size=4 doesn't seem to work. I think the -ompi=openmpi@5 is technically an option so it should. Update: we figured this out, at least for the headless case.
• We also need testing for the web UI views, and this will be harder and possibly require something like selenium. We need to be sure that entrypoint continues to function (and I don't break it) Notably the flux start -> flux mini run does not work for running the web interface. (we are using the restful API as a proxy for this - if the API is running and main page is 200 response, we are good).
• It would be good to have a two-container (service and flux runner) test case. We have some locally but I can't put on GitHub. added with nginx-servic example.
• For within Go we need both unit tests and e2e tests. I have a lot of examples for these, and it mostly will be about deciding when t's priority to do.