There is no record of test result metadata

[kbenne]

BOPTEST Service stores test results as object storage in Minio/s3, however we lack key metadata. Currently the test objects are indexed only by testid. It would be nice to use the s3 tagging feature to record metadata for the user identifier and the test case name. If we did this we could search results by user and test case name. For example we could determine how many times a test has been run by a certain user.

@haraldwalnum

[kbenne]

You have to use the Minio or S3 console. There is no client facing tool to search and filter test results, however the service admin would be able to search results as a result of these new tags. It wouldn't be too hard to add an admin level API to get all test results for a user or user + test case combination.

[haraldwalnum]

Sounds interesting for creating statistics about boptest usage. I guess client id will only be available when users identify themselves, which for standard emulators, are only if they submit to the dashboard.

For competition purposes, it is also interesting, as it looks like a way to avoid participants having to care about their test ids.

I think it is important to think through how to give access to this. Especially if we start having multiple competitions. This also goes for elevated rights in general. At some point it might be necessary with something like namespace specific rights.

[kbenne]

As of right now, this is a behind the scenese feature so there aren't any public access concerns. The dashboard image I showed is something only the server admin would be able to access.

That said, I was thinking this could set the stage for some future enhancements. We could create some new admin level APIs to query test results. This could also be extended in many other dimensions. For example, what if the test cases themselves (within the FMU) contained some form of access settings? We could have settings such as "a user selecting this test case must identify" or "user can only select this test case one time". These types of features would fit within the current architecture and would not require a database.

At this stage I'd just like to push out this change and then see what we want to do with it in future revisions.

@dhblum