Skip to content
This repository has been archived by the owner on Sep 26, 2023. It is now read-only.

Allow setting CUE_DEBUG_SORT_ARCS=2 for a unity run #108

Open
myitcv opened this issue Aug 26, 2022 · 0 comments
Open

Allow setting CUE_DEBUG_SORT_ARCS=2 for a unity run #108

myitcv opened this issue Aug 26, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@myitcv
Copy link
Contributor

myitcv commented Aug 26, 2022

When we make changes to CUE that result in unity failures, change in output order is nearly always the reason for the failure. In this majority of such cases, there is no semantic difference.

The way we know there are no semantics differences is by setting CUE_DEBUG_SORT_ARCS=2 then comparing the result of running unity tests against the last known good version (generally speaking this should be current tip) and the CL under test. Note we do not include the current baseline versions for projects in the corpus in this comparison: CUE_DEBUG_SORT_ARCS landed well after many of the current baseline versions.

Doing all this by hand is painful.

This issue tracks adding support for two things:

  • Support in cmd/unity for such a mode. We already have --skip-base to not run baseline versions, but this is not enough. We need to do something like say "your base is X, run against that with CUE_DEBUG_SORT_ARCS=2 accepting the resulting files as "good", then run against Y with CUE_DEBUG_SORT_ARCS=2 (the CL "version") and ensure the output is the same as for X).
  • Support for passing the relevant flags, versions etc via cmd/cueckoo runtrybot.

FYI @mvdan

@myitcv myitcv added the enhancement New feature or request label Aug 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Development

No branches or pull requests

1 participant