You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Sep 26, 2023. It is now read-only.
When we make changes to CUE that result in unity failures, change in output order is nearly always the reason for the failure. In this majority of such cases, there is no semantic difference.
The way we know there are no semantics differences is by setting CUE_DEBUG_SORT_ARCS=2 then comparing the result of running unity tests against the last known good version (generally speaking this should be current tip) and the CL under test. Note we do not include the current baseline versions for projects in the corpus in this comparison: CUE_DEBUG_SORT_ARCS landed well after many of the current baseline versions.
Doing all this by hand is painful.
This issue tracks adding support for two things:
Support in cmd/unity for such a mode. We already have --skip-base to not run baseline versions, but this is not enough. We need to do something like say "your base is X, run against that with CUE_DEBUG_SORT_ARCS=2 accepting the resulting files as "good", then run against Y with CUE_DEBUG_SORT_ARCS=2 (the CL "version") and ensure the output is the same as for X).
Support for passing the relevant flags, versions etc via cmd/cueckoo runtrybot.
When we make changes to CUE that result in unity failures, change in output order is nearly always the reason for the failure. In this majority of such cases, there is no semantic difference.
The way we know there are no semantics differences is by setting
CUE_DEBUG_SORT_ARCS=2
then comparing the result of running unity tests against the last known good version (generally speaking this should be current tip) and the CL under test. Note we do not include the current baseline versions for projects in the corpus in this comparison:CUE_DEBUG_SORT_ARCS
landed well after many of the current baseline versions.Doing all this by hand is painful.
This issue tracks adding support for two things:
cmd/unity
for such a mode. We already have--skip-base
to not run baseline versions, but this is not enough. We need to do something like say "your base isX
, run against that withCUE_DEBUG_SORT_ARCS=2
accepting the resulting files as "good", then run againstY
withCUE_DEBUG_SORT_ARCS=2
(the CL "version") and ensure the output is the same as forX
).cmd/cueckoo runtrybot
.FYI @mvdan
The text was updated successfully, but these errors were encountered: