Fix Via-IR bytecode divergence when compiling multiple files that are already included via imports

[r0qs]

Fixes #14541.

[ekpyron]

Ah, ok - you said that bytecode comparison was failing - I thought you only meant in the new "randomized AST IDs" version, but it looks like it's also failing just due to this change?
I guess that means that we do not always add functions to this queue in a deterministic order after all, so we still need to check that.

[cameel]

It's probably the fact that AscendingFunctionIDCompare was removed from the definition of DispatchSet so it now sorts based on pointer addresses.

[nikola-matic]

It's probably the fact that AscendingFunctionIDCompare was removed from the definition of DispatchSet so it now sorts based on pointer addresses.

This will just default to std::less, which should have well defined total ordering, right?

[cameel]

This will just default to std::less, which should have well defined total ordering, right?

Ordering is well defined, but it does not help when addresses are different on each run :)

[nikola-matic]

This will just default to std::less, which should have well defined total ordering, right?

Ordering is well defined, but it does not help when addresses are different on each run :)

Ah, right as always @cameel :)

[r0qs]

It's probably the fact that AscendingFunctionIDCompare was removed from the definition of DispatchSet so it now sorts based on pointer addresses.

Yes, I thought that it should not interfere in code generation, but I guess I was wrong. The function IRGenerationContext::initializeInternalDispatch iterates over the set and calls enqueueFunctionForCodeGeneration, so it affects the order as well, as you pointed out. I completely replaced the set by a queue, and this should fix the problem now.

[cameel]

It would be best if we could add the repro from the issue as a test. How about adding a custom CLI test in the same vein as e.g. ~via_ir_equivalence?

BTW, note that these custom CLI test can be written directly in Python. The current ones are all Bash simply because they used to be a part of cmdlineTests.sh.

[r0qs]

It would be best if we could add the repro from the issue as a test. How about adding a custom CLI test in the same vein as e.g. ~via_ir_equivalence?

BTW, note that these custom CLI test can be written directly in Python. The current ones are all Bash simply because they used to be a part of cmdlineTests.sh.

Indeed. I did it in another branch to ease the review process. Please let me know if that is what you have in mind #14565.

[cameel]

Needs some small tweaks, but overall looks like a good solution.

[cameel]

In terms of correctness this looks fine.

I added some small tweaks. If you're fine with them, please squash and we can merge it.