Backwards Compatibility Tests

[TimoImhof]

Pull Request: Backward Compatibility Testing

Description

This Pull Request introduces backward compatibility tests to ensure that different branches produce identical model outputs under the same conditions. The primary goal of these tests is to confirm that there are no regressions or inconsistencies in the model's behavior when developing or refactoring features on a development branch based on the main branch.

Changes

• Added a new shell script compare.sh for running backward compatibility tests.
• The script sets a reference output directory that must be located outside the adapters repository to ensure it's available when switching branches.
• Created two Python scripts, create_outputs.py and compare_outputs.py, to generate and compare reference model outputs.
• The script performs the following steps:
  1. Installs the adapters version of the current branch.
  2. Creates reference model outputs using create_outputs.py.
  3. Checks out the target branch to be tested.
  4. Installs the adapters version of the target branch.
  5. Compares the model outputs produced by the target branch with the reference outputs using compare_outputs.py.

How to Use

1. Navigate to the back_comp directory in the terminal
2. Run the following command: sh compare.sh

[calpt]

Thanks for working on this. Could you add a short readme to the folder describing how to run the checks/ configure them?