Please support big-endian architectures

[vpa1977]

Hi,

Would it be possible to consider supporting BIG_ENDIAN architectures such as S390X ?

Running tests for them at the moment results in failures due to the assumptions about the byte order:

Comparing ./tests vs ./tests/wfa.utest.check
[UTest::test.affine] Error
[UTest::test.affine.p0] Error
[UTest::test.affine.p1] Error
[UTest::test.affine.p2] Error
[UTest::test.affine.p3] Error
[UTest::test.affine.p4] Error
[UTest::test.affine.p5] Error
[UTest::test.affine.wfapt0] Error
[UTest::test.affine.wfapt1] Error
[UTest::test.affine2p] Error
[UTest::test.biwfa.affine] Error
[UTest::test.biwfa.affine.p0] Error
[UTest::test.biwfa.affine.p1] Error
[UTest::test.biwfa.affine.p2] Error
[UTest::test.biwfa.affine.p3] Error
[UTest::test.biwfa.affine.p4] Error
[UTest::test.biwfa.affine.p5] Error
[UTest::test.biwfa.affine.wfapt0] Error
[UTest::test.biwfa.affine.wfapt1] Error
[UTest::test.biwfa.affine2p] Error
[UTest::test.biwfa.edit] Error
[UTest::test.biwfa.indel] Error
[UTest::test.biwfa.score.affine] Error
[UTest::test.biwfa.score.affine.p0] Error
[UTest::test.biwfa.score.affine.p1] Error
[UTest::test.biwfa.score.affine.p2] Error
[UTest::test.biwfa.score.affine.p3] Error
[UTest::test.biwfa.score.affine.p4] Error
[UTest::test.biwfa.score.affine.p5] Error
[UTest::test.biwfa.score.affine.wfapt0] Error
[UTest::test.biwfa.score.affine.wfapt1] Error
[UTest::test.biwfa.score.affine2p] Error
[UTest::test.biwfa.score.edit] Error
[UTest::test.biwfa.score.indel] Error
[UTest::test.edit] Error
[UTest::test.indel] Error
[UTest::test.pb.affine] Error
[UTest::test.pb.affine.p0] Error
[UTest::test.pb.affine.p1] Error
[UTest::test.pb.affine.p2] Error
[UTest::test.pb.affine.p3] Error
[UTest::test.pb.affine.p4] Error
[UTest::test.pb.affine.p5] Error
[UTest::test.pb.affine.wfapt0] Error
[UTest::test.pb.affine.wfapt1] Error
[UTest::test.pb.affine2p] Error
[UTest::test.pb.edit] Error
[UTest::test.pb.indel] Error
[UTest::test.score.affine] Error
[UTest::test.score.affine.p0] Error
[UTest::test.score.affine.p1] Error
[UTest::test.score.affine.p2] Error
[UTest::test.score.affine.p3] Error
[UTest::test.score.affine.p4] Error
[UTest::test.score.affine.p5] Error
[UTest::test.score.affine.wfapt0] Error
[UTest::test.score.affine.wfapt1] Error
[UTest::test.score.affine2p] Error
[UTest::test.score.edit] Error
[UTest::test.score.indel] Error

[smarco]

Hi,

If the arch is not LITTLE-ENDIAN, I've tried to implement a patch that makes no assumption and resorts to a much simpler and less efficient implementation. I would appreciate it if you could try it on your BIG-ENDIAN machine (I would like to avoid building a qemu BIG-ENDIAN ARM). Let me know.

[vpa1977]

Hi,

I've re run the test on s390x:

$ ./tests/wfa.utest.sh 
>>> Testing 'test' (--check=correct)
>>> Testing 'test.score' (--wfa-score-only)
>>> Testing 'test.pb' (--wfa-memory-mode=med --check=correct)
>>> Testing 'test.biwfa' (--wfa-memory-mode=ultralow --check=correct)
>>> Testing 'test.biwfa.score' (--wfa-memory-mode=ultralow --wfa-score-only)
>>> Performance Time (s): 
2.22 s	8.44 s
2.76 s	8.59 s
3.80 s	9.37 s
7.81 s	32.84 s
>>> Performance Mem (KB): 
410692	410240
558932	558336
827556	826880
1783444	1782656
> Comparing ./tests vs ./tests/wfa.utest.check
[UTest::test.affine]   	OK
[UTest::test.affine.p0]   	OK
[UTest::test.affine.p1]   	OK
[UTest::test.affine.p2]   	OK
[UTest::test.affine.p3]   	OK
[UTest::test.affine.p4]   	OK
[UTest::test.affine.p5]   	OK
[UTest::test.affine.wfapt0]	OK
[UTest::test.affine.wfapt1]	OK
[UTest::test.affine2p]   	OK
[UTest::test.biwfa.affine]	OK
[UTest::test.biwfa.affine.p0]	OK
[UTest::test.biwfa.affine.p1]	OK
[UTest::test.biwfa.affine.p2]	OK
[UTest::test.biwfa.affine.p3]	OK
[UTest::test.biwfa.affine.p4]	OK
[UTest::test.biwfa.affine.p5]	OK
[UTest::test.biwfa.affine.wfapt0]	OK
[UTest::test.biwfa.affine.wfapt1]	OK
[UTest::test.biwfa.affine2p]	OK
[UTest::test.biwfa.edit]	OK
[UTest::test.biwfa.indel]	OK
[UTest::test.biwfa.score.affine]	OK
[UTest::test.biwfa.score.affine.p0]	OK
[UTest::test.biwfa.score.affine.p1]	OK
[UTest::test.biwfa.score.affine.p2]	OK
[UTest::test.biwfa.score.affine.p3]	OK
[UTest::test.biwfa.score.affine.p4]	OK
[UTest::test.biwfa.score.affine.p5]	OK
[UTest::test.biwfa.score.affine.wfapt0]	OK
[UTest::test.biwfa.score.affine.wfapt1]	OK
[UTest::test.biwfa.score.affine2p]	OK
[UTest::test.biwfa.score.edit]	OK
[UTest::test.biwfa.score.indel]	OK
[UTest::test.edit]	OK
[UTest::test.indel]   	OK
[UTest::test.pb.affine]   	Error
[UTest::test.pb.affine.p0]	Error
[UTest::test.pb.affine.p1]	Error
[UTest::test.pb.affine.p2]	Error
[UTest::test.pb.affine.p3]	Error
[UTest::test.pb.affine.p4]	Error
[UTest::test.pb.affine.p5]	Error
[UTest::test.pb.affine.wfapt0]	Error
[UTest::test.pb.affine.wfapt1]	Error
[UTest::test.pb.affine2p]	Error
[UTest::test.pb.edit]   	ok
[UTest::test.pb.indel]   	ok
[UTest::test.score.affine]	OK
[UTest::test.score.affine.p0]	OK
[UTest::test.score.affine.p1]	OK
[UTest::test.score.affine.p2]	OK
[UTest::test.score.affine.p3]	OK
[UTest::test.score.affine.p4]	OK
[UTest::test.score.affine.p5]	OK
[UTest::test.score.affine.wfapt0]	OK
[UTest::test.score.affine.wfapt1]	OK
[UTest::test.score.affine2p]	OK
[UTest::test.score.edit]	OK
[UTest::test.score.indel]	OK
>>> Correct: ExitStatus() Signal() Correct(     24 (100.00%)      12 (6.23 )
>>>
>>> ERROR
>>>

I have attached the test results to the launchpad bug

[smarco]

Ok, this looks good.

The failing tests are those using --wfa-memory-mode equal to med or low. These modes trigger the WFA-piggyback implementation and are now deprecated in favour of the BiWFA (ultralow). Thus, in the upcoming updates, they will be removed. There is no need to keep on maintaining them, as BiWFA works consistently better in all cases.

Therefore, avoid using med/low memory modes in the BIG-ENDIAN and you are good to go. In any case, I always suggest using the classic WFA or the memory-efficient BiWFA.

I hope this is satisfactory to you.