BLAS tests still fail with larger N

[loveshack]

#333 was closed but never fixed. It still reports "suspect" with 0.2.11.

[xianyi]

How about 0.2.12?

Thank you

Zhang Xianyi
2014年10月25日 上午2:29于 "loveshack" notifications@github.com写道：

#333 #333 was closed but never
fixed. It still reports "suspect" with 0.2.11.

—
Reply to this email directly or view it on GitHub
#458.

[loveshack]

Zhang Xianyi notifications@github.com writes:

How about 0.2.12?

Thank you

It's the same.

[xianyi]

I just run zblat3. Did you suspect zsyr2k?

TESTS OF THE COMPLEX*16       LEVEL 3 BLAS

 THE FOLLOWING PARAMETER VALUES WILL BE USED:
   FOR N                 100
   FOR ALPHA          ( 0.0, 0.0)  ( 1.0, 0.0)  ( 0.7,-0.9)  
   FOR BETA           ( 0.0, 0.0)  ( 1.0, 0.0)  ( 1.3,-1.1)  

 ROUTINES PASS COMPUTATIONAL TESTS IF TEST RATIO IS LESS THAN   16.00

 RELATIVE MACHINE PRECISION IS TAKEN TO BE  2.2D-16

 ZGEMM  WAS NOT TESTED

 ZHEMM  WAS NOT TESTED

 ZSYMM  WAS NOT TESTED

 ZTRMM  WAS NOT TESTED

 ZTRSM  WAS NOT TESTED

 ZHERK  WAS NOT TESTED

 ZSYRK  WAS NOT TESTED

 ZHER2K WAS NOT TESTED

 ZSYR2K PASSED THE TESTS OF ERROR-EXITS

 ZSYR2K COMPLETED THE COMPUTATIONAL TESTS (    36 CALLS)
 ******* BUT WITH MAXIMUM TEST RATIO   17.63 - SUSPECT *******

 END OF TESTS

[xianyi]

When I linked the test with netlib reference BLAS, I got the outputs as following.

 TESTS OF THE COMPLEX*16       LEVEL 3 BLAS

 THE FOLLOWING PARAMETER VALUES WILL BE USED:
   FOR N                 100
   FOR ALPHA          ( 0.7,-0.9)  
   FOR BETA           ( 1.3,-1.1)  

 ROUTINES PASS COMPUTATIONAL TESTS IF TEST RATIO IS LESS THAN    1.00

 RELATIVE MACHINE PRECISION IS TAKEN TO BE  2.2D-16

 ZGEMM  WAS NOT TESTED

 ZHEMM  WAS NOT TESTED

 ZSYMM  WAS NOT TESTED

 ZTRMM  WAS NOT TESTED

 ZTRSM  WAS NOT TESTED

 ZHERK  WAS NOT TESTED

 ZSYRK  WAS NOT TESTED

 ZHER2K WAS NOT TESTED

 ZSYR2K PASSED THE TESTS OF ERROR-EXITS

 ZSYR2K COMPLETED THE COMPUTATIONAL TESTS (     1 CALLS)
 ******* BUT WITH MAXIMUM TEST RATIO    6.29 - SUSPECT *******

 END OF TESTS

[loveshack]

Zhang Xianyi notifications@github.com writes:

I just run zblat3. Did you suspect zsyr2k?

...

ZSYR2K COMPLETED THE COMPUTATIONAL TESTS ( 36 CALLS)
******* BUT WITH MAXIMUM TEST RATIO 17.63 - SUSPECT *******

Yes, we saw similar results, though the test ratio is 17.89 for the
complete set on RHEL6, Sandybridge.

[loveshack]

Zhang Xianyi notifications@github.com writes:

When I linked the test with netlib reference BLAS, I got the outputs as following.

...

ROUTINES PASS COMPUTATIONAL TESTS IF TEST RATIO IS LESS THAN 1.00
...
ZSYR2K COMPLETED THE COMPUTATIONAL TESTS ( 1 CALLS)
******* BUT WITH MAXIMUM TEST RATIO 6.29 - SUSPECT *******

Is the different test ratio intentional? Reference, atlas, mkl, and
acml all passed with the ratio of 16 from the original test.

I don't know how significant this is, and I can't ask the BLAS expert
who pointed it out about it until next week, but he thought it was
a problem.

[xianyi]

2014-11-04 19:45 GMT+08:00 Dave Love notifications@github.com:

Zhang Xianyi notifications@github.com writes:

When I linked the test with netlib reference BLAS, I got the outputs as
following.

...

ROUTINES PASS COMPUTATIONAL TESTS IF TEST RATIO IS LESS THAN 1.00
...
ZSYR2K COMPLETED THE COMPUTATIONAL TESTS ( 1 CALLS)
******* BUT WITH MAXIMUM TEST RATIO 6.29 - SUSPECT *******

Is the different test ratio intentional? Reference, atlas, mkl, and
acml all passed with the ratio of 16 from the original test.

I just want to show the test ratio by setting 1.0.

I don't know how significant this is, and I can't ask the BLAS expert
who pointed it out about it until next week, but he thought it was
a problem.

—
Reply to this email directly or view it on GitHub
#458 (comment).

[wernsaar]

Included in OpenBLAS are 3 BLAS tests.

1. in directory test
2. in directory ctest
3. in directory lapack-netlib/BLAS

In all these tests, you cannot run tests for values of N, that are > 65

Which blas test do you use?

[loveshack]

wernsaar notifications@github.com writes:

In all these tests, you cannot run tests for values of N, that are > 65

Which blas test do you use?

The netlib-blas ones, with the nmax parameter increased appropriately.
I sent xianyi a tarball at some stage, but it's a trivial edit of the
code and input.

It's pretty useless not being able to attach stuff to github issues
through the web interface, and I assume these MIME parts for the zblat
case won't make it through either, but I can mail again if necessary.

[wernsaar]

Hi,

you can send attachments directly to
my mail address wernsaar@googlemail.com

Best regards

Werner

On 12/22/2014 12:53 PM, Dave Love wrote:

wernsaar notifications@github.com writes:

In all these tests, you cannot run tests for values of N, that are > 65

Which blas test do you use?

The netlib-blas ones, with the nmax parameter increased appropriately.
I sent xianyi a tarball at some stage, but it's a trivial edit of the
code and input.

It's pretty useless not being able to attach stuff to github issues
through the web interface, and I assume these MIME parts for the zblat
case won't make it through either, but I can mail again if necessary.

—
Reply to this email directly or view it on GitHub
#458 (comment).

[wernsaar]

I can now reproduce this issue and will try to fix it

[wernsaar]

An optimized zgemm kernel for sandybridge will be available soon.

[loveshack]

This is fixed in 0.2.14 for zblat3 but the equivalent for dblat3 and sblat3 still reports a suspect result with n=100.

[loveshack]

Closed accidentally

[loveshack]

With version 0.2.16, DBLAT3 passes as well as ZBLAT3, though SBLAT3 still fails in those conditions (n=100 on sandybridge, single-threaded).

[martin-frbg]

Revisiting this as I was running tests in the context of fixing #601, single-threaded SANDYBRIDGE target (on Kaby Lake hardware) with current "develop":
CBLAT3 passes even at N=265 (with what appears to be an isolated "suspect" result for CSYR2K at N=150),
SBLAT3 and DBLAT3 likewise (with a suspect SSYR2K/DSYR2K at N=100 and N=150, again bracketing values like N=99,101 and N=149,151 pass)
ZBLAT3 test was run only up to N=205 due to memory constraints but behaves similar to CBLAT3 (mostly OK with seemingly isolated ZSYRK2 hickups at 150, and again at 200 - at the latter datum ZHER2K is flagged as suspect as well).

OK to close this, or should the intermittent "suspect" results for SYR2K be cause for concern ?

[loveshack]

I asked my former boss, on whose behalf I reported the issue. He said he'd have a look when he had time, but I haven't heard back yet; I'll let you know when I do. (I hadn't realized there had been an improvement since I checked a version or two ago -- good!)