Test failures on Linux ARM64

[martin-g]

Bug Report

1. Minimal reproduce step (Required)

Running make test on Ubuntu 20.04.2 64bit reports several failures:

FAIL: aggregate_test.go:55: testSuiteAgg.TestAggregation

aggregate_test.go:410:
    result.Check(testkit.Rows("2584338.6666666665 29.840000178019228 1.1808222222222229 12.666666666666666"))
/home/ubuntu/git/tidb/util/testkit/testkit.go:63:
    res.c.Assert(resBuff.String(), check.Equals, needBuff.String(), res.comment)
... obtained string = "[2584338.6666666665 29.840000178019228 1.1808222222222222 12.666666666666666]\n"
... expected string = "[2584338.6666666665 29.840000178019228 1.1808222222222229 12.666666666666666]\n"
... sql:select variance(b), variance(c), variance(d), variance(e) from t group by a, args:[]

FAIL: integration_test.go:330: testIntegrationSuite2.TestMathBuiltin

integration_test.go:358:
    result.Check(testkit.Rows("0 1"))
/home/ubuntu/git/tidb/util/testkit/testkit.go:63:
    res.c.Assert(resBuff.String(), check.Equals, needBuff.String(), res.comment)
... obtained string = "[0 0.9999999999999998]\n"
... expected string = "[0 1]\n"
... sql:select tan(0.00), tan(PI()/4), args:[]

FAIL: builtin_math_test.go:943: testEvaluatorSuite.TestTan

builtin_math_test.go:973:
    c.Assert(d.GetFloat64(), Equals, t.expected)
... obtained float64 = 0.9999999999999998
... expected float64 = 1

FAIL: lock_test.go:435: testLockSuite.TestLockTTL

lock_test.go:455:
    s.ttlEquals(c, l.TTL, uint64(ttlFactor*2)+uint64(time.Since(start)/time.Millisecond))
lock_test.go:430:
    c.Assert(int(math.Abs(float64(x-y))), LessEqual, 2)
... compare_one int = 9223372036854775807
... compare_two int = 2

2. What did you expect to see? (Required)

All tests to pass

3. What did you see instead (Required)

Test failures

4. What is your TiDB version? (Required)

Current master.

[2021/03/25 09:00:11.872 +00:00] [INFO] [printer.go:33] ["Welcome to TiDB."] ["Release Version"=v4.0.0-beta.2-2449-g78fc47c9a-dirty] [Edition=Community] ["Git Commit Hash"=78fc47c9a256a7e6758453cbdd0bc77c5074398b] ["Git Branch"=master] ["UTC Build Time"="2021-03-25 08:37:04"] [GoVersion=go1.14.4] ["Race Enabled"=false] ["Check Table Before Drop"=true] ["TiKV Min Version"=v3.0.0-60965b006877ca7234adaced7890d7b029ed1306]

[martin-g]

I see the project uses CircleCI (although I'd expect the CircleCI config to be placed at .circleci/config.yml).
Since recently CircleCI provides ARM64 builders - https://github.com/CircleCI-Public/arm-preview-docs.
If you are interested I could work on a Pull Request to add CI job to build and test on ARM64.
TravisCI also makes it easy to build and test on ARM64. One advantage of CircleCI is it is possible to SSH to a VM and debug any issues. TravisCI does not allow this.

[martin-g]

Yet another way to get access to ARM64 is by using Docker+QEMU.
In short:

• (dev-machine) $ docker run -it --rm --privileged multiarch/qemu-user-static --credential yes --persistent yes
• (dev-machine) docker run -it --rm -v /path/to/tidb:/tidb arm64v8/ubuntu:focal bash
• (docker) # # install any prerequisites
• (docker) # cd /tidb
• (docker) # make test

More details about this setup could be found here

[martin-g]

There are two new failing tests:

FAIL: expr_to_pb_test.go:807: testEvaluatorSuite.TestExprOnlyPushDownToFlash

expr_to_pb_test.go:851:
    c.Assert(canPush, Equals, true)
... obtained bool = false
... expected bool = true

and

FAIL: tidb_test.go:1176: tidbTestTopSQLSuite.TestTopSQLCPUProfile

tidb_test.go:1353:
    checkFn(ca.sql, ca.planRegexp)
tidb_test.go:1326:
    // since 1 sql may has many plan, check `len(stats) > 0` instead of `len(stats) == 1`.
    c.Assert(len(stats) > 0, IsTrue, commentf)
... obtained bool = false
... sql: insert into t () values (),(),(),(),(),(),();

[martin-g]

testEvaluatorSuite.TestExprOnlyPushDownToFlash also fails on my x86_64 machine, so it seems it is not something caused by ARM64

[martin-g]

I've just re-tested with v4.0.0-beta.2-6129-g297455d7e:

--- FAIL: TestTan (0.00s)
    builtin_math_test.go:1009: 
                Error Trace:    builtin_math_test.go:1009
                Error:          Not equal: 
                                expected: 1
                                actual  : 0.9999999999999998
                Test:           TestTan

--- FAIL: TestSelectClusterTable (0.34s)
    result.go:50: 
                Error Trace:    result.go:50
                                                        cluster_tables_test.go:241
                Error:          Not equal: 
                                expected: "[:10080 1 root 127.0.0.1 <nil> Query 9223372036  <nil>  0 0 ]\n"
                                actual  : "[:34887 1 root 127.0.0.1 <nil> Query 9223372036  <nil>  0 0 ]\n"
                                
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,2 +1,2 @@
                                -[:10080 1 root 127.0.0.1 <nil> Query 9223372036  <nil>  0 0 ]
                                +[:34887 1 root 127.0.0.1 <nil> Query 9223372036  <nil>  0 0 ]
                                 
                Test:           TestSelectClusterTable
                Messages:       sql:select * from `CLUSTER_PROCESSLIST`, args:[]

--- FAIL: TestStmtSummaryEvictedCountTable (0.33s)
    result.go:50: 
                Error Trace:    result.go:50
                                                        cluster_tables_test.go:344
                Error:          Not equal: 
                                expected: "[0]\n"
                                actual  : "[1]\n"
                                
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,2 +1,2 @@
                                -[0]
                                +[1]
                                 
                Test:           TestStmtSummaryEvictedCountTable
                Messages:       sql:select count(*) from information_schema.cluster_statements_summary_evicted;, args:[]