go-ethereum eth-tests are hanging

[evgeniy-scherbina]

Hi, I'm trying to setup Hive Framework to run integration tests on my go-ethereum fork, and I faced up with an issue.
I'm running tests in a such way:

hive \
  --sim devp2p \
  --sim.limit eth \
  --client go-ethereum \
  --client-file $GITHUB_WORKSPACE/.github/workflows/hive-clients.yml

hive-clients.yml file

- client: go-ethereum
  dockerfile: git

Some of the tests are hanging, for ex in this group all broadcast tests are hanging

func (s *Suite) EthTests() []utesting.Test {
	return []utesting.Test{
		// status
		{Name: "TestStatus", Fn: s.TestStatus},
		// get block headers
		{Name: "TestGetBlockHeaders", Fn: s.TestGetBlockHeaders},
		{Name: "TestSimultaneousRequests", Fn: s.TestSimultaneousRequests},
		{Name: "TestSameRequestID", Fn: s.TestSameRequestID},
		{Name: "TestZeroRequestID", Fn: s.TestZeroRequestID},
		// get block bodies
		{Name: "TestGetBlockBodies", Fn: s.TestGetBlockBodies},
		// broadcast
		{Name: "TestBroadcast", Fn: s.TestBroadcast},
		{Name: "TestLargeAnnounce", Fn: s.TestLargeAnnounce},
		{Name: "TestOldAnnounce", Fn: s.TestOldAnnounce},
		{Name: "TestBlockHashAnnounce", Fn: s.TestBlockHashAnnounce},
		// malicious handshakes + status
		{Name: "TestMaliciousHandshake", Fn: s.TestMaliciousHandshake},
		{Name: "TestMaliciousStatus", Fn: s.TestMaliciousStatus},
		// test transactions
		{Name: "TestTransaction", Fn: s.TestTransaction},
		{Name: "TestMaliciousTx", Fn: s.TestMaliciousTx},
		{Name: "TestLargeTxRequest", Fn: s.TestLargeTxRequest},
		{Name: "TestNewPooledTxs", Fn: s.TestNewPooledTxs},
	}
}

I started to debug, and realized it happens here:
https://github.com/ethereum/go-ethereum/blob/master/cmd/devp2p/internal/ethtest/helpers.go#L298-L331

in waitForBlockImport function https://github.com/ethereum/go-ethereum/blob/master/cmd/devp2p/internal/ethtest/helpers.go#L325

More specifically in this loop: https://github.com/ethereum/go-ethereum/blob/master/cmd/devp2p/internal/ethtest/helpers.go#L398

I wanted to ask if it's a known issue? Maybe you can point me to the repo where it's properly setup and works without issues?

Follow-up:
Another example is running such command:

hive --sim devp2p --sim.limit discv5 --client go-ethereum

Returns: 6 tests failed
So is it expected to fail? or I have invalid setup?

[fjl]

We are fixing it right now! See #942

[evgeniy-scherbina]

@fjl got it, thanks for answer!

and follow-up question:
I also tried to run other test suites:

• devp2p/discv5
• devp2p/snap
• ethereum/engine
  for all of them some of the test failed, for ex:
• ethereum/engine returned 9 tests failed

is it expected?

[fjl]

Some tests fail sometimes. It's kind of normal. We would love to have all clients pass all tests, but it's not a reality yet.

[evgeniy-scherbina]

got it, last question before closing discussion:

so it's expected even for ethereum/go-ethereum master branch to fail some tests time-to-time?

[fjl]

Yes, but we do want to fix them too. So it's not like they should always be broken, rather, it will sometimes take a while to get fixed.

[evgeniy-scherbina]

got it, thanks for responses!