Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: TestFuncTxnProduceNoBegin flakey, doesn’t handle/ignore/retry-after loading of offsets #2529

Closed
dnwe opened this issue Jul 28, 2023 · 5 comments
Closed

test: TestFuncTxnProduceNoBegin flakey, doesn’t handle/ignore/retry-after loading of offsets #2529

dnwe opened this issue Jul 28, 2023 · 5 comments
Labels
stale Issues and pull requests without any recent activity

Comments

@dnwe
Copy link
Collaborator

dnwe commented Jul 28, 2023

TestFuncTxnProduceNoBegin periodically fails if the leader is still loading offsets:

TestFuncTxnProduceNoBegin is #2 (localhost:29092)
2023-07-28T09:13:52.8501522Z [sarama] 2023/07/28 09:13:52 txnmgr/transition [TestFuncTxnProduceNoBegin] transition from ProducerTxnStateUninitialized to ProducerTxnStateInError|ProducerTxnStateFatalError
2023-07-28T09:13:52.8510148Z     functional_producer_test.go:113: 
2023-07-28T09:13:52.8510873Z         	Error Trace:	github.com/IBM/sarama/functional_producer_test.go:113
2023-07-28T09:13:52.8511456Z         	Error:      	Received unexpected error:
2023-07-28T09:13:52.8513039Z         	            	kafka server: The broker is still loading offsets after a leader change for that offset's topic partition
2023-07-28T09:13:52.8513678Z         	Test:       	TestFuncTxnProduceNoBegin
2023-07-28T09:13:52.8524719Z --- FAIL: TestFuncTxnProduceNoBegin (1.23s)
@dnwe dnwe mentioned this issue Jul 28, 2023
Merged
@hindessm hindessm mentioned this issue Jul 28, 2023
Merged
hindessm added a commit to hindessm/sarama that referenced this issue Jul 28, 2023
@hindessm
fix(fvt): ensure fully-replicated at test start
60c1f11 
Potential workaround for issues like IBM#2529. It would be good to
understand the root cause though.

Signed-off-by: Mark Hindess <mark.hindess@gmail.com>
dnwe pushed a commit that referenced this issue Jul 28, 2023
@hindessm
fix(fvt): ensure fully-replicated at test start (#2531)
c28ecc0 
Potential workaround for issues like #2529. It would be good to
understand the root cause though.

Signed-off-by: Mark Hindess <mark.hindess@gmail.com>
@dnwe
Copy link
Collaborator Author

dnwe commented Aug 4, 2023

Not seen this happen since the associated PR was merged, closing

@dnwe dnwe closed this as completed Aug 4, 2023
@dnwe dnwe reopened this Aug 29, 2023
@dnwe
Copy link
Collaborator Author

dnwe commented Aug 29, 2023

Failed just now in https://github.com/IBM/sarama/actions/runs/6012746470/job/16308708213?pr=2627 (see latest.txt)

--- FAIL: TestFuncTxnProduceNoBegin (1.20s)

[DEBUG] 13:18:54.644896 Initializing new client
[DEBUG] 13:18:54.645123 Successfully initialized new client
[DEBUG] 13:18:54.645912 Connected to broker at 127.0.0.1:29094 (unregistered)
[DEBUG] 13:18:54.652176 Closed connection to broker 127.0.0.1:29094
[DEBUG] 13:18:54.652324 Closing Client
[DEBUG] 13:18:54.652748 Initializing new client
[DEBUG] 13:18:54.652826 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[DEBUG] 13:18:54.652970 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[DEBUG] 13:18:54.653667 Connected to broker at 127.0.0.1:29094 (unregistered)
[DEBUG] 13:18:54.654042 client/metadata fetching metadata for all topics from broker 127.0.0.1:29094
[DEBUG] 13:18:54.661022 client/brokers registered new broker #5 at localhost:29095
[DEBUG] 13:18:54.661100 client/brokers registered new broker #4 at localhost:29094
[DEBUG] 13:18:54.661135 client/brokers registered new broker #1 at localhost:29091
[DEBUG] 13:18:54.661191 client/brokers registered new broker #2 at localhost:29092
[DEBUG] 13:18:54.661222 client/brokers registered new broker #3 at localhost:29093
[DEBUG] 13:18:54.662561 Successfully initialized new client
[DEBUG] 13:18:54.662658 txnmgr/init-producer-id [TestFuncTxnProduceNoBegin] invoking InitProducerId with current producer ID -1 and epoch -1 in order to bump the epoch
[DEBUG] 13:18:54.662725 client/coordinator requesting coordinator for TestFuncTxnProduceNoBegin from 127.0.0.1:29094
[DEBUG] 13:18:54.772046 client/coordinator coordinator for TestFuncTxnProduceNoBegin is not available
[DEBUG] 13:18:54.772281 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[DEBUG] 13:18:54.772478 client/metadata fetching metadata for [__transaction_state] from broker 127.0.0.1:29094
[DEBUG] 13:18:54.773782 Connected to broker at localhost:29094 (registered as #4)
[DEBUG] 13:18:54.796503 client/metadata found some partitions to be leaderless
[DEBUG] 13:18:55.047279 client/metadata retrying after 250ms... (2 attempts remaining)
[DEBUG] 13:18:55.047487 client/metadata fetching metadata for [__transaction_state] from broker 127.0.0.1:29094
[DEBUG] 13:18:55.049811 client/metadata found some partitions to be leaderless
[DEBUG] 13:18:55.300936 client/metadata retrying after 250ms... (1 attempts remaining)
[DEBUG] 13:18:55.301160 client/metadata fetching metadata for [__transaction_state] from broker 127.0.0.1:29094
[DEBUG] 13:18:55.309637 client/metadata found some partitions to be leaderless
[DEBUG] 13:18:55.560038 client/metadata retrying after 250ms... (0 attempts remaining)
[DEBUG] 13:18:55.560104 client/metadata fetching metadata for [__transaction_state] from broker 127.0.0.1:29094
[DEBUG] 13:18:55.574788 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[DEBUG] 13:18:55.575057 client/coordinator retrying after 250ms... (2 attempts remaining)
[DEBUG] 13:18:55.576040 Connected to broker at localhost:29091 (registered as #1)
[DEBUG] 13:18:55.825379 client/coordinator requesting coordinator for TestFuncTxnProduceNoBegin from 127.0.0.1:29094
[DEBUG] 13:18:55.828478 client/coordinator coordinator for TestFuncTxnProduceNoBegin is #1 (localhost:29091)
[DEBUG] 13:18:55.839448 txnmgr/transition [TestFuncTxnProduceNoBegin] transition from ProducerTxnStateUninitialized to ProducerTxnStateInError|ProducerTxnStateFatalError
    functional_producer_test.go:112: 
        	Error Trace:	github.com/IBM/sarama/functional_producer_test.go:112
        	Error:      	Received unexpected error:
        	            	kafka server: The broker is still loading offsets after a leader change for that offset's topic partition
        	Test:       	TestFuncTxnProduceNoBegin

@github-actions GitHub Actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Jan 25, 2024
@dnwe dnwe removed the stale Issues and pull requests without any recent activity label Jan 25, 2024
@github-actions GitHub Actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added stale Issues and pull requests without any recent activity and removed stale Issues and pull requests without any recent activity labels Apr 24, 2024
@github-actions GitHub Actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Jul 23, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale Issues and pull requests without any recent activity
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dnwe