Timeout setup in data mover s3 client #2521

benzekrimaha · 2024-07-19T13:04:05Z

Linked issue : https://scality.atlassian.net/browse/BB-594

bert-e · 2024-07-19T13:04:09Z

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2024-07-19T13:04:14Z

Incorrect fix version

The Fix Version/s in issue BB-594 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

8.6.44
8.7.0

Please check the Fix Version/s of BB-594, or the target
branch of this pull request.

The S3 client didn’t have any configured timeout which made the request hang forever when a cloudserver worker crashesd thus blocking the data-mover workers. Linked issue : https://scality.atlassian.net/browse/BB-594

bert-e · 2024-07-19T13:06:55Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

codecov · 2024-07-19T13:08:07Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.99%. Comparing base (e19a1e5) to head (31bf72f).

Additional details and impacted files

Files	Coverage Δ
extensions/replication/tasks/CopyLocationTask.js	`62.23% <100.00%> (ø)`

... and 3 files with indirect coverage changes

Components	Coverage Δ
Bucket Notification	`66.11% <ø> (ø)`
Core Library	`74.19% <ø> (+0.18%)`	⬆️
Ingestion	`69.24% <ø> (ø)`
Lifecycle	`74.54% <ø> (ø)`
Oplog Populator	`75.90% <ø> (ø)`
Replication	`57.22% <100.00%> (ø)`
Bucket Scanner	`85.76% <ø> (ø)`

@@                 Coverage Diff                 @@
##           development/8.6    #2521      +/-   ##
===================================================
+ Coverage            68.91%   68.99%   +0.07%     
===================================================
  Files                  192      192              
  Lines                12681    12681              
===================================================
+ Hits                  8739     8749      +10     
+ Misses                3932     3922      -10     
  Partials                10       10

Flag	Coverage Δ
api:retry	`9.75% <0.00%> (ø)`
api:routes	`9.65% <0.00%> (ø)`
bucket-scanner	`85.76% <ø> (ø)`
ingestion	`12.63% <0.00%> (-0.01%)`	⬇️
lib	`7.62% <0.00%> (ø)`
lifecycle	`19.56% <0.00%> (ø)`
notification	`0.89% <0.00%> (ø)`
replication	`19.20% <100.00%> (+0.08%)`	⬆️
unit	`42.12% <100.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

benzekrimaha · 2024-07-25T07:33:57Z

/create_integration_branches

bert-e · 2024-07-25T07:34:11Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge improvement/BB-594-Data-mover-S3-clients-timeout into
development/8.6
w/8.7/improvement/BB-594-Data-mover-S3-clients-timeout will be merged into development/8.7

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70
development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

bert-e · 2024-07-25T07:34:12Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

benzekrimaha · 2024-07-25T07:34:47Z

@bert-e create_pull_requests

bert-e · 2024-07-25T07:34:58Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge improvement/BB-594-Data-mover-S3-clients-timeout into
development/8.6
w/8.7/improvement/BB-594-Data-mover-S3-clients-timeout will be merged into development/8.7 (pull request INTEGRATION [PR#2521 > development/8.7] Timeout setup in data mover s3 client #2523)

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70
development/8.5

Follow integration pull requests if you would like to be notified of
build statuses by email.

The following options are set: create_pull_requests, create_integration_branches

bert-e · 2024-07-25T07:34:59Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_pull_requests, create_integration_branches

benzekrimaha · 2024-07-25T07:37:27Z

/approve

bert-e · 2024-07-25T08:07:38Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/8.6
✔️ development/8.7

The following branches have NOT changed:

development/7.10
development/7.4
development/7.70
development/8.5

Please check the status of the associated issue BB-594.

Goodbye benzekrimaha.

The following options are set: approve, create_pull_requests, create_integration_branches

Timeout setup in data moved s3 client

31bf72f

The S3 client didn’t have any configured timeout which made the request hang forever when a cloudserver worker crashesd thus blocking the data-mover workers. Linked issue : https://scality.atlassian.net/browse/BB-594

benzekrimaha force-pushed the improvement/BB-594-Data-mover-S3-clients-timeout branch from 5e542b8 to 31bf72f Compare July 19, 2024 13:06

benzekrimaha changed the title ~~Timeout setup in data moved s3 client~~ Timeout setup in data mover s3 client Jul 19, 2024

benzekrimaha marked this pull request as ready for review July 19, 2024 13:06

Kerkesni approved these changes Jul 19, 2024

View reviewed changes

williamlardier approved these changes Jul 19, 2024

View reviewed changes

bert-e mentioned this pull request Jul 25, 2024

INTEGRATION [PR#2521 > development/8.7] Timeout setup in data mover s3 client #2523

Merged

bert-e merged commit 31bf72f into development/8.6 Jul 25, 2024
9 checks passed

bert-e deleted the improvement/BB-594-Data-mover-S3-clients-timeout branch July 25, 2024 08:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Timeout setup in data mover s3 client #2521

Timeout setup in data mover s3 client #2521

benzekrimaha commented Jul 19, 2024 •

edited

Loading

bert-e commented Jul 19, 2024

bert-e commented Jul 19, 2024

bert-e commented Jul 19, 2024

codecov bot commented Jul 19, 2024 •

edited

Loading

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

bert-e commented Jul 25, 2024

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

bert-e commented Jul 25, 2024

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

Timeout setup in data mover s3 client #2521

Timeout setup in data mover s3 client #2521

Conversation

benzekrimaha commented Jul 19, 2024 • edited Loading

bert-e commented Jul 19, 2024

Hello benzekrimaha,

bert-e commented Jul 19, 2024

Incorrect fix version

bert-e commented Jul 19, 2024

Request integration branches

codecov bot commented Jul 19, 2024 • edited Loading

Codecov Report

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

Integration data created

bert-e commented Jul 25, 2024

Waiting for approval

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

Integration data created

bert-e commented Jul 25, 2024

Waiting for approval

benzekrimaha commented Jul 25, 2024

bert-e commented Jul 25, 2024

benzekrimaha commented Jul 19, 2024 •

edited

Loading

codecov bot commented Jul 19, 2024 •

edited

Loading