Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message #2542

Kerkesni · 2024-09-19T15:48:27Z

The check of forceLegacyListing in the bucket processor was moved to the conductor as the listing type also depends on the MongoDB indexes, which is not something used in S3C.

Lifecycle conductor in 7.x doesn't set the listing type in the bucketProcessor messages. So we now fallback to the forceLegacyListing param inside the config when encountering this case during the transition phase from backbeat 7.x to 8.x/9.x.

Also changed the default value of forceLegacyListing to how it's set in Backbeat 7.x. This is done to avoid any behaviour change in S3C in case the value is not set (which should in theory not be the case in RING10 but just in case)

Issue: BB-550

…ot in the kafka message Lifecycle conductor in 7.x doesn't set the listing type in the bucketProcessor messages. Falling back to the forceLegacyListing param inside the config when encountering this case during the transition phase from backbeat 7.x to 8.x/9.x The check of forceLegacyListing in the bucket processor was moved to the conductor as the listing type also depends on the MongoDB indexes, which is not something used in S3C. Issue: BB-550

This was done to avoid any unwanted behaviour change in S3C. Backbeat 8.x always passes the listing type in the Kafka messages. Issue: BB-550

bert-e · 2024-09-19T15:48:30Z

Hello kerkesni,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2024-09-19T15:49:39Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

codecov · 2024-09-19T15:50:06Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.36%. Comparing base (a1e9895) to head (d8c9131).

Additional details and impacted files

Files with missing lines	Coverage Δ
extensions/lifecycle/LifecycleConfigValidator.js	`83.33% <ø> (ø)`
...ecycle/bucketProcessor/LifecycleBucketProcessor.js	`79.22% <100.00%> (+0.64%)`	⬆️

... and 1 file with indirect coverage changes

Components	Coverage Δ
Bucket Notification	`66.66% <ø> (ø)`
Core Library	`74.00% <ø> (ø)`
Ingestion	`68.70% <ø> (ø)`
Lifecycle	`75.00% <100.00%> (+0.14%)`	⬆️
Oplog Populator	`82.95% <ø> (ø)`
Replication	`57.42% <ø> (ø)`
Bucket Scanner	`85.76% <ø> (ø)`

@@                 Coverage Diff                 @@
##           development/8.6    #2542      +/-   ##
===================================================
+ Coverage            69.33%   69.36%   +0.03%     
===================================================
  Files                  194      194              
  Lines                12791    12791              
===================================================
+ Hits                  8869     8873       +4     
+ Misses                3912     3908       -4     
  Partials                10       10

Flag	Coverage Δ
api:retry	`9.66% <0.00%> (ø)`
api:routes	`9.56% <0.00%> (ø)`
bucket-scanner	`85.76% <ø> (ø)`
ingestion	`12.47% <0.00%> (ø)`
lib	`7.55% <0.00%> (ø)`
lifecycle	`19.43% <100.00%> (+0.04%)`	⬆️
notification	`0.88% <0.00%> (ø)`
replication	`18.97% <0.00%> (ø)`
unit	`43.14% <100.00%> (+0.20%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

tests/unit/lifecycle/LifecycleBucketProcessor.spec.js

Issue: BB-550

Kerkesni · 2024-10-02T13:52:30Z

/approve

bert-e · 2024-10-02T13:52:44Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge improvement/BB-550 into
development/8.6
w/8.7/improvement/BB-550 will be merged into development/8.7

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70
development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

bert-e · 2024-10-02T14:26:33Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/8.6
✔️ development/8.7

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70
development/8.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

bert-e · 2024-10-02T15:02:16Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/8.6
✔️ development/8.7

The following branches have NOT changed:

development/7.10
development/7.4
development/7.70
development/8.5

Please check the status of the associated issue BB-550.

Goodbye kerkesni.

Kerkesni added 2 commits September 19, 2024 17:41

Use the same default value of forceLegacyListing as in S3C

3bc7a49

This was done to avoid any unwanted behaviour change in S3C. Backbeat 8.x always passes the listing type in the Kafka messages. Issue: BB-550

scality deleted a comment from bert-e Sep 19, 2024

Kerkesni changed the title ~~Fallback into chacking the forceLegacyListing parameter when the listing type is not included in the Kafka message~~ Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message Sep 20, 2024

Kerkesni requested review from francoisferrand, KillianG, williamlardier and benzekrimaha September 20, 2024 14:39

williamlardier requested changes Sep 20, 2024

View reviewed changes

tests/unit/lifecycle/LifecycleBucketProcessor.spec.js Outdated Show resolved Hide resolved

add tests

d8c9131

Issue: BB-550

Kerkesni force-pushed the improvement/BB-550 branch from 6ac365e to d8c9131 Compare September 23, 2024 08:50

Kerkesni requested a review from williamlardier September 23, 2024 08:50

williamlardier approved these changes Sep 23, 2024

View reviewed changes

KillianG approved these changes Oct 2, 2024

View reviewed changes

bert-e merged commit 0579c54 into development/8.6 Oct 2, 2024
9 checks passed

bert-e deleted the improvement/BB-550 branch October 2, 2024 15:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message #2542

Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message #2542

Kerkesni commented Sep 19, 2024

bert-e commented Sep 19, 2024

bert-e commented Sep 19, 2024

codecov bot commented Sep 19, 2024 •

edited

Loading

Kerkesni commented Oct 2, 2024

bert-e commented Oct 2, 2024

bert-e commented Oct 2, 2024

bert-e commented Oct 2, 2024

Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message #2542

Fallback into checking the forceLegacyListing parameter when the listing type is not included in the Kafka message #2542

Conversation

Kerkesni commented Sep 19, 2024

bert-e commented Sep 19, 2024

Hello kerkesni,

bert-e commented Sep 19, 2024

Request integration branches

codecov bot commented Sep 19, 2024 • edited Loading

Codecov Report

Kerkesni commented Oct 2, 2024

bert-e commented Oct 2, 2024

Integration data created

bert-e commented Oct 2, 2024

In the queue

bert-e commented Oct 2, 2024

codecov bot commented Sep 19, 2024 •

edited

Loading