Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v24.2.x] [CORE-6992] cloud_storage_clients/abs_client: use item filter in list_objects #24454

Merged
merged 2 commits into from
Dec 5, 2024
Merged

[v24.2.x] [CORE-6992] cloud_storage_clients/abs_client: use item filter in list_objects #24454

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b3fe80
cloud_storage_clients/abs_client: use item filter in list_objects
andrwng Dec 4, 2024
04314cd
rptest: paramterize test_clusters_share_bucket
andrwng Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions src/v/cloud_storage_clients/abs_client.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -851,7 +851,7 @@ ss::future<abs_client::list_bucket_result> abs_client::do_list_objects(
std::optional<ss::sstring> marker,
ss::lowres_clock::duration timeout,
std::optional<char> delimiter,
std::optional<item_filter>) {
std::optional<item_filter> collect_item_if) {
auto header = _requestor.make_list_blobs_request(
name,
_adls_client.has_value(),
Expand Down Expand Up @@ -884,8 +884,9 @@ ss::future<abs_client::list_bucket_result> abs_client::do_list_objects(
co_return co_await ss::do_with(
response_stream->as_input_stream(),
xml_sax_parser{},
[](ss::input_stream<char>& stream, xml_sax_parser& p) mutable {
p.start_parse(std::make_unique<abs_parse_impl>());
[pred = std::move(collect_item_if)](
ss::input_stream<char>& stream, xml_sax_parser& p) mutable {
p.start_parse(std::make_unique<abs_parse_impl>(std::move(pred)));
return ss::do_until(
[&stream] { return stream.eof(); },
[&stream, &p] {
Expand Down
6 changes: 4 additions & 2 deletions tests/rptest/tests/remote_label_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@
from rptest.clients.types import TopicSpec
from rptest.services.cluster import cluster
from rptest.clients.rpk import RpkTool
from rptest.services.redpanda import RedpandaService, SISettings, make_redpanda_service
from rptest.services.redpanda import RedpandaService, SISettings, get_cloud_storage_type, make_redpanda_service
from rptest.services.kgo_verifier_services import KgoVerifierProducer
from rptest.tests.redpanda_test import RedpandaTest
from rptest.tests.read_replica_e2e_test import hwms_are_identical, create_read_replica_topic
from rptest.util import wait_until
from ducktape.mark import matrix
from ducktape.tests.test import TestContext
from rptest.utils.si_utils import BucketView, NT, quiesce_uploads

Expand Down Expand Up @@ -90,7 +91,8 @@ def produce(self, cluster: RedpandaService, topic_name: str,
producer.free()

@cluster(num_nodes=3)
def test_clusters_share_bucket(self) -> None:
@matrix(cloud_storage_type=get_cloud_storage_type())
def test_clusters_share_bucket(self, cloud_storage_type) -> None:
"""
cluster 1 creates topic_a
cluster 2 creates topic_a
Expand Down
Loading