Add support for using Oxia as the metadata store for Pulsar and BookKeeper #544

yuweisung · 2024-11-06T15:43:09Z

Feature: add oxia template

Motivation

Adding oxia templates in helmchart.

Modifications

Just add the template with the correct variables.
The oxia component is disabled in default values.yaml.

Verifying this change

Make sure that the change passes the CI checks.

yuweisung · 2024-11-06T15:44:17Z

next step: disable zk and modify the init script to integrate pulsar with oxia.

yuweisung · 2024-11-06T15:46:38Z

what's done:

add oxia templates from streamnative.
add oxia components in Values.yaml
modify the oxia templates to fit Values.yaml
verify the template by "helm template test1 ." locally.

…nabled

…bled

…onents

lhotari · 2024-11-21T13:04:09Z

@merlimat what are the recommended default Oxia namespaces for brokers and bookies? Is it simply broker and bookkeeper?

Yes, it should be a good start, with 3 shards each.

What command is needed for creating Oxia namespaces?

The namespaces are automatically created by updating coordinator config-map.

eg: https://github.com/streamnative/oxia/blob/main/deploy/charts/oxia-cluster/templates/coordinator-configmap.yaml#L23-L26

@merlimat I've made changes accordingly.

I can see that Oxia is working, however there are problems.

│ Name: pulsar-ci-oxia-coordinator-status │
│ Namespace: pulsar │
│ Labels: │
│ Annotations: │
│ │
│ Data │
│ ==== │
│ status: │
│ ---- │
│ namespaces: │
│ bookkeeper: │
│ replicationFactor: 1 │
│ shards: │
│ 1: │
│ status: 1 │
│ term: 13 │
│ leader: │
│ public: pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648 │
│ internal: pulsar-ci-oxia-svc.pulsar.svc:6649 │
│ ensemble: │
│ - public: pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648 │
│ internal: pulsar-ci-oxia-svc.pulsar.svc:6649 │
│ removedNodes: [] │
│ int32HashRange: │
│ min: 0 │
│ max: 4294967295 │
│ broker: │
│ replicationFactor: 1 │
│ shards: │
│ 0: │
│ status: 1 │
│ term: 13 │
│ leader: │
│ public: pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648 │
│ internal: pulsar-ci-oxia-svc.pulsar.svc:6649 │
│ ensemble: │
│ - public: pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648 │
│ internal: pulsar-ci-oxia-svc.pulsar.svc:6649 │
│ removedNodes: [] │
│ int32HashRange: │
│ min: 0 │
│ max: 4294967295 │
│ shardIdGenerator: 2 │
│ serverIdx: 0

The problem is that the bookkeeper init job fails. the last error message in logs is:

2024-11-21T12:58:53,552+0000 [oxia-client-1-1] INFO  io.streamnative.oxia.client.shard.ShardManager - Retry creating stream for shard assignments namespace=bookkeeper
2024-11-21T12:58:53,555+0000 [grpc-default-worker-ELG-2-1] WARN  io.streamnative.oxia.client.shard.ShardManager - Failed receiving shard assignments.
io.grpc.StatusRuntimeException: UNIMPLEMENTED: unknown service io.streamnative.oxia.proto.OxiaClient
    at io.grpc.Status.asRuntimeException(Status.java:539) ~[io.grpc-grpc-api-1.56.1.jar:1.56.1]
    at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:491) ~[io.grpc-grpc-stub-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.SerializeReentrantCallsDirectExecutor.execute(SerializeReentrantCallsDirectExecutor.java:49) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closedInternal(ClientCallImpl.java:743) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closed(ClientCallImpl.java:683) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.RetriableStream$4.run(RetriableStream.java:830) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:94) ~[io.grpc-grpc-api-1.56.1.jar:1.56.1]
    at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:126) ~[io.grpc-grpc-api-1.56.1.jar:1.56.1]
    at io.grpc.internal.RetriableStream.safeCloseMasterListener(RetriableStream.java:825) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.RetriableStream.access$2200(RetriableStream.java:55) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.RetriableStream$Sublistener.closed(RetriableStream.java:1018) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.ForwardingClientStreamListener.closed(ForwardingClientStreamListener.java:34) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.InternalSubchannel$CallTracingTransport$1$1.closed(InternalSubchannel.java:691) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.closeListener(AbstractClientStream.java:458) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.access$400(AbstractClientStream.java:221) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState$1.run(AbstractClientStream.java:441) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.deframerClosed(AbstractClientStream.java:278) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.Http2ClientStreamTransportState.deframerClosed(Http2ClientStreamTransportState.java:31) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.MessageDeframer.close(MessageDeframer.java:234) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.MessageDeframer.closeWhenComplete(MessageDeframer.java:192) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractStream$TransportState.closeDeframer(AbstractStream.java:201) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.transportReportStatus(AbstractClientStream.java:444) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.transportReportStatus(AbstractClientStream.java:400) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.AbstractClientStream$TransportState.inboundTrailersReceived(AbstractClientStream.java:383) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.internal.Http2ClientStreamTransportState.transportTrailersReceived(Http2ClientStreamTransportState.java:183) ~[io.grpc-grpc-core-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.grpc.netty.NettyClientStream$TransportState.transportHeadersReceived(NettyClientStream.java:334) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler.onHeadersRead(NettyClientHandler.java:379) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler.access$1200(NettyClientHandler.java:93) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler$FrameListener.onHeadersRead(NettyClientHandler.java:936) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:409) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:337) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onHeadersRead(Http2InboundFrameLogger.java:56) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$2.processFragment(DefaultHttp2FrameReader.java:476) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readHeadersFrame(DefaultHttp2FrameReader.java:484) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:253) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:159) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:173) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:393) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:453) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.grpc-grpc-netty-shaded-1.56.1.jar:1.56.1]
    at java.base/java.lang.Thread.run(Unknown Source) [?:?]

in conf/bookkeeper.conf there's

metadataServiceUri=metadata-store:oxia://pulsar-ci-oxia-svc:6649/bookkeeper

Connection works from the pod:

pulsar-ci-bookie-init-gz88q:/pulsar$ nc -v -z pulsar-ci-oxia-svc 6649
pulsar-ci-oxia-svc (10.244.1.20:6649) open

I wonder how the problem can be fixed? /cc @mattisonchao

lhotari · 2024-11-21T13:06:08Z

Last logs on coordinator:

│ {"level":"info","time":"2024-11-21T12:38:36.001675891Z","component":"shard-controller","namespace":"bookkeeper","shard":1,"term":13,"time":"2024-11-21T12:38:36.001711227Z","message":"Starting leader election"}                                                                                                                                                              │
│ {"level":"info","time":"2024-11-21T12:38:36.033715641Z","component":"shard-controller","entry-id":{"term":"-1", "offset":"-1"},"namespace":"bookkeeper","server-address":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"shard":1,"time":"2024-11-21T12:38:36.033813915Z","message":"Processed newTerm response │
│ {"level":"info","time":"2024-11-21T12:38:36.033840474Z","component":"shard-controller","followers":[],"namespace":"bookkeeper","new-leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"shard":1,"term":13,"time":"2024-11-21T12:38:36.033903832Z","message":"Successfully moved ensemble to a new term"}   │
│ {"level":"info","time":"2024-11-21T12:38:36.04050441Z","component":"shard-controller","leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"namespace":"bookkeeper","shard":1,"term":13,"time":"2024-11-21T12:38:36.040516162Z","message":"Elected new leader"}                                              │
│ {"level":"info","time":"2024-11-21T12:38:36.040532943Z","component":"shard-controller","leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"namespace":"bookkeeper","shard":1,"time":"2024-11-21T12:38:36.040539566Z","message":"Shard is ready"}                                                           │
│ {"level":"info","time":"2024-11-21T12:38:37.967300662Z","component":"shard-controller","namespace":"broker","shard":0,"term":13,"time":"2024-11-21T12:38:37.967319758Z","message":"Starting leader election"}                                                                                                                                                                  │
│ {"level":"info","time":"2024-11-21T12:38:37.99129862Z","component":"shard-controller","entry-id":{"term":"-1", "offset":"-1"},"namespace":"broker","server-address":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"shard":0,"time":"2024-11-21T12:38:37.991327354Z","message":"Processed newTerm response"}    │
│ {"level":"info","time":"2024-11-21T12:38:37.991407553Z","component":"shard-controller","followers":[],"namespace":"broker","new-leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"shard":0,"term":13,"time":"2024-11-21T12:38:37.991421479Z","message":"Successfully moved ensemble to a new term"}       │
│ {"level":"info","time":"2024-11-21T12:38:38.002184498Z","component":"shard-controller","leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"namespace":"broker","shard":0,"term":13,"time":"2024-11-21T12:38:38.002203163Z","message":"Elected new leader"}                                                 │
│ {"level":"info","time":"2024-11-21T12:38:38.002277151Z","component":"shard-controller","leader":{"public":"pulsar-ci-oxia-svc.pulsar.svc.cluster.local:6648","internal":"pulsar-ci-oxia-svc.pulsar.svc:6649"},"namespace":"broker","shard":0,"time":"2024-11-21T12:38:38.002287801Z","message":"Shard is ready"}

lhotari · 2024-11-21T16:00:14Z

@yuweisung If you'd like to debug issues in GitHub Actions, you can ssh into the running build job VM when you open a PR to your own fork. ssh authentication will happen with your public keys at https://github.com/yuweisung.keys .
This is how I've been working on this: lhotari#12 .
Each job will have ssh connection details.

You can run "k9s" after the k8s cluster is running.

… there are in other components

Attempt to fix io.grpc.StatusRuntimeException: UNIMPLEMENTED: unknown service io.streamnative.oxia.proto.OxiaClient Since found this error in the logs: rpc error: code = Code(110) desc = oxia: namespace not found Failed to add client for shards assignments notifications

This reverts commit 39319b5.

… affinity and securityContext

- org.apache.bookkeeper.stream.storage.impl.cluster.ZkClusterInitializer is the only implementation

This reverts commit 7740a2d.

lhotari · 2024-11-22T00:51:31Z

One gap in Pulsar is the lack of a org.apache.pulsar.packages.management.core.PackagesStorage implementation that works with Oxia. The org.apache.pulsar.packages.management.storage.bookkeeper.BookKeeperPackagesStorage implementation uses the DistributedLog library which requires Zookeeper.

error message:

pulsar-ci-toolset-0:/pulsar$ bin/pulsar-admin functions create --tenant pulsar-ci --namespace test --name test-function --inputs "pulsar-ci/test/test_input" --output "pulsar-ci/test/test_output" --parallelism 1 --classname org.apache.pulsar.functions.api.examples.ExclamationFunction --jar /pulsar/examples/api-examples.jar
Cannot invoke "org.apache.distributedlog.api.namespace.Namespace.logExists(String)" because "dlogNamespace" is null

Reason: Cannot invoke "org.apache.distributedlog.api.namespace.Namespace.logExists(String)" because "dlogNamespace" is null

I have disabled function tests for Oxia in CI to make the tests pass. We can merge this PR without functions support for Oxia.

yuweisung added 2 commits November 6, 2024 09:40

change oxia templates

4de4f7e

add oxia templates

8b2eff2

yuweisung changed the title ~~Add-oxia-template~~ feature: add-oxia-template Nov 6, 2024

yuweisung changed the title ~~feature: add-oxia-template~~ feat: add-oxia-template Nov 6, 2024

yuweisung changed the title ~~feat: add-oxia-template~~ feat: add oxia template Nov 6, 2024

change to Apache license.

b9532b8

merlimat approved these changes Nov 8, 2024

View reviewed changes

yuweisung and others added 21 commits November 11, 2024 10:30

add oxia to pulsar-init-job

205eaf6

licenses

782988c

fix comment space

496daa4

add namespace and name

79927af

fix pvc name

d325c12

fix doublequotes

4438a4f

fix more doublequotes

33bfdc1

remove outputfile

5f6fcd3

fix init container

8c09109

fixing init job

3b3280e

fixing init job

6d8ded4

fixed init script

2cbedf5

fix syntax

6baf0c0

add oxia values example

1a2437a

rollback gitignore

c8750a1

fix bookie common tpl

989172f

add metastore timeout

24fa456

Test Oxia in CI

817803e

Make the default for configurationStorePort to be 6648 when oxia is e…

c94546d

…nabled

Add license header and minimize the example

31579ff

Disable wait-zookeeper-ready init containers when zookeeper isn't ena…

1ee80a2

…bled

lhotari added 5 commits November 21, 2024 14:02

Remove invalid rules in oxia-coordinator-role

fc319c9

Add "Limit" postfix to coordinator limits in values.yaml

650cdfe

Use podmonitor to enable metrics for Oxia coordinator and server comp…

50d4986

…onents

Remove redundant servicemonitor templates

682a51f

Enable oxia podmonitors only when oxia is enabled

e57c050

fix oxia tpl

cd4b293

lhotari added 18 commits November 21, 2024 22:48

Use similar solution for matchLabels and labels in Oxia components as…

2ec9cd5

… there are in other components

Validate Oxia templates with kubeconform

4564332

Fix extra args logging

1255018

Wait for Oxia in initialization

3d9abca

Wait for Oxia before bookkeeper initialization

ea2f387

Add waiting for Oxia before broker and proxy get started

c26126b

Use most recent Oxia version

c4ec6bf

Downgrade Oxia to 0.8.0 to match oxia-java 0.4.5

39319b5

Use public port for connecting

7f2909f

Add mandatory configuration-store parameter to initialization

d58676c

Revert "Downgrade Oxia to 0.8.0 to match oxia-java 0.4.5"

8a0bd8f

This reverts commit 39319b5.

Add support for nodeSelector, tolerations, topologySpreadConstraints,…

214a8b0

… affinity and securityContext

Trim trailing whitespace

02c7f98

Specify configurationMetadataStoreUrl

053a4e3

Don't enable state store unless Zookeeper is used

7740a2d

- org.apache.bookkeeper.stream.storage.impl.cluster.ZkClusterInitializer is the only implementation

Revert "Don't enable state store unless Zookeeper is used"

bf3ee75

This reverts commit 7740a2d.

Disable testing functions with Oxia

9c78dfb

lhotari merged commit c6ce11a into apache:master Nov 22, 2024
32 checks passed

lhotari mentioned this pull request Nov 22, 2024

[Bug] Oxia cluster with more than 1 server doesn't become available #551

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for using Oxia as the metadata store for Pulsar and BookKeeper #544

Add support for using Oxia as the metadata store for Pulsar and BookKeeper #544

yuweisung commented Nov 6, 2024

yuweisung commented Nov 6, 2024

yuweisung commented Nov 6, 2024 •

edited

Loading

lhotari commented Nov 21, 2024 •

edited

Loading

lhotari commented Nov 21, 2024

lhotari commented Nov 21, 2024

lhotari commented Nov 22, 2024

Add support for using Oxia as the metadata store for Pulsar and BookKeeper #544

Add support for using Oxia as the metadata store for Pulsar and BookKeeper #544

Conversation

yuweisung commented Nov 6, 2024

Motivation

Modifications

Verifying this change

yuweisung commented Nov 6, 2024

yuweisung commented Nov 6, 2024 • edited Loading

lhotari commented Nov 21, 2024 • edited Loading

lhotari commented Nov 21, 2024

lhotari commented Nov 21, 2024

lhotari commented Nov 22, 2024

yuweisung commented Nov 6, 2024 •

edited

Loading

lhotari commented Nov 21, 2024 •

edited

Loading