Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loadbalancer-experimental: add provider for enabling DefaultLoadBalancer #2900

Merged
55 changes: 55 additions & 0 deletions servicetalk-loadbalancer-experimental-provider/README.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
= DefaultLoadBalancer Providers

This package provides providers for enabling the DefaultLoadBalancer via system properties to allow for easy
experimentation that doesn't require a recompilation of the application.

> WARNING: this package is only for experimentation and will be removed in the future.


== Enabling DefaultLoadBalancer via System Properties

=== Dynamically Loading the DefaultHttpLoadBalancerProvider

This package uses the standard providers pattern. To enable the provider you need to both include this package as
part of your application bundle and also include a file in the resources as follows:
```
resources/META-INF/services/io.servicetalk.http.api.HttpProviders$SingleAddressHttpClientBuilderProvider
daschl marked this conversation as resolved.
Show resolved Hide resolved
```

The contents of this must contain the line

```
io.servicetalk.loadbalancer.experimental.DefaultHttpLoadBalancerProvider
```

=== Targeting Clients for Which to Enable DefaultLoadBalancer

The `DefaultHttpLoadBalancerProvider` supports enabling the load balancer either for all clients or only a set of
specific clients. Enabling the load balancer for all clients can be done by setting the following system property:

```
io.servicetalk.loadbalancer.experimental.clientsEnabledFor=all
```

The experimental load balancer can also be enabled for only a subset of clients. This can be done via setting the
system property to a comma separated list:

```
io.servicetalk.loadbalancer.experimental.clientsEnabledFor=service1,service2
```

The specific names will depend on how the client is built. If the client is built using a `HostAndPort`, the names are
only the host component. If the client is built using some other unresolved address form then the string representation
of that is used.

=== Customizing Name Extraction

The provider depends on the service name for selecting which client to use. If you're using a custom naming system
the default implementation may not be able to decode the unresolved address type to the appropriate name. Custom naming
schemes can be supported by extending the `DefaultHttpLoadBalancerProvider` and overriding the `clientNameFromAddress`
method. Then this custom provider can be added to the service load list as described in the section above.

=== All Supported Properties

All system properties contain the prefix "io.servicetalk.loadbalancer.experimental.". A comprehensive list of the
supported properties can be found in the `DefaultLoadBalancerProviderConfig` class for reference.
34 changes: 34 additions & 0 deletions servicetalk-loadbalancer-experimental-provider/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Copyright © 2024 Apple Inc. and the ServiceTalk project authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

apply plugin: "io.servicetalk.servicetalk-gradle-plugin-internal-library"

dependencies {
implementation platform(project(":servicetalk-dependencies"))
testImplementation enforcedPlatform("org.junit:junit-bom:$junit5Version")

api project(":servicetalk-client-api")
api project(":servicetalk-concurrent-api")
Copy link
Member

@idelpivnitskiy idelpivnitskiy May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only api dependency should be :servicetalk-http-api. Others are transitive dependencies through http-api. If you'd like, you can keep an explicit definition of :servicetalk-client-api as well, but there is no dependency on :servicetalk-concurrent-api.


implementation project(":servicetalk-annotations")
implementation project(":servicetalk-loadbalancer")
implementation project(":servicetalk-loadbalancer-experimental")
implementation project(":servicetalk-http-api")
implementation project(":servicetalk-http-netty")
implementation project(":servicetalk-utils-internal")
implementation "com.google.code.findbugs:jsr305"
implementation "org.slf4j:slf4j-api"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/*
* Copyright © 2024 Apple Inc. and the ServiceTalk project authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.servicetalk.loadbalancer.experimental;

import io.servicetalk.client.api.LoadBalancerFactory;
import io.servicetalk.http.api.DelegatingSingleAddressHttpClientBuilder;
import io.servicetalk.http.api.FilterableStreamingHttpLoadBalancedConnection;
import io.servicetalk.http.api.HttpLoadBalancerFactory;
import io.servicetalk.http.api.HttpProviders;
import io.servicetalk.http.api.SingleAddressHttpClientBuilder;
import io.servicetalk.http.netty.DefaultHttpLoadBalancerFactory;
import io.servicetalk.loadbalancer.LoadBalancers;
import io.servicetalk.transport.api.HostAndPort;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static java.util.Objects.requireNonNull;

/**
* A client builder provider that supports enabling the new `DefaultLoadBalancer` in applications via property flags.
* See the packages README.md for more details.
*/
public class DefaultHttpLoadBalancerProvider implements HttpProviders.SingleAddressHttpClientBuilderProvider {

private static final Logger LOGGER = LoggerFactory.getLogger(DefaultHttpLoadBalancerProvider.class);

private final DefaultLoadBalancerProviderConfig config;

public DefaultHttpLoadBalancerProvider() {
this(DefaultLoadBalancerProviderConfig.INSTANCE);
}

// exposed for testing
DefaultHttpLoadBalancerProvider(final DefaultLoadBalancerProviderConfig config) {
this.config = requireNonNull(config, "config");
}

@Override
public final <U, R> SingleAddressHttpClientBuilder<U, R> newBuilder(U address,
SingleAddressHttpClientBuilder<U, R> builder) {
final String serviceName = clientNameFromAddress(address);
if (config.enabledForServiceName(serviceName)) {
try {
HttpLoadBalancerFactory<R> loadBalancerFactory = DefaultHttpLoadBalancerFactory.Builder.<R>from(
defaultLoadBalancer(serviceName)).build();
builder = builder.loadBalancerFactory(loadBalancerFactory);
return new LoadBalancerIgnoringBuilder(builder, serviceName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing <> after class name

} catch (Throwable ex) {
LOGGER.warn("Failed to enabled DefaultLoadBalancer for client to address {}.", address, ex);
}
}
return builder;
}

private <R> LoadBalancerFactory<R, FilterableStreamingHttpLoadBalancedConnection> defaultLoadBalancer(
String serviceName) {
return LoadBalancers.<R, FilterableStreamingHttpLoadBalancedConnection>
builder("experimental-load-balancer")
.loadBalancerObserver(new DefaultLoadBalancerObserver(serviceName))
// set up the new features.
.outlierDetectorConfig(config.outlierDetectorConfig())
.loadBalancingPolicy(config.getLoadBalancingPolicy())
.build();
}

/**
* Extract the service name from the address object.
* Note: this is a protected method to allow overriding for custom address types.
* @param <U> the unresolved type of the address.
* @param address the address from which to extract the service name.
* @return the String representation of the provided address.
*/
protected <U> String clientNameFromAddress(U address) {
String serviceName;
if (address instanceof HostAndPort) {
serviceName = ((HostAndPort) address).hostName();
daschl marked this conversation as resolved.
Show resolved Hide resolved
} else if (address instanceof String) {
serviceName = (String) address;
} else {
LOGGER.warn("Unknown service address type={} was provided, "
+ "default 'toString()' will be used as serviceName", address.getClass());
serviceName = address.toString();
}
return serviceName;
}

private static final class LoadBalancerIgnoringBuilder<U, R>
extends DelegatingSingleAddressHttpClientBuilder<U, R> {

private final String serviceName;

LoadBalancerIgnoringBuilder(final SingleAddressHttpClientBuilder<U, R> delegate, final String serviceName) {
super(delegate);
this.serviceName = serviceName;
}

@Override
public SingleAddressHttpClientBuilder<U, R> loadBalancerFactory(
HttpLoadBalancerFactory<R> loadBalancerFactory) {
LOGGER.info("Ignoring http load balancer factory of type {} for client to {} which has " +
"DefaultLoadBalancer enabled.", loadBalancerFactory.getClass(), serviceName);
return this;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
/*
* Copyright © 2024 Apple Inc. and the ServiceTalk project authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.servicetalk.loadbalancer.experimental;

import io.servicetalk.client.api.NoActiveHostException;
import io.servicetalk.client.api.ServiceDiscovererEvent;
import io.servicetalk.loadbalancer.LoadBalancerObserver;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Collection;
import javax.annotation.Nullable;

import static java.util.Objects.requireNonNull;

final class DefaultLoadBalancerObserver implements LoadBalancerObserver {

private static final Logger LOGGER = LoggerFactory.getLogger(DefaultLoadBalancerObserver.class);

private final String clientName;

DefaultLoadBalancerObserver(final String clientName) {
this.clientName = requireNonNull(clientName, "clientName");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be ideal if the DefaultLoadBalancer could propagate its lbDescription. The main motivation is being able to distinguish one instance of a client to "servicetalk.io" (or any "serviceFoo") from another instance to exact same address. Additionally, it will help to correlate observer events with DefaultLoadBalancer logs.

}

@Override
public HostObserver hostObserver(Object resolvedAddress) {
return new HostObserverImpl(resolvedAddress);
}

@Override
public void onNoHostsAvailable() {
LOGGER.debug("{}- onNoHostsAvailable()", clientName);
}

@Override
public void onServiceDiscoveryEvent(Collection<? extends ServiceDiscovererEvent<?>> events, int oldHostSetSize,
int newHostSetSize) {
LOGGER.debug("{}- onServiceDiscoveryEvent(events: {}, oldHostSetSize: {}, newHostSetSize: {})",
clientName, events, oldHostSetSize, newHostSetSize);
}

@Override
public void onNoActiveHostsAvailable(int hostSetSize, NoActiveHostException exception) {
LOGGER.debug("{}- No active hosts available. Host set size: {}.", clientName, hostSetSize, exception);
bryce-anderson marked this conversation as resolved.
Show resolved Hide resolved
}

private final class HostObserverImpl implements HostObserver {

private final Object resolvedAddress;

HostObserverImpl(final Object resolvedAddress) {
this.resolvedAddress = resolvedAddress;
}

@Override
public void onHostMarkedExpired(int connectionCount) {
LOGGER.debug("{}:{}- onHostMarkedExpired(connectionCount: {})",
clientName, resolvedAddress, connectionCount);
}

@Override
public void onActiveHostRemoved(int connectionCount) {
LOGGER.debug("{}:{}- onActiveHostRemoved(connectionCount: {})",
clientName, resolvedAddress, connectionCount);
}

@Override
public void onExpiredHostRevived(int connectionCount) {
LOGGER.debug("{}:{}- onExpiredHostRevived(connectionCount: {})",
clientName, resolvedAddress, connectionCount);
}

@Override
public void onExpiredHostRemoved(int connectionCount) {
LOGGER.debug("{}:{}- onExpiredHostRemoved(connectionCount: {})",
clientName, resolvedAddress, connectionCount);
}

@Override
public void onHostMarkedUnhealthy(@Nullable Throwable cause) {
LOGGER.debug("{}:{}- onHostMarkedUnhealthy(ex)", clientName, resolvedAddress, cause);
}

@Override
public void onHostRevived() {
LOGGER.debug("{}:{}- onHostRevived()", clientName, resolvedAddress);
}
}
}
Loading
Loading