Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update offloading documentation #2039

Merged
merged 23 commits into from
Mar 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions docs/modules/ROOT/pages/blocking-safe-by-default.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -52,31 +52,31 @@ __offload__ invocation of user code out of the event loop to a different thread.

Although the outcome is simple that no user code should run on the event loop, implementations of this offloading are
often either naive hence sub-optimal or fairly complex. ServiceTalk internally does the heavy lifting to make sure that
it does not offload more than what is required. In other words, it reduces offloading when it can deduce that no user
it does not offload more than what is required. In other words, it reduces offloading when it can determine that no user
code can interact with the event loop on a certain path.

[#execution-strategy]
=== Execution Strategy

An link:{source-root}/servicetalk-transport-api/src/main/java/io/servicetalk/transport/api/ExecutionStrategy.java[Execution Strategy]
has two primary purposes:

. Define which interaction paths require offloading.
. Optionally specify an
link:{source-root}/servicetalk-concurrent-api/src/main/java/io/servicetalk/concurrent/api/Executor.java[Executor]
to use for offloading. In absence of a specified `Executor`, ServiceTalk will use a default `Executor`.

At a general transport layer (no application level protocol), only two offload paths are available:
The primary purpose of an link:{source-root}/servicetalk-transport-api/src/main/java/io/servicetalk/transport/api/ExecutionStrategy.java[Execution Strategy]
is to define which interaction paths of a particular transport or protocol layer require offloading. For
example, at the xref:{page-version}@servicetalk-http-api::blocking-safe-by-default.adoc[HTTP] transport layer, four
offload paths are available:

. Sending data to the transport.
. Receiving data from the transport.
. Handling transport events.
. Closing the transport.

However, different protocols, eg: xref:{page-version}@servicetalk-http-api::blocking-safe-by-default.adoc[HTTP] may
provide more sophisticated offloading paths that can be controlled by a strategy.
A given application, filter or component may indicate that it requires offloading for none, some or all of these
interactions. Protocols other than HTTP will have their own APIs that would otherwise execute user code on an
`EventLoop` thread and define their own `ExecutionStrategy` to control which of those interaction paths APIs are
subject to offloading.

[#influencing-offloading-decisions]
=== Influencing offloading decisions

ServiceTalk does the heavy lifting of determining the optimal offloading strategy. This optimal
ServiceTalk will determine the optimal offloading strategy for handling requests and responses. This optimal
strategy is determined based on different inputs as outlined below:

. __xref:{page-version}@servicetalk::programming-paradigms.adoc[Programming Paradigms].__: A certain programming model
Expand All @@ -103,5 +103,6 @@ the strategy for a client/server, but it comes with additional responsibility fr

=== ServiceTalk developers

Internals of ServiceTalk still discourages blocking code and hence should be avoided while contributing to ServiceTalk.
Unless there is a valid reason to do it, ServiceTalk internals are always non-blocking.
The internal implementation of ServiceTalk generally discourages blocking code and hence introducing blocking should be
avoided while contributing to ServiceTalk. Unless there is a valid reason to do it, ServiceTalk internals are always
non-blocking.
20 changes: 7 additions & 13 deletions docs/modules/ROOT/pages/performance.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -115,22 +115,16 @@ Because ServiceTalk is asynchronous and non-blocking at the core it needs to def
that potentially blocks IO EventLoop threads. Read the chapter on
xref:{page-version}@servicetalk::blocking-safe-by-default.adoc[blocking code safe by default] for more details.

`ServiceTalk` has
xref:{page-version}@servicetalk-concurrent-api::blocking-safe-by-default.adoc#executor-affinity[Executor Affinity] which
ensures that the same `Executor` is used for each `Subscriber` chain on asynchronous operator boundaries. By default
when using the streaming APIs this requires wrapping on the asynchronous operator boundaries and may result in
additional thread hops between ``Executor``s (and even if ``Executor`` happens to be the same). Depending upon the use
case, performance costs maybe relatively high but we have the following compensatory strategies in place:

* ServiceTalk team is investigating ways to reduce the cost of offloading for streaming APIs
* choosing the appropriate xref:{page-version}@servicetalk::performance.adoc#offloading-and-flushing[programming model]
for your use case allows us to be more optimal and reduce offloading
* you can opt-out of (some or all) offloading via
Offloading is expected to add some overhead which can be mitigated as follows:

* Choosing the appropriate xref:{page-version}@servicetalk::performance.adoc#offloading-and-flushing[programming model]
for your use case allows ServiceTalk to be more optimal and reduce offloading
* Opting-out of some or all offloading through use of
xref:{page-version}@servicetalk::performance.adoc#ExecutionStrategy[ExecutionStrategy]

== Tuning options and recommendations

Below sections will offer suggestions that may improve performance depending on your use-case.
The following sections will offer suggestions that may improve performance depending on your use-case.

[#offloading-and-flushing]
=== Programming model (offloading & flushing)
Expand Down Expand Up @@ -242,7 +236,7 @@ HttpServers.forPort(8080)
_`FlushStrategies` and related APIs are advanced, internal, and subject to change._

[#ExecutionStrategy]
==== ExecutionStrategy (offloading)
==== Offloading and `ExecutionStrategy`
link:{source-root}/servicetalk-transport-api/src/main/java/io/servicetalk/transport/api/ExecutionStrategy.java[ExecutionStrategy]
is the core abstraction ServiceTalk uses to drive offloading delivering signals and data from the IO EventLoop threads.
For HTTP there is
Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/pages/programming-paradigms.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ complexity of asynchronous control flow in these cases. This can dramatically lo
compared with most non-blocking I/O frameworks and avoid "application re-write" if scaling/data size characteristics
change over time.

[#blocking-vs-synchronous]
== Blocking vs Synchronous
ServiceTalk APIs may use the term "blocking" in areas where the APIs may be identified as "synchronous". "blocking" in
this context is meant to declare that the API "may block" the calling thread. This is done because there is no general
Expand Down
Binary file not shown.
Loading