Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/content.zh/docs/deployment/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,10 +457,10 @@ These options are for the network stack that handles the streaming and batch dat

{{< generated/all_taskmanager_network_section >}}

### RPC / Akka
### RPC / Pekko

Flink uses Akka for RPC between components (JobManager/TaskManager/ResourceManager).
Flink does not use Akka for data transport.
Flink uses Pekko for RPC between components (JobManager/TaskManager/ResourceManager).
Flink does not use Pekko for data transport.

{{< generated/akka_configuration >}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Flink 需要多少 *JVM 堆内存*,很大程度上取决于运行的作业数
请参考[常见问题]({{< ref "docs/deployment/memory/mem_trouble" >}}#outofmemoryerror-direct-buffer-memory)。

以下情况可能用到堆外内存:
* Flink 框架依赖(例如 Akka 的网络通信)
* Flink 框架依赖(例如 Pekko 的网络通信)
* 在作业提交时(例如一些特殊的批处理 Source)及 Checkpoint 完成的回调函数中执行的用户代码

<span class="label label-info">提示</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -309,8 +309,8 @@ data:
# 下面几行将公共 libraries 或 connectors 的日志级别保持在 INFO 级别。
# root logger 的配置不会覆盖此处配置。
# 你必须手动修改这里的日志级别。
logger.akka.name = akka
logger.akka.level = INFO
logger.pekko.name = org.apache.pekko
logger.pekko.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
Expand Down Expand Up @@ -380,8 +380,8 @@ data:
# 下面几行将公共 libraries 或 connectors 的日志级别保持在 INFO 级别。
# root logger 的配置不会覆盖此处配置。
# 你必须手动修改这里的日志级别。
logger.akka.name = akka
logger.akka.level = INFO
logger.pekko.name = org.apache.pekko
logger.pekko.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
Expand Down
2 changes: 1 addition & 1 deletion docs/content.zh/docs/deployment/resource-providers/yarn.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ Overwriting this configuration parameter can lead to multiple YARN clusters affe

- **YARN 2.3.0 < version < 2.4.0**. All containers are restarted if the application master fails.
- **YARN 2.4.0 < version < 2.6.0**. TaskManager containers are kept alive across application master failures. This has the advantage that the startup time is faster and that the user does not have to wait for obtaining the container resources again.
- **YARN 2.6.0 <= version**: Sets the attempt failure validity interval to the Flinks' Akka timeout value. The attempt failure validity interval says that an application is only killed after the system has seen the maximum number of application attempts during one interval. This avoids that a long lasting job will deplete it's application attempts.
- **YARN 2.6.0 <= version**: Sets the attempt failure validity interval to the Flinks' Pekko timeout value. The attempt failure validity interval says that an application is only killed after the system has seen the maximum number of application attempts during one interval. This avoids that a long lasting job will deplete it's application attempts.

{{< hint danger >}}
Hadoop YARN 2.4.0 has a major bug (fixed in 2.5.0) preventing container restarts from a restarted Application Master/Job Manager container. See <a href="https://issues.apache.org/jira/browse/FLINK-4142">FLINK-4142</a> for details. We recommend using at least Hadoop 2.5.0 for high availability setups on YARN.</p>
Expand Down
2 changes: 1 addition & 1 deletion docs/content.zh/docs/deployment/security/security-ssl.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ When `security.ssl.internal.enabled` is set to `true`, you can set the following

- `taskmanager.data.ssl.enabled`: Data communication between TaskManagers
- `blob.service.ssl.enabled`: Transport of BLOBs from JobManager to TaskManager
- `akka.ssl.enabled`: Akka-based RPC connections between JobManager / TaskManager / ResourceManager
- `pekko.ssl.enabled`: Pekko-based RPC connections between JobManager / TaskManager / ResourceManager

### Keystores and Truststores

Expand Down
6 changes: 3 additions & 3 deletions docs/content/docs/deployment/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -459,10 +459,10 @@ These options are for the network stack that handles the streaming and batch dat

{{< generated/all_taskmanager_network_section >}}

### RPC / Akka
### RPC / Pekko

Flink uses Akka for RPC between components (JobManager/TaskManager/ResourceManager).
Flink does not use Akka for data transport.
Flink uses Pekko for RPC between components (JobManager/TaskManager/ResourceManager).
Flink does not use Pekko for data transport.

{{< generated/akka_configuration >}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ option. This option can be tuned e.g. if the JobManager process throws ‘OutOfM

There can be the following possible sources of *Off-heap* memory consumption:

* Flink framework dependencies (e.g. Akka network communication)
* Flink framework dependencies (e.g. Pekko network communication)
* User code executed during job submission (e.g. for certain batch sources) or in checkpoint completion callbacks

<span class="label label-info">Note</span> If you have configured the [Total Flink Memory]({{< ref "docs/deployment/memory/mem_setup" >}}#configure-total-memory)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,8 @@ data:
# The following lines keep the log level of common libraries/connectors on
# log level INFO. The root logger does not override this. You have to manually
# change the log levels here.
logger.akka.name = akka
logger.akka.level = INFO
logger.pekko.name = org.apache.pekko
logger.pekko.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
Expand Down Expand Up @@ -369,8 +369,8 @@ data:
# The following lines keep the log level of common libraries/connectors on
# log level INFO. The root logger does not override this. You have to manually
# change the log levels here.
logger.akka.name = akka
logger.akka.level = INFO
logger.pekko.name = org.apache.pekko
logger.pekko.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/deployment/resource-providers/yarn.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ Overwriting this configuration parameter can lead to multiple YARN clusters affe

- **YARN 2.3.0 < version < 2.4.0**. All containers are restarted if the application master fails.
- **YARN 2.4.0 < version < 2.6.0**. TaskManager containers are kept alive across application master failures. This has the advantage that the startup time is faster and that the user does not have to wait for obtaining the container resources again.
- **YARN 2.6.0 <= version**: Sets the attempt failure validity interval to the Flinks' Akka timeout value. The attempt failure validity interval says that an application is only killed after the system has seen the maximum number of application attempts during one interval. This avoids that a long lasting job will deplete it's application attempts.
- **YARN 2.6.0 <= version**: Sets the attempt failure validity interval to the Flinks' Pekko timeout value. The attempt failure validity interval says that an application is only killed after the system has seen the maximum number of application attempts during one interval. This avoids that a long lasting job will deplete it's application attempts.

{{< hint danger >}}
Hadoop YARN 2.4.0 has a major bug (fixed in 2.5.0) preventing container restarts from a restarted Application Master/Job Manager container. See <a href="https://issues.apache.org/jira/browse/FLINK-4142">FLINK-4142</a> for details. We recommend using at least Hadoop 2.5.0 for high availability setups on YARN.</p>
Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/deployment/security/security-ssl.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ When `security.ssl.internal.enabled` is set to `true`, you can set the following

- `taskmanager.data.ssl.enabled`: Data communication between TaskManagers
- `blob.service.ssl.enabled`: Transport of BLOBs from JobManager to TaskManager
- `akka.ssl.enabled`: Akka-based RPC connections between JobManager / TaskManager / ResourceManager
- `pekko.ssl.enabled`: Pekko-based RPC connections between JobManager / TaskManager / ResourceManager

### Keystores and Truststores

Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/ops/state/checkpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ new JobManagerCheckpointStorage(MAX_MEM_STATE_SIZE);
Limitations of the `JobManagerCheckpointStorage`:

- The size of each individual state is by default limited to 5 MB. This value can be increased in the constructor of the `JobManagerCheckpointStorage`.
- Irrespective of the configured maximal state size, the state cannot be larger than the Akka frame size (see [Configuration]({{< ref "docs/deployment/config" >}})).
- Irrespective of the configured maximal state size, the state cannot be larger than the Pekko frame size (see [Configuration]({{< ref "docs/deployment/config" >}})).
- The aggregate state must fit into the JobManager memory.

The JobManagerCheckpointStorage is encouraged for:
Expand Down
46 changes: 23 additions & 23 deletions docs/layouts/shortcodes/generated/akka_configuration.html
Original file line number Diff line number Diff line change
Expand Up @@ -9,121 +9,121 @@
</thead>
<tbody>
<tr>
<td><h5>akka.ask.callstack</h5></td>
<td><h5>pekko.ask.callstack</h5></td>
<td style="word-wrap: break-word;">true</td>
<td>Boolean</td>
<td>If true, call stack for asynchronous asks are captured. That way, when an ask fails (for example times out), you get a proper exception, describing to the original method call and call site. Note that in case of having millions of concurrent RPC calls, this may add to the memory footprint.</td>
</tr>
<tr>
<td><h5>akka.ask.timeout</h5></td>
<td><h5>pekko.ask.timeout</h5></td>
<td style="word-wrap: break-word;">10 s</td>
<td>Duration</td>
<td>Timeout used for all futures and blocking Akka calls. If Flink fails due to timeouts then you should try to increase this value. Timeouts can be caused by slow machines or a congested network. The timeout value requires a time-unit specifier (ms/s/min/h/d).</td>
<td>Timeout used for all futures and blocking Pekko calls. If Flink fails due to timeouts then you should try to increase this value. Timeouts can be caused by slow machines or a congested network. The timeout value requires a time-unit specifier (ms/s/min/h/d).</td>
</tr>
<tr>
<td><h5>akka.client-socket-worker-pool.pool-size-factor</h5></td>
<td><h5>pekko.client-socket-worker-pool.pool-size-factor</h5></td>
<td style="word-wrap: break-word;">1.0</td>
<td>Double</td>
<td>The pool size factor is used to determine thread pool size using the following formula: ceil(available processors * factor). Resulting size is then bounded by the pool-size-min and pool-size-max values.</td>
</tr>
<tr>
<td><h5>akka.client-socket-worker-pool.pool-size-max</h5></td>
<td><h5>pekko.client-socket-worker-pool.pool-size-max</h5></td>
<td style="word-wrap: break-word;">2</td>
<td>Integer</td>
<td>Max number of threads to cap factor-based number to.</td>
</tr>
<tr>
<td><h5>akka.client-socket-worker-pool.pool-size-min</h5></td>
<td><h5>pekko.client-socket-worker-pool.pool-size-min</h5></td>
<td style="word-wrap: break-word;">1</td>
<td>Integer</td>
<td>Min number of threads to cap factor-based number to.</td>
</tr>
<tr>
<td><h5>akka.fork-join-executor.parallelism-factor</h5></td>
<td><h5>pekko.fork-join-executor.parallelism-factor</h5></td>
<td style="word-wrap: break-word;">2.0</td>
<td>Double</td>
<td>The parallelism factor is used to determine thread pool size using the following formula: ceil(available processors * factor). Resulting size is then bounded by the parallelism-min and parallelism-max values.</td>
</tr>
<tr>
<td><h5>akka.fork-join-executor.parallelism-max</h5></td>
<td><h5>pekko.fork-join-executor.parallelism-max</h5></td>
<td style="word-wrap: break-word;">64</td>
<td>Integer</td>
<td>Max number of threads to cap factor-based parallelism number to.</td>
</tr>
<tr>
<td><h5>akka.fork-join-executor.parallelism-min</h5></td>
<td><h5>pekko.fork-join-executor.parallelism-min</h5></td>
<td style="word-wrap: break-word;">8</td>
<td>Integer</td>
<td>Min number of threads to cap factor-based parallelism number to.</td>
</tr>
<tr>
<td><h5>akka.framesize</h5></td>
<td><h5>pekko.framesize</h5></td>
<td style="word-wrap: break-word;">"10485760b"</td>
<td>String</td>
<td>Maximum size of messages which are sent between the JobManager and the TaskManagers. If Flink fails because messages exceed this limit, then you should increase it. The message size requires a size-unit specifier.</td>
</tr>
<tr>
<td><h5>akka.jvm-exit-on-fatal-error</h5></td>
<td><h5>pekko.jvm-exit-on-fatal-error</h5></td>
<td style="word-wrap: break-word;">true</td>
<td>Boolean</td>
<td>Exit JVM on fatal Akka errors.</td>
<td>Exit JVM on fatal Pekko errors.</td>
</tr>
<tr>
<td><h5>akka.log.lifecycle.events</h5></td>
<td><h5>pekko.log.lifecycle.events</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
<td>Turns on the Akka’s remote logging of events. Set this value to 'true' in case of debugging.</td>
<td>Turns on the Pekko’s remote logging of events. Set this value to 'true' in case of debugging.</td>
</tr>
<tr>
<td><h5>akka.lookup.timeout</h5></td>
<td><h5>pekko.lookup.timeout</h5></td>
<td style="word-wrap: break-word;">10 s</td>
<td>Duration</td>
<td>Timeout used for the lookup of the JobManager. The timeout value has to contain a time-unit specifier (ms/s/min/h/d).</td>
</tr>
<tr>
<td><h5>akka.retry-gate-closed-for</h5></td>
<td><h5>pekko.retry-gate-closed-for</h5></td>
<td style="word-wrap: break-word;">50</td>
<td>Long</td>
<td>Milliseconds a gate should be closed for after a remote connection was disconnected.</td>
</tr>
<tr>
<td><h5>akka.server-socket-worker-pool.pool-size-factor</h5></td>
<td><h5>pekko.server-socket-worker-pool.pool-size-factor</h5></td>
<td style="word-wrap: break-word;">1.0</td>
<td>Double</td>
<td>The pool size factor is used to determine thread pool size using the following formula: ceil(available processors * factor). Resulting size is then bounded by the pool-size-min and pool-size-max values.</td>
</tr>
<tr>
<td><h5>akka.server-socket-worker-pool.pool-size-max</h5></td>
<td><h5>pekko.server-socket-worker-pool.pool-size-max</h5></td>
<td style="word-wrap: break-word;">2</td>
<td>Integer</td>
<td>Max number of threads to cap factor-based number to.</td>
</tr>
<tr>
<td><h5>akka.server-socket-worker-pool.pool-size-min</h5></td>
<td><h5>pekko.server-socket-worker-pool.pool-size-min</h5></td>
<td style="word-wrap: break-word;">1</td>
<td>Integer</td>
<td>Min number of threads to cap factor-based number to.</td>
</tr>
<tr>
<td><h5>akka.ssl.enabled</h5></td>
<td><h5>pekko.ssl.enabled</h5></td>
<td style="word-wrap: break-word;">true</td>
<td>Boolean</td>
<td>Turns on SSL for Akka’s remote communication. This is applicable only when the global ssl flag security.ssl.enabled is set to true.</td>
</tr>
<tr>
<td><h5>akka.startup-timeout</h5></td>
<td><h5>pekko.startup-timeout</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>String</td>
<td>Timeout after which the startup of a remote component is considered being failed.</td>
</tr>
<tr>
<td><h5>akka.tcp.timeout</h5></td>
<td><h5>pekko.tcp.timeout</h5></td>
<td style="word-wrap: break-word;">"20 s"</td>
<td>String</td>
<td>Timeout for all outbound connections. If you should experience problems with connecting to a TaskManager due to a slow network, you should increase this value.</td>
</tr>
<tr>
<td><h5>akka.throughput</h5></td>
<td><h5>pekko.throughput</h5></td>
<td style="word-wrap: break-word;">15</td>
<td>Integer</td>
<td>Number of messages that are processed in a batch before returning the thread to the pool. Low values denote a fair scheduling whereas high values can increase the performance at the cost of unfairness.</td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@
<td><h5>taskmanager.slot.timeout</h5></td>
<td style="word-wrap: break-word;">10 s</td>
<td>Duration</td>
<td>Timeout used for identifying inactive slots. The TaskManager will free the slot if it does not become active within the given amount of time. Inactive slots can be caused by an out-dated slot request. If no value is configured, then it will fall back to <code class="highlighter-rouge">akka.ask.timeout</code>.</td>
<td>Timeout used for identifying inactive slots. The TaskManager will free the slot if it does not become active within the given amount of time. Inactive slots can be caused by an out-dated slot request. If no value is configured, then it will fall back to <code class="highlighter-rouge">pekko.ask.timeout</code>.</td>
</tr>
</tbody>
</table>
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<td><h5>metrics.internal.query-service.thread-priority</h5></td>
<td style="word-wrap: break-word;">1</td>
<td>Integer</td>
<td>The thread priority used for Flink's internal metric query service. The thread is created by Akka's thread pool executor. The range of the priority is from 1 (MIN_PRIORITY) to 10 (MAX_PRIORITY). Warning, increasing this value may bring the main Flink components down.</td>
<td>The thread priority used for Flink's internal metric query service. The thread is created by Pekko's thread pool executor. The range of the priority is from 1 (MIN_PRIORITY) to 10 (MAX_PRIORITY). Warning, increasing this value may bring the main Flink components down.</td>
</tr>
<tr>
<td><h5>metrics.job.status.enable</h5></td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@
<td><h5>taskmanager.slot.timeout</h5></td>
<td style="word-wrap: break-word;">10 s</td>
<td>Duration</td>
<td>Timeout used for identifying inactive slots. The TaskManager will free the slot if it does not become active within the given amount of time. Inactive slots can be caused by an out-dated slot request. If no value is configured, then it will fall back to <code class="highlighter-rouge">akka.ask.timeout</code>.</td>
<td>Timeout used for identifying inactive slots. The TaskManager will free the slot if it does not become active within the given amount of time. Inactive slots can be caused by an out-dated slot request. If no value is configured, then it will fall back to <code class="highlighter-rouge">pekko.ask.timeout</code>.</td>
</tr>
</tbody>
</table>
Loading