Skip to content

[hotfix][docs] Re-introduce note about FLINK_CONF_DIR #15845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rmetzger wants to merge 3 commits into from
Closed

[hotfix][docs] Re-introduce note about FLINK_CONF_DIR #15845

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b95caf8
[hotfix] Make reactive warning less strong, clarifications
rmetzger May 6, 2021
147808d
[hotfix][docs] Re-introduce note about FLINK_CONF_DIR
rmetzger May 6, 2021
1b449c5
address comments
rmetzger May 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/content/docs/deployment/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ The configuration is parsed and evaluated when the Flink processes are started.

The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in `conf/flink-conf.yaml` if you want to manually override the Java runtime to use.

You can specify a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource providers which provide non-session deployments, you can specify per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. On Docker-based deployments, you can use the `FLINK_PROPERTIES` environment variable for passing configuration values.

On session clusters, the provided configuration will only be used for configuring [execution](#execution) parameters, e.g. configuration parameters affecting the job, not the underlying cluster.

Copy link
Contributor

@xintongsong xintongsong May 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also mention FLINK_PROPERTIES as the alternative in docker/standalone-k8s deployments. This, instead of specifying a different configuration directory, appends given configurations to the flink-conf file in the image.
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/standalone/docker/

# Basic Setup

The default configuration supports starting a single-node Flink session cluster without any changes.
Expand Down
6 changes: 3 additions & 3 deletions docs/content/docs/deployment/elastic_scaling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ This page describes options where Flink automatically adjusts the parallelism in

## Reactive Mode

{{< hint danger >}}
{{< hint info >}}
Reactive mode is an MVP ("minimum viable product") feature. The Flink community is actively looking for feedback by users through our mailing lists. Please check the limitations listed on this page.
{{< /hint >}}

Expand Down Expand Up @@ -122,8 +122,8 @@ The [limitations of Adaptive Scheduler](#limitations-1) also apply to Reactive M

## Adaptive Scheduler

{{< hint danger >}}
Using Adaptive Scheduler directly (not through Reactive Mode) is only advised for advanced users.
{{< hint warning >}}
Using Adaptive Scheduler directly (not through Reactive Mode) is only advised for advanced users because slot allocation on a session cluster with multiple jobs is not defined.
{{< /hint >}}

The Adaptive Scheduler can adjust the parallelism of a job based on available slots. It will automatically reduce the parallelism if not enough slots are available to run the job with the originally configured parallelism; be it due to not enough resources being available at the time of submission, or TaskManager outages during the job execution. If new slots become available the job will be scaled up again, up to the configured parallelism.
Expand Down