Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node options from cfg file for production #62468

Merged
merged 82 commits into from
Jul 13, 2020
Merged
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
8dfa276
chore(NA): load NODE_OPTIONS from options files across environments
jbudz Jul 11, 2016
76cde2e
chore(NA): move node.ci.options to config folder
mistic Mar 19, 2020
7b7c98e
Merge branch 'master' into node-options-from-cfg-file
mistic Mar 20, 2020
ff02155
Merge branch 'master' into node-options-from-cfg-file
mistic Mar 30, 2020
09764ca
docs(NA): update docs to explain how to set node options from the cfg…
mistic Mar 30, 2020
fd04141
chore(NA): removed test npm scripts
mistic Mar 30, 2020
7b74c3e
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic Apr 3, 2020
fbed0eb
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic Apr 3, 2020
3a2382b
fix(NA): typo on setup script for CI
mistic Apr 3, 2020
b59eb08
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic Apr 3, 2020
f551fd8
chore(NA): add debug info
mistic Apr 4, 2020
00c9bad
chore(NA): export options on CI
mistic Apr 4, 2020
ed3f6fa
chore(NA): remove debug info
mistic Apr 4, 2020
305a7f6
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 6, 2020
891eda5
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 6, 2020
7b529a2
Merge branch 'master' into node-options-from-cfg-file
mistic Apr 7, 2020
20ac1e2
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 9, 2020
f08f4c8
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 13, 2020
715a054
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 16, 2020
a23f0bc
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 20, 2020
82bbe37
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 22, 2020
66b6d52
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 24, 2020
2e78e64
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 27, 2020
14d5e39
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Apr 29, 2020
21ab8b1
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 5, 2020
ef39f4e
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 7, 2020
e210b24
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 7, 2020
a4efc93
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic May 8, 2020
094d967
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic May 8, 2020
5b81398
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 11, 2020
56b492f
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 11, 2020
bb45be0
Merge branch 'master' into node-options-from-cfg-file
mistic May 14, 2020
747659a
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 18, 2020
ae8496e
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 20, 2020
5452f1b
Merge branch 'master' into node-options-from-cfg-file
mistic May 21, 2020
a4128eb
chore(NA): support for configurable config folder using env var
mistic May 21, 2020
74721cd
chore(NA): add node.options file into docker img
mistic May 21, 2020
90c2ab8
fix(NA): use calculated config dir on node options for ci
mistic May 21, 2020
9c72b46
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 25, 2020
3569eaa
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 27, 2020
f009d47
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic May 29, 2020
ac52efa
Merge branch 'node-options-from-cfg-file' of github.com:mistic/kibana…
mistic May 29, 2020
cb75423
Merge branch 'master' into node-options-from-cfg-file
elasticmachine May 30, 2020
862eb16
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 1, 2020
7bcceac
chore(NA): node bin scripts bootstrap and node_with_options implement…
mistic Jun 3, 2020
5667087
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 3, 2020
c892867
chore(NA): complete node_with_options scripts with bat version
mistic Jun 3, 2020
2b1e3a6
Merge branch 'node-options-from-cfg-file' of github.com:mistic/kibana…
mistic Jun 3, 2020
64500e7
chore(NA): merge and solve conflicts with master
mistic Jun 4, 2020
e8a0386
chore(NA): add bin/node dev script and remove cli for run_with_node_o…
mistic Jun 5, 2020
7100be0
chore(NA): increase default maxBuffer
mistic Jun 5, 2020
65da39f
chore(NA): remove run with options script from package.json
mistic Jun 5, 2020
7d7d184
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic Jun 5, 2020
dbf85ee
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 8, 2020
a4ba6d7
chore(NA): include kbn-node script and underlying usage of it
mistic Jun 8, 2020
838bacb
chore(NA): remove change on eslint
mistic Jun 9, 2020
5145ab5
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 9, 2020
b30e9b6
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 14, 2020
6fa71eb
chore(NA): correct typo on kbn node script comment
mistic Jun 15, 2020
09823d4
chore(NA): correct typo on kbn node script comment
mistic Jun 15, 2020
6a3a2cf
chore(NA): add line to describe each option should be specified in a …
mistic Jun 15, 2020
075c459
chore(NA): remove node options from dev and ci env
mistic Jun 19, 2020
e424d3c
Merge remote-tracking branch 'upstream/master' into node-options-from…
mistic Jun 19, 2020
c4f3310
chore(NA): remove changes from package.json
mistic Jun 19, 2020
ac85b37
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 23, 2020
372f380
chore(NA): fix docker image build
mistic Jun 24, 2020
89460b0
chore(NA): change value for example of --max-old-space-size in the no…
mistic Jun 25, 2020
3757649
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 25, 2020
fc2af99
chore(NA): remove --no-warnings from node.options and force it in the…
mistic Jun 25, 2020
e2f13d6
chore(NA): prevent 'The system cannot find the file' error message
mistic Jun 25, 2020
0f6d1eb
chore(NA): introduce slash when building path for %DIR%
mistic Jun 25, 2020
1a2f1b9
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jun 28, 2020
a3b0f06
chore(NA): read options from file only if it exists
mistic Jun 29, 2020
8f79338
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 1, 2020
d6fbb4b
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 2, 2020
d7630bc
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 3, 2020
00b04dc
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 5, 2020
6027f49
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 6, 2020
72244dd
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 6, 2020
a7d4a46
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 8, 2020
5e0c586
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 9, 2020
27b9dc0
Merge branch 'master' into node-options-from-cfg-file
elasticmachine Jul 13, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ disabledPlugins
webpackstats.json
/config/*
!/config/kibana.yml
!/config/node.options
coverage
selenium
.babel_register_cache.json
Expand Down
6 changes: 6 additions & 0 deletions config/node.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## Node command line options
## See `node --help` and `node --v8-options` for available options
mistic marked this conversation as resolved.
Show resolved Hide resolved
## Please note you should specify one option per line

## max size of old space in megabytes
#--max-old-space-size=4096
4 changes: 2 additions & 2 deletions docs/setup/production.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,9 @@ These can be used to automatically update the list of hosts as a cluster is resi
Kibana has a default maximum memory limit of 1.4 GB, and in most cases, we recommend leaving this unconfigured. In some scenarios, such as large reporting jobs,
it may make sense to tweak limits to meet more specific requirements.

You can modify this limit by setting `--max-old-space-size` in the `NODE_OPTIONS` environment variable. For deb and rpm, packages this is passed in via `/etc/default/kibana` and can be appended to the bottom of the file.
You can modify this limit by setting `--max-old-space-size` in the `node.options` config file that can be found inside `kibana/config` folder or any other configured with the environment variable `KIBANA_PATH_CONF` (for example in debian based system would be `/etc/kibana`).

The option accepts a limit in MB:
--------
NODE_OPTIONS="--max-old-space-size=2048" bin/kibana
--max-old-space-size=2048
--------
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
"uiFramework:documentComponent": "cd packages/kbn-ui-framework && yarn documentComponent",
"kbn:watch": "node scripts/kibana --dev --logging.json=false",
"build:types": "tsc --p tsconfig.types.json",
"docs:acceptApiChanges": "node --max-old-space-size=6144 scripts/check_published_api_changes.js --accept",
"docs:acceptApiChanges": "node --max-old-space-size=6144 scripts/check_published_api_changes.js --accept",
"kbn:bootstrap": "node scripts/register_git_hook",
"spec_to_console": "node scripts/spec_to_console",
"backport-skip-ci": "backport --prDescription \"[skip-ci]\"",
Expand Down
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
mistic marked this conversation as resolved.
Show resolved Hide resolved
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

NODE_OPTIONS="--no-warnings --max-http-header-size=65536 ${NODE_OPTIONS}" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli" ${@}
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi

NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli" ${@}
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-keystore
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

"${NODE}" "${DIR}/src/cli_keystore" "$@"
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi

NODE_OPTIONS="$KBN_NODE_OPTS $NODE_OPTIONS" "${NODE}" "${DIR}/src/cli_keystore" "$@"
17 changes: 16 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-keystore.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -12,6 +12,21 @@ If Not Exist "%NODE%" (
Exit /B 1
)

set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

TITLE Kibana Keystore
"%NODE%" "%DIR%\src\cli_keystore" %*

Expand Down
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-plugin
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

NODE_OPTIONS="--no-warnings ${NODE_OPTIONS}" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli_plugin" "$@"
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi

NODE_OPTIONS="--no-warnings $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli_plugin" "$@"
23 changes: 20 additions & 3 deletions src/dev/build/tasks/bin/scripts/kibana-plugin.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -13,9 +13,26 @@ If Not Exist "%NODE%" (
Exit /B 1
)

TITLE Kibana Server
set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

:: Include pre-defined node option
set "NODE_OPTIONS=--no-warnings %NODE_OPTIONS%"

set "NODE_OPTIONS=--no-warnings %NODE_OPTIONS%" && "%NODE%" "%DIR%\src\cli_plugin" %*
TITLE Kibana Server
"%NODE%" "%DIR%\src\cli_plugin" %*

:finally

Expand Down
24 changes: 22 additions & 2 deletions src/dev/build/tasks/bin/scripts/kibana.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -14,7 +14,27 @@ If Not Exist "%NODE%" (
Exit /B 1
)

set "NODE_OPTIONS=--no-warnings --max-http-header-size=65536 %NODE_OPTIONS%" && "%NODE%" "%DIR%\src\cli" %*
set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

:: Include pre-defined node option
set "NODE_OPTIONS=--no-warnings --max-http-header-size=65536 %NODE_OPTIONS%"

:: This should run independently as the last instruction
:: as we need NODE_OPTIONS previously set to expand
"%NODE%" "%DIR%\src\cli" %*

:finally

Expand Down
1 change: 1 addition & 0 deletions src/dev/build/tasks/copy_source_task.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export const CopySourceTask = {
'typings/**',
'webpackShims/**',
'config/kibana.yml',
'config/node.options',
'tsconfig*.json',
'.i18nrc.json',
'kibana.d.ts',
Expand Down