-m, unexpected behavior? #97

hakavlad · 2018-11-01T11:34:03Z

  -m PERCENT[,KILL_PERCENT] set available memory minimum to PERCENT of total (default 10 %).
                            earlyoom sends SIGTERM once below PERCENT, then SIGKILL once below
                            KILL_PERCENT (default PERCENT/2).

I do not understand how this option works.

earlyoom -m 22[,20]
earlyoom v1.2-2-gb52cc56
mem total: 5875 MiB, sending SIGTERM at 22 %, SIGKILL at 11 %
swap total: 0 MiB, sending SIGTERM at 10 %, SIGKILL at 5 %
Could not lock memory - continuing anyway: Cannot allocate memory
mem avail: 3697 of 5875 MiB (62 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 3693 of 5875 MiB (62 %), swap free: 0 of 0 MiB ( 0 %)

Is it normal?

How to set 22% for SIGTERM and 20% for SIGKILL?
Explain this in more detail, please.

Another example:

earlyoom -m 22[,16] -s 18[,28]
earlyoom v1.2-2-gb52cc56
mem total: 5875 MiB, sending SIGTERM at 22 %, SIGKILL at 11 %
swap total: 0 MiB, sending SIGTERM at 18 %, SIGKILL at 9 %
Could not lock memory - continuing anyway: Cannot allocate memory
mem avail: 3659 of 5875 MiB (62 %), swap free: 0 of 0 MiB ( 0 %)

The text was updated successfully, but these errors were encountered:

hakavlad · 2018-11-01T11:43:24Z

earlyoom -M 220[,160] -S 180[,170]
earlyoom v1.2-2-gb52cc56
fatal: -S: sigterm value in '180[,170]' exceeds limit 0
WTF?

rfjakob · 2018-11-01T11:45:26Z

Remove the "[ ]", for example,

earlyoom -m 22,20
earlyoom v1.1-54-g20e4a01-dirty
mem  total: 7835 MiB, sending SIGTERM at 22 %, SIGKILL at 20 %
swap total:    0 MiB, sending SIGTERM at 10 %, SIGKILL at  5 %

rfjakob · 2018-11-01T11:46:19Z

The "[ ]" just mean that the value is optional. I will add examples to make that clear

hakavlad · 2018-11-01T11:56:09Z

Ok, thank you.

What about

earlyoom -M 220000,160000 -S 180000,170000
earlyoom v1.2-2-gb52cc56
fatal: -S: sigterm value in '180000,170000' exceeds limit 0

?

The new optional KILL_PERCENT is confusing to users, add examples. #97

hakavlad · 2018-11-01T12:01:13Z

earlyoom -S 200000,100000
earlyoom v1.2-2-gb52cc56
fatal: -S: sigterm value in '200000,100000' exceeds limit 0

It is not normal.

rfjakob · 2018-11-01T12:01:34Z

Strange. How much swap do you have? (output of free -k ?)

hakavlad · 2018-11-01T12:02:06Z

swap = 0 MiB

rfjakob · 2018-11-01T12:04:31Z

Well, if you have 0 swap, you cannot set -S to anything above zero.

hakavlad · 2018-11-01T12:05:15Z

with swap works normal

earlyoom -S 200000,100000
earlyoom v1.2-2-gb52cc56
mem total: 5875 MiB, sending SIGTERM at 10 %, SIGKILL at 5 %
swap total: 5875 MiB, sending SIGTERM at 3 %, SIGKILL at 1 %
mem avail: 3621 of 5875 MiB (61 %), swap free: 5875 of 5875 MiB (100 %)
mem avail: 3618 of 5875 MiB (61 %), swap free: 5875 of 5875 MiB (100 %)

hakavlad · 2018-11-01T12:07:36Z

cannot set -S to anything above zero

Maybe It is not well.

rfjakob · 2018-11-01T12:14:39Z

Hmm, earlyoom v1.1 had a better message, but it would still exit with an error:

earlyoom -S 1000
earlyoom v1.1-26-g867d0e8-dirty
-S: the value you passed (1000 kiB) is above total swap (0 kiB)

The error message is different in v1.2 because the parsing of the "X,Y" values is more complicated. I'll try to improve it.

hakavlad · 2018-11-01T12:14:44Z

I can start earlyoom with swap:
earlyoom -M 220000,160000 -S 180000,170000
and all is well. Next, I turn off the swap and all is well again, earlyoom works without swap with options earlyoom -M 220000,160000 -S 180000,170000

But I can not start earlyoom without swap with this options. Why?

Next, If I set earlyoom with -M 220000,160000 -S 180000,170000 In /etc, it will works fine. Next I will disable swap and restart earlyoom: Here the daemon will fall!

rfjakob · 2018-11-01T12:16:52Z

Hmm, should earlyoom ignore the swap settings when there is zero swap? What do you think?

PS: If you restart with less RAM, the daemon will fail as well

hakavlad · 2018-11-01T12:24:45Z

should earlyoom ignore the swap settings when there is zero swap?

Only if SwapTotal = 0.
It seems obvious to me that swap settings should only be applied when SwapTotal>0.

hakavlad · 2018-11-01T13:09:56Z

@rfjakob Do you agree with me?

hakavlad · 2018-11-01T13:47:15Z

How it can works if swap disabled.

Config:

    MemAvailable levels.

mem_min_sigterm = 10 %
mem_min_sigkill = 5 %

    SwapFree levels.

swap_min_sigterm = 200M
swap_min_sigkill = 100M

Output:

sudo nohang
The path to the config: /etc/nohang/nohang.conf
Monitoring started!
TRIGGERED!
  MemAvailable (577 MiB, 9.8 %) < mem_min_sigterm (588 MiB, 10.0 %)
  SwapFree (0 MiB, 0.0 %) < swap_min_sigterm (200 MiB, - %)
  Finding the process with the highest badness
  Victim is python3, pid: 13654, badness: 0, VmRSS: 4141 MiB, VmSwap: 0 MiB
  Sending SIGTERM to the victim; signal received; response time: 18 ms

hakavlad · 2018-11-01T13:57:11Z

earlyoom -S 200000,100000
earlyoom v1.2-2-gb52cc56
fatal: -S: sigterm value in '200000,100000' exceeds limit 0

It is not normal, it should be fixed, IMHO.

hakavlad · 2018-11-01T13:58:34Z

mem avail: 1110 of 5875 MiB (18 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 989 of 5875 MiB (16 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 867 of 5875 MiB (14 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 744 of 5875 MiB (12 %), swap free: 0 of 0 MiB ( 0 %)
Low Memory! At or below sigterm limits (mem: 10 %, swap: 3 %)
mem avail: 635 of 5875 MiB (10 %), swap free: 0 of 0 MiB ( 0 %)
Killing process 'python3' with signal 15, pid: 14069, badness: 583, VmRSS: 3422 MiB
mem avail: 2336 of 5875 MiB (39 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 4059 of 5875 MiB (69 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 4059 of 5875 MiB (69 %), swap free: 0 of 0 MiB ( 0 %)

Earlyoom can works with -S 200000,100000 if swap will be disabled after earlyoom starts.

Why it is forbidden to start with -S 200000,100000 without swap?

hakavlad · 2018-11-01T14:41:36Z

How about this

  -m SIGTERM_PERCENT,SIGKILL_PERCENT set available memory minimum to SIGTERM_PERCENT of total (default 10 %).
                                     Second value is optional and by default SIGKILL_PERCENT = 1/2 SIGTERM_PERCENT
                                     earlyoom sends SIGTERM once below SIGTERM_PERCENT, then SIGKILL once below SIGKILL_PERCENT.
  -s SIGTERM_PERCENT,SIGKILL_PERCENT set free swap minimum to SIGTERM_PERCENT of total (default 10 %),
                                     second value is optional and by default equals 1/2 SIGTERM_PERCENT
  -M SIGTERM_SIZE,SIGKILL_SIZE       set available memory minimum to SIGTERM_SIZE KiB (second value is optional)
  -S SIGTERM_SIZE,SIGKILL_SIZE       set free swap minimum to SIGTERM_SIZE KiB (second value is optional)

hakavlad · 2018-11-01T14:48:48Z

or this

  -m SIGTERM_PERCENT,SIGKILL_PERCENT set available memory minimum to SIGTERM_PERCENT of total (default 10 %).
                                     Second value is optional and by default SIGKILL_PERCENT = 1/2 SIGTERM_PERCENT
                                     earlyoom sends SIGTERM once below SIGTERM_PERCENT, then SIGKILL once below SIGKILL_PERCENT.
  -s SIGTERM_PERCENT,SIGKILL_PERCENT set free swap minimum to SIGTERM_PERCENT of total (default 10 %),
                                     second value is optional and by default equals 1/2 SIGTERM_PERCENT
  -M SIGTERM_KiB,SIGKILL_KiB         set available memory minimum to SIGTERM_KiB KiB (second value is optional)
  -S SIGTERM_KiB,SIGKILL_KiB         set free swap minimum to SIGTERM_KiB KiB (second value is optional)

?

hakavlad · 2018-11-01T15:05:47Z

earlyoom -m 5,9
earlyoom v1.2-2-gb52cc56
fatal: -m: sigkill value exceeds sigterm value in '5,9'

In this case I expect 9% for SIGKILL and ignore SIGTERM, not fatal error.

hakavlad · 2018-11-01T15:19:31Z

earlyoom -m 5,0
earlyoom v1.2-2-gb52cc56
fatal: -m: zero sigkill value in '5,0'

In this case I expect 5% SIGTERM and ignore SIGKILL

hakavlad · 2018-11-01T15:29:28Z

SIGTERM_PERCENT should be ignored if SIGKILL_PERCENT > SIGTERM_PERCENT, not daemon fails.
0 value shouldn't cause err. It should cause only ignoring the signal.

hakavlad · 2018-11-01T15:32:44Z

Also, the documentation does not indicate what values SIGTERM_PERCENT and SIGKILL_SIZE can take (forbidden zero value and KLL can't be higher than TERM size)

#97

You can get a zero SIGKILL percentage by using earlyoom -m 1 so why not allow it on the command line? #97

Saying earlyoom -m 0,15 disables SIGTERM completely. Allow it. #97

rfjakob · 2018-11-01T22:17:47Z

These two work now:

earlyoom -m 5,0

earlyoom -m 5,9

As stated in the man page, the preferred way to disable SIGTERM is to set SIGTERM = SIGKILL. For a better user experience, don't exit with a fatal error is the user sets it to zero or below SIGKILL, but fix the value. #97

Instead of exiting with a fatal error, ignore "-S" if there is zero swap space. #97

rfjakob · 2018-11-03T15:43:26Z

Startup succeeds now with 0 swap even if you pass -S. It cannot calculate the percentages, so it uses defaults.

rfjakob · 2018-11-03T15:45:12Z

About the help message, I like it how it is. I think the square brackets are the usual way to show that a value is optional. Thanks for the report, I think this is done!

hakavlad · 2018-11-04T03:19:08Z

this is done!

Sorry, seems like No!

earlyoom -M 220000,160000 -S 180000,170000
earlyoom v1.2-12-ge600459
warning: -S: total swap is zero, using default percentages
mem total: 5875 MiB, swap total: 0 MiB
Sending SIGTERM when mem <= 3 % and swap <= 10 %,
SIGKILL when mem <= 2 % and swap <= 5 %
Could not lock memory - continuing anyway: Cannot allocate memory
mem avail: 4000 of 5875 MiB (68 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 3996 of 5875 MiB (68 %), swap free: 0 of 0 MiB ( 0 %)
mem avail: 3997 of 5875 MiB (68 %), swap free: 0 of 0 MiB ( 0 %)
...
mem avail: 0 of 5875 MiB ( 0 %), swap free: 3053 of 5875 MiB (51 %)
mem avail: 0 of 5875 MiB ( 0 %), swap free: 2283 of 5875 MiB (38 %)
mem avail: 0 of 5875 MiB ( 0 %), swap free: 1521 of 5875 MiB (25 %)
mem avail: 0 of 5875 MiB ( 0 %), swap free: 804 of 5875 MiB (13 %)
Low Memory! At or below SIGTERM limits (mem: 3 %, swap: 10 %)
mem avail: 0 of 5875 MiB ( 0 %), swap free: 576 of 5875 MiB ( 9 %)
Killing process 'tail' with signal 15, pid: 5355, badness: 760, VmRSS: 4830 MiB
mem avail: 4933 of 5875 MiB (83 %), swap free: 4523 of 5875 MiB (76 %)
mem avail: 4917 of 5875 MiB (83 %), swap free: 4529 of 5875 MiB (77 %)
mem avail: 4914 of 5875 MiB (83 %), swap free: 4531 of 5875 MiB (77 %)

In this case I expect -S 180000,170000, not 10% (after enable swap)
Using 10% is also unexpected behavior. User set exact -S 180000,170000!
After mount swapspace earlyoom should use -S 180000,170000, not 10% that was set at startup.

hakavlad · 2018-11-04T03:27:35Z

Good practice:

earlyoom -M 220000,160000 -S 180000,170000
earlyoom v1.2...
warning: -S: total swap is zero, that's why swap will be ignored now. After mounting swapspace earlyoom will use -S 18000,17000

rfjakob · 2018-11-04T11:10:04Z

Internally, earlyoom always uses percentages.
The -S and -M options are there for user friendlyness, but are converted to percentages at startup. If there is no swap, we cannot calculate the percentages, and we use default values.

I think this is good enough, and i do not want to change the internal design for this.

hakavlad · 2018-11-04T11:18:20Z

You just don't want to fix the bug.

rfjakob · 2018-11-04T11:19:13Z

Yes.

#97

rfjakob · 2018-11-04T12:05:32Z

And now I have even documented it: https://github.com/rfjakob/earlyoom/blob/master/MANPAGE.md#bugs
Thanks

rfjakob added a commit that referenced this issue Nov 1, 2018

MANPAGE: add example for KILL_PERCENT

23228fb

The new optional KILL_PERCENT is confusing to users, add examples. #97

rfjakob added a commit that referenced this issue Nov 1, 2018

tests: add unit tests for parse_term_kill_tuple

79f5d43

#97

rfjakob added a commit that referenced this issue Nov 1, 2018

Allow zero SIGKILL value

b1c0483

You can get a zero SIGKILL percentage by using earlyoom -m 1 so why not allow it on the command line? #97

rfjakob added a commit that referenced this issue Nov 1, 2018

Allow SIGTERM value below SIGKILL

28084c3

Saying earlyoom -m 0,15 disables SIGTERM completely. Allow it. #97

rfjakob added a commit that referenced this issue Nov 3, 2018

Ignore -S if there is no swap

582ac1b

Instead of exiting with a fatal error, ignore "-S" if there is zero swap space. #97

rfjakob closed this as completed Nov 3, 2018

rfjakob added a commit that referenced this issue Nov 4, 2018

MANPAGE: document that -S does not work with zero swap

8ff3800

#97

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

-m, unexpected behavior? #97

-m, unexpected behavior? #97

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018 •

edited

Loading

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018 •

edited

Loading

rfjakob commented Nov 1, 2018

rfjakob commented Nov 3, 2018

rfjakob commented Nov 3, 2018

hakavlad commented Nov 4, 2018 •

edited

Loading

hakavlad commented Nov 4, 2018 •

edited

Loading

rfjakob commented Nov 4, 2018

hakavlad commented Nov 4, 2018

rfjakob commented Nov 4, 2018

rfjakob commented Nov 4, 2018

-m, unexpected behavior? #97

-m, unexpected behavior? #97

Comments

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018 • edited Loading

rfjakob commented Nov 1, 2018

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018 • edited Loading

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018

hakavlad commented Nov 1, 2018 • edited Loading

rfjakob commented Nov 1, 2018

rfjakob commented Nov 3, 2018

rfjakob commented Nov 3, 2018

hakavlad commented Nov 4, 2018 • edited Loading

hakavlad commented Nov 4, 2018 • edited Loading

rfjakob commented Nov 4, 2018

hakavlad commented Nov 4, 2018

rfjakob commented Nov 4, 2018

rfjakob commented Nov 4, 2018

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 1, 2018 •

edited

Loading

hakavlad commented Nov 4, 2018 •

edited

Loading

hakavlad commented Nov 4, 2018 •

edited

Loading