Persistent term config #3367

chrzaszcz · 2021-10-26T14:45:52Z

Merge the feature branch to master as ejabberd_config is fully replaced with persistent terms in mongoose_config.

Future work regarding ejabberd_modules will be done separately.

Load tests showed no performance degradation. They were performed for two cases:

Maximum load of one-to-one messaging with Inbox for 100k users from 1k domains
Maximum load of MUC Light for 35-36k users from 1k domains

The results are almost identical to the ones for version 5.0.0 and the current master. Initially for MUC Light with 35k users this PR showed slightly worse performance (queues accumulating out of control 2 times in 6 test runs), but after ramping up the user count to 36k and repeating the whole load test tens of times it was the new version this time which was a bit better - there were no cases of such "runaway" queues in 11 runs, while the master branch showed one such incident in 5 test runs. Such phenomenon is normal considered that the system was stressed with extreme load almost to the point of failure. Increasing user count to just 37k caused such issues in every test run.

I also performed an inspection with Erlang Doctor for 10 seconds during peak load and it showed the following:

For master there were about 2M calls to ejabberd_config to get the all_metrics are global option taking 2.5s in total.
For this PR there were about 2M calls to mongoose_config to get the same option taking 2s in total.

There were no other calls to the respective config modules.

This shows that there is no performance degradation and this PR can be merged now.

They could be used to force the removal of global Mnesia options, but there will be no such options anymore.

Also: fix small bugs discovered after making the mocked functions accept only the correct host type.

Use the new helper in muc_SUITE

Replace config with local_config

The config is now stored in persistent terms. Other changes: - Removed checks like 'is_integer' for already validated options - Eliminated the error-prone and ambiguous default value 'undefined' - Removed obsolete code, e.g. handlers for non-existing hooks

- Do not mock the config as it is easier to just set the values

Also: make the config backup/change/restore helpers more consistent.

This module was handling the old versions of ejabberd

This option is set only for host types

Module options should be set only in module opts, stored by gen_mod in the 'ejabberd_modules' ETS table. The only exception was mod_mam_meta, which stored config for mod_mam. It is possible to avoid this completely and just use the module opts. There is no dynamic module reload now, so nothing will get out of sync.

This was used only by ejabberd_auth_jwt to store the secret. This secret can be stored in a persistent term as it remains constant for the whole runtime. Also: do not convert the algorithm from binary to list and back

This key needs to be the same for all nodes for successful server dialback. It was stored in global config before.

Listeners are configured only once at startup and their setup does not change.

There is no need to rewrite config options by system metrics.

Remove the tests relying on the overwritten config options, replacing them with actually checking all the tracking ids.

Do not change the options in mongoose_config

mongoose-im · 2021-10-26T14:57:09Z

small_tests_24 / small_tests / d4ff52f
Reports root / small

internal_mnesia_24 / internal_mnesia / d4ff52f
Reports root/ big
OK: 1586 / Failed: 0 / User-skipped: 297 / Auto-skipped: 0

small_tests_23 / small_tests / d4ff52f
Reports root / small

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / d4ff52f
Reports root/ big
OK: 2699 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

ldap_mnesia_24 / ldap_mnesia / d4ff52f
Reports root/ big
OK: 1485 / Failed: 1 / User-skipped: 400 / Auto-skipped: 0

sm_SUITE:unacknowledged_message_hook:unacknowledged_message_hook_offline

{error,{{badmatch,false},
    [{escalus_session,stream_resumption,2,
              [{file,"/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_session.erl"},
               {line,259}]},
     {escalus_connection,connection_step,2,
               [{file,"/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
                {line,160}]},
     {lists,foldl,3,[{file,"lists.erl"},{line,1267}]},
     {escalus_connection,start,2,
               [{file,"/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
                {line,144}]},
     {sm_SUITE,unacknowledged_message_hook_offline,4,
           [{file,"/home/circleci/app/big_tests/tests/sm_SUITE.erl"},
          {line,830}]},
     {sm_SUITE,unacknowledged_message_hook_common,2,
           [{file,"/home/circleci/app/big_tests/tests/sm_SUITE.erl"},
          {line,882}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1292}]}]}}

Persistent term config #3367

Persistent term config #3367

Conversation

chrzaszcz commented Oct 26, 2021 • edited Loading

mongoose-im commented Oct 26, 2021 • edited Loading

codecov bot commented Oct 26, 2021 • edited Loading

Codecov Report

mongoose-im commented Oct 27, 2021 • edited Loading

mongoose-im commented Oct 27, 2021 • edited Loading

NelsonVides left a comment

Choose a reason for hiding this comment

chrzaszcz commented Oct 26, 2021 •

edited

Loading

mongoose-im commented Oct 26, 2021 •

edited

Loading

codecov bot commented Oct 26, 2021 •

edited

Loading

mongoose-im commented Oct 27, 2021 •

edited

Loading

mongoose-im commented Oct 27, 2021 •

edited

Loading