Listener configuration rework #3628
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
3c7ea03
to
2aa1d84
Compare
Codecov Report
@@ Coverage Diff @@
## master #3628 +/- ##
==========================================
+ Coverage 81.00% 81.06% +0.05%
==========================================
Files 427 427
Lines 32085 31963 -122
==========================================
- Hits 25991 25911 -80
+ Misses 6094 6052 -42
Continue to review full report at Codecov.
|
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
2aa1d84
to
e532831
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
e532831
to
4b86eb0
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
4b86eb0
to
d6e8f3d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
d6e8f3d
to
e8a9a98
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
e8a9a98
to
f11efa4
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
2 times, most recently
from
a017e78
to
6b90f54
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
listener-config-rework
branch
3 times, most recently
from
088506b
to
d4e2df8
Compare
- Use maps with defaults for listener-sepcific opts. - Avoid changing keys for options, e.g. for 'handlers' and 'tcp'. - Accept only TCP in proto - all listeners support only TCP The 'proto' opt could be removed, but I didn't want to do it all at once. - Make 'password' required for service - it was documented as such and the server wouldn't start without it. HTTP handlers and TLS options will be amended separately.
Separate per-module logic from per-listener logic The module is now a behaviour with the following implementations: - ejabberd_c2s - ejabberd_s2s_in - ejabberd_service - ejabberd_cowboy
This way the code is more organized
Define a type that enforces mandatory options. - Make the options with defaults mandatory - SSL opt filtering is now unnecessary
Using maps allowed for simpler option processing
The new logic is implemented in mongoose_listener
Add a type for expected mandatory options. The map might contain general listener options as well.
Add a type for expected mandatory options. The map might contain general listener options as well.
Add a type for expected mandatory options. The map might contain general listener options as well.
Use maps and separate options from connection details. Pass correct options to the receiver, as previously inet options were passed there (which did not make sense, defaults were used)
Move the type to mongoose_listener
Make a type for required opts, the map might include additional listener-specific ones. Simplify options processing.
There are no listeners using it and the code is dead.
- New names - No UDP - New require opts
The paths are TOML ones, e.g. [listen, s2s] The resulting path would be [listen] or, more precisely, [listen, N] where N is the N-th element of the list. However, this would make it impossible to disinguish the listener types.
- Group the tests by listener types rather than by options. This makes test layout more similar to the config spec layout. - Introduce a new path notion - an integer in the path means an index in a list (TOML array), e.g. [listen, 1] is the first listener. The indexes start at one to be consistent with lists:nth/2 This change allows testing config options individually, just like e.g. for modules. - Add several missing tests.
Adjust the changed options accordingly.
Previous logic would put 'undefined' in the config options.
The default value is 'infinity' so it makes sense to make this value valid.
chrzaszcz
force-pushed
the
listener-config-rework
branch
from
d4e2df8
to
ef3e1e4
Compare
|
small_tests_24 / small_tests / ef3e1e4 small_tests_23 / small_tests / ef3e1e4 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / ef3e1e4 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / ef3e1e4 dynamic_domains_mysql_redis_24 / mysql_redis / ef3e1e4 ldap_mnesia_24 / ldap_mnesia / ef3e1e4 ldap_mnesia_23 / ldap_mnesia / ef3e1e4 internal_mnesia_24 / internal_mnesia / ef3e1e4 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / ef3e1e4 elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / ef3e1e4 pgsql_mnesia_24 / pgsql_mnesia / ef3e1e4 pgsql_mnesia_23 / pgsql_mnesia / ef3e1e4 mysql_redis_24 / mysql_redis / ef3e1e4 riak_mnesia_24 / riak_mnesia / ef3e1e4 mssql_mnesia_24 / odbc_mssql_mnesia / ef3e1e4 |
| TLSOpts = verify_opts(Verify) ++ TLSOpts1, | ||
| [ssl_crl_cache:insert({file, CRL}) || CRL <- proplists:get_value(crlfiles, Opts, [])], | ||
| TLSOpts = verify_opts(Verify) ++ TLSConfig, | ||
| [ssl_crl_cache:insert({file, CRL}) || CRL <- proplists:get_value(crlfiles, TLSConfig, [])], |
There was a problem hiding this comment.
interesting, that crlfiles are not tested and insert should probably be on the listener init.
(not related to the PR though)
There was a problem hiding this comment.
Yes, I left TLS options unchanged for now.
| fsm_limit_opts(#{max_fsm_queue := N}) -> | ||
| [{max_queue, N}]; | ||
| fsm_limit_opts(#{}) -> | ||
| case mongoose_config:lookup_opt(max_fsm_queue) of |
There was a problem hiding this comment.
the lookup could be done once in the init of the listener
There was a problem hiding this comment.
Maybe limit the extent of changes for this PR?
|
small_tests_24 / small_tests / 01ed2ab small_tests_23 / small_tests / 01ed2ab dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 01ed2ab dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 01ed2ab dynamic_domains_mysql_redis_24 / mysql_redis / 01ed2ab dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 01ed2ab ldap_mnesia_23 / ldap_mnesia / 01ed2ab ldap_mnesia_24 / ldap_mnesia / 01ed2ab internal_mnesia_24 / internal_mnesia / 01ed2ab pgsql_mnesia_23 / pgsql_mnesia / 01ed2ab elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 01ed2ab pgsql_mnesia_24 / pgsql_mnesia / 01ed2ab mysql_redis_24 / mysql_redis / 01ed2ab muc_SUITE:hibernation:hibernated_room_can_be_queried_for_archive{error,{{assertion_failed,assert,is_groupchat_message,
[<<"Restorable message">>],
undefined,"undefined"},
[{escalus_new_assert,assert_true,2,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
{line,84}]},
{muc_SUITE,wait_for_mam_result,3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4383}]},
{muc_SUITE,'-hibernated_room_can_be_queried_for_archive/1-fun-0-',3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4124}]},
{escalus_story,story,4,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{muc_SUITE,hibernated_room_can_be_queried_for_archive,1,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4120}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}mssql_mnesia_24 / odbc_mssql_mnesia / 01ed2ab muc_SUITE:hibernation:hibernated_room_can_be_queried_for_archive{error,{{assertion_failed,assert,is_groupchat_message,
[<<"Restorable message">>],
undefined,"undefined"},
[{escalus_new_assert,assert_true,2,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
{line,84}]},
{muc_SUITE,wait_for_mam_result,3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4383}]},
{muc_SUITE,'-hibernated_room_can_be_queried_for_archive/1-fun-0-',3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4124}]},
{escalus_story,story,4,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{muc_SUITE,hibernated_room_can_be_queried_for_archive,1,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4120}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}riak_mnesia_24 / riak_mnesia / 01ed2ab |
The main goal is to use the maps with defaults for all listener options and to pass such maps to all started listeners.
Several parts of listener management logic are changed:
mongoose_listener, which is a behaviour. The modulesejabberd_c2s,ejabberd_s2s_in,ejabberd_serviceandejabberd_cowboyimplement this behaviour. Thestart_listener/1function allows for a unified way to start particular listeners. They might delegate the startup tomongoose_tcp_listeneror have their own special logic (onlyejabberd_cowboyis doing so).mongoose_listener_sup. This makes it easier to understand the logic.The listeners are (and were) parsed from a TOML section (which contained a nested list for each listener type) to a flat map. I decided to use two kinds of paths in test helpers:
config_parser_helper:default_config/1has TOML paths, like[listen, c2s], which allows specifying defaults per listener type.config_parser_SUITEuses the resulting paths, e.g.[listen, 1], which means the first listener. This way expected option values can be tested one by one. The support for list indexes (1in the example) is a new feature. We can add it tomongoose_configin the future, should we need it outside the tests.A bug was also fixed - wrong options were passed to the modules from BOSH/websockets. The new type specs revealed this issue.
The rework of
TLSandhandlerswill be done in a separate task.