Merge pull request #3666 from esl/rel-5.1.0

Prepare release 5.1.0

CHANGELOG.md

@@ -1,3 +1,252 @@
+# [MongooseIM 5.1.0](https://github.com/esl/MongooseIM/releases/tag/5.1.0) - 2022-06-08
+
+## Highlights
+
+- Internal configuration rework
+- Inbox functionality and performance improvements
+- Remove the dynamically compiled modules
+- The new OTP 25 support
+- Miscellaneous enhancements, optimizations & improvements
+
+## All changes
+
+### Added
+- Cache consistency. Use `segmented_cache` as a cache backend and have all caches use it (#3330)
+- Persistent term config. Replace `ejabberd_config` with new persistent terms `mongoose_config`(#3367, #3338, #3343, #3356)
+- Create async pools basing on the MAM async workers concept (#3433)
+- Add new `ensure_subscribed_metric/3` to mongoose metrics module (#3353)
+- Add some property based tests for MAM's `is_archivable_message/4` (#3450)
+- Create SECURITY.md (#3562)
+- Allow to use podman instead of docker (#3522, #3543)
+- Support the new OTP 25 (#3667)
+
+### Removed
+- Hook handler records don't need the `key` field (#3342)
+- Remove dead code (#3451)
+- Remove deprecated `mod_aws_sns` (#3607)
+- Remove deprecated `mod_revproxy` (#3617)
+- Remove custom `eldap.hrl` (#3647)
+- Removed in internal configuration rework
+   - Remove all calls to `gen_mod:set_module_opt` from modules, making the module config constant(#3389)
+   - Remove calls to `gen_mod:set_module_opt(s)` in big tests (#3398)
+   - Remove `gen_mod:set_module_opt(s)` (#3468)
+   - Remove the ETS table from `gen_mod` (#3484)
+   - Eliminate `gen_mod:get_opt_subhost` (#3626)
+- Remove dynamically compiled modules (#3394)
+   - POC module `mod_private` (#3323)
+   - Remove `ejabberd_rdbms` and `mongoose_rdbms_type` (#3392)
+   - Allow `global` to be passed into `mongoose_backend` as an arg (#3386)
+   - MAM cassandra backend without dynamically compiled modules (#3373)
+   - Use persistent terms to store `backend_module` in `ejabberd_sm` (#3379)
+   - Add `mongoose_backend:get_backend_name` (#3345)
+   - `mod_auth_token` (#3336)
+   - `mod_last` (#3339)
+   - `mod_event_pusher_push` (#3340, #3348)
+   - `mod_privacy` (#3347)
+   - `mod_muc` (#3349)
+   - `mod_inbox` (#3350, #3362, #3440)
+   - `mod_muc_light` (#3351, #3418)
+   - `mod_offline` (#3352, #3371)
+   - `mod_smart_markers` (#3357)
+   - `mod_vcard` (#3358)
+   - `mod_roster` (#3364)
+   - `mod_pubsub` (#3368)
+   - `mod_http_upload` (#3372)
+   - `mod_keystore` (#3375)
+   - `mongoose_rdbms` (#3390)
+   - `ejabberd_router`(#3381)
+   - `mod_global_distrib_mapping` (#3582)
+
+### Changed
+- Hook functions (#3331)
+- Extend `safely` module API (#3337, #3341)
+- Make `mongoose_commands` errors human-readable (#3346)
+- Add the accumulator to the `is_muc_room_owner` and `can_access_room` room hooks (#3417)
+- Worker pool cleanups (#3419)
+- Upgrade mongoose JID (#3525)
+- Add backend logic to `mod_stream_management` (#3556)
+- Don't use Mnesia for IQ response tracking (#3560)
+- Use dynamic aware check for global domains in ACL module (#3327, #3332)
+- Store ACL specs in the global config (#3335)
+- Allow skipping cover compilation for small tests (#3382)
+- REST `/commands` endpoint now lists commands with more detailed info (#3391)
+- Revert scope to `global` for all global distributed Redis presets (#3393)
+- Refactor some usage of stanza types in `ejabberd_sm` (#3396)
+- Simplify stream management suite (#3453)
+- Performance and metrics touch on async workers (#3502, #3514, #3529)
+- Use ETS backend in `mongoose_router` instead of Mnesia (#3505)
+- Refactor `can_change_ra` function in `mod_muc` (#3581)
+- Reconfigure supervision flags in pools (#3593)
+- Install a hook to filter messages being buffered (#3654)
+- Make RDBMS asynchronous
+   - Add RDBMS `execute_cast` to use when the return value is not needed (#3485)
+   - Add async RDBMS order (#3611)
+   - Add RDBMS transaction async requests (#3643)
+- Perf optimizations
+   - Pre-build metric prefixes and fetch-or-rebuild when needed (#3634, #3649)
+   - Pattern matches the map key for more efficiency (#3635)
+   - Skip pretty-printing of unused accumulator field (#3637)
+- Message Archive Management improvements
+   - Use `mongoose_acc` timestamp in MAM (#3320)
+   - Same stanza id for peers (#3376)
+   - Retract on stanza ids (#3377, #3384, #3385)
+   - Retraction events (#3497, #3498, #3513)
+   - Report wrong stanza id more friendly (#3591)
+   - Make MAM return timestamps in microseconds (#3595)
+   - Reorder MAM's `filter_local_packet` hook order (#3623)
+   - MAM shaper improvements (#3641)
+- Smart markers improvements (#3590)
+   - Implement all the automatic removals (e.g. `remove_domain`) for the smart markers (#3544)
+   - Prepare a documentation page for smart markers (#3535)
+   - Implement all configurations for the smart markers (#3546, #3592)
+   - Do not display a warning when processing a message with repeated markers (#3673)
+- Inbox improvements (#3359, #3596, #3604)
+   - Improve types for dialyzer and log errors when they happen (#3366)
+   - Simplify time-unit conversions (#3405)
+   - Fix inbox filter bug that puts unnecessary load on the auth backend (#3449)
+   - Refactor Inbox tests (#3452)
+   - Use async pools in inbox (#3462, #3500)
+   - Fix `queryid` (#3494)
+   - Extend queries (#3597)
+   - Implement boxes that allow the classifying of entries (#3608)
+   - Add async removes - make asynchronous inbox consistent (#3616, #3670)
+- Internal config rework
+   - Simplify MUC Light config (#3387)
+   - Unfold global options without the temporary functions (#3406)
+   - Prepare tests for config defaults (#3408)
+   - Introduce config defaults for the 'general' section (#3409)
+   - Replace `local_config` records with key-value pairs (#3410)
+   - Add an option to format a section/list as a map (#3420)
+   - Move auth config spec to auth backend modules (#3437)
+   - Introduce defaults in the auth section (#3439)
+   - Section-based auth config (#3446)
+   - Introduce default values for auth methods (#3458)
+   - Auth password section config rework (#3463)
+   - Store module config in a map and keep it updated (#3469, #3488)
+   - Use `dynamic_modules` in big tests (#3481)
+   - Store listener config in a map (#3495)
+   - Keep ACL conditions as maps (#3501, #3504)
+   - Test config parsing with post-processing (#3507)
+   - Move `domain_certfile` to the 'general' section (#3512)
+   - Keep s2s options: 'outgoing', 'dns', 'address' in maps (#3516)
+   - Put all s2s options in a map and allow specifying them globally or per host-type (#3523)
+   - Module config as a map (#3534)
+   - Allow customizing auth module list per listener (#3539)
+   - Migrate segmented cache to maps (#3572, #3594)
+   - Test config metrics (#3613)
+  -  Put `mod_adhoc` opts in a map with defaults (#3537)
+   - Put `mod_bosh` opts in a map with defaults (#3540)
+   - Put `outgoing_pools` opts in a map with defaults (#3541)
+   - Put `mod_inbox` opts in a map with defaults (#3547)
+   - Put `mod_private` opts in a map with defaults (#3549)
+   - Put `mod_disco` opts in a map with defaults (#3550)
+   - Put `mod_version` opts in a map with defaults (#3552)
+   - Put `mod_vcard` opts in a map with defaults (#3553)
+   - Put `mod_time` opts in a map with defaults (#3554)
+   - Put `gen_mod_deps` opts in a map with defaults (#3555)
+   - Put `mod_last` opts in a map with defaults (#3557)
+   - Put `mod_mam` opts in a map with defaults (#3559)
+   - Put `mod_shared_roster_ldap` opts in a map with defaults (#3558)
+   - Put `mod_privacy` opts in a map with defaults (#3567)
+   - Put `mod_sic` opts in a map with defaults (#3568)
+   - Put `mod_roster` opts in a map with defaults (#3569)
+   - Put `mod_stream_management` opts in a map with defaults (#3571)
+   - Put `mod_muc` opts in a map with defaults (#3575)
+   - Put `mod_push_service_mongoosepush` opts in a map with defaults (#3577)
+   - Put `mod_pubsub` opts in a map with defaults (#3578)
+   - Put `mod_ping` opts in a map with defaults (#3579)
+   - Put `mod_muc_light` opts in a map with defaults (#3580)
+   - Put `mod_global_distrib` opts in a map with defaults (#3587)
+   - Put `mod_offline` opts in a map with defaults (#3589)
+   - Put `mod_caps` opts in a map with defaults (#3598)
+   - Put `mod_keystore` opts in a map with defaults (#3599)
+   - Put `mod_jingle_sip` opts in a map with defaults (#3600)
+   - Put `mod_carboncopy` opts in a map with defaults (#3602)
+   - Put `mod_event_pusher` opts in a map with defaults (#3603)
+   - Put `mod_csi` opts in a map with defaults (#3605)
+   - Put `mod_register` opts in a map with defaults (#3606)
+   - Put `mod_offline_chatmarkers` opts in a map with defaults (#3609)
+   - Put `mod_http_upload` opts in a map with defaults (#3614)
+   - Rework service configuration and management (#3620)
+   - Store service options in maps with defaults (#3624)
+   - Service and module options only in maps (#3625)
+   - Listener configuration rework (#3628)
+   - Rework HTTP handler configuration (#3636)
+   - Use maps for `mongoose_wpool` options (#3645)
+   - TLS config rework (#3653)
+   - Use `format_items = map` by default for config sections (#3655)
+   - Access commands in maps (#3656)
+   - Final internal config cleanup (#3659)
+
+### Fixed
+- Edoc generation (#3333)
+- Rest admin API - return 404 when command not found instead of 500 (#3383)
+- Identify only stored users on offline routing (#3395)
+- Fix typos (#3475, #3621)
+- Fix certs (#3447, #3435, #3447, #3632, #3640)
+- Fix print-dots for Mac OS Monterey (#3480)
+- Add a newline after module options to improve config readability (#3402)
+- Correctly pass connection options to Fusco (#3426)
+- Fix Github Actions CI - run MSSQL as root again (#3456)
+- Fix `ejabberd_c2s` - don't drop the host type in `update_stanza` (#3464)
+- Adding `queryId` to archived inbox messages (#3482, #3492)
+- MAM - Avoid asking for user exist with a room (#3470)
+- Fix starting new caches (#3508, #3518)
+- Add `server_name_indication_host` config option (#3510)
+- Less MIM compiler warnings (#3524)
+- Add config flag to be able to set sni hostname matching (#3528)
+- Fix `is_configured` in async pools (#3566)
+- Do not call the obsolete backend API in CT hook (#3570)
+- Fix metrics for auth backend not being reported (#3585)
+- Fix `cowboy_static` options (#3618)
+- Fix `mod_ping` with `host_types` (#3638)
+- Fix a registration timeout cleanup (#3601, #3671)
+- Fix restart RabbitMQ connection (#3631, #3672)
+- Use metric probes instead of eval for slow calculated metrics (#3644)
+- Fix flaky tests and speed up CI:
+   - Gather list of failed test cases in a separate issue (#3361)
+   - Fix `rest_client_SUITE:muc:messages_can_be_paginated_in_room` (#3363)
+   - Fill the gaps in the event table for `service_domain_db` (#3365)
+   - Run muclight inbox tests one by one instead of parallel (#3374)
+   - Print preset in `tools/circle-publish-github-summary-comment.sh` (#3378)
+   - Enable `mod_last` only on its test suite (#3413)
+   - Enable DB modules in tests only when used (#3416)
+   - Retry insert MAM prefs on deadlock (#3421)
+   - Fix `domain_removal_SUITE:last_removal` (#3422)
+   - Remove wait from `unavailable_resources_dont_get_carbons` (#3431)
+   - Run tests in pubsub:basic in parallel (#3432)
+   - Use in-memory riak backend to speed-up IO (#3434)
+   - Fix `sm_SUITE` (#3436)
+   - Use `wait_for_archive_size` inside `send_rsm_messages` helper (#3443)
+   - Avoid a race condition in `sm_SUITE:resume_session_state_stop_c2s` (#3444)
+   - Fix `offline_SUITE:max_offline_messages_reached` (#3445)
+   - Reroute presences correctly on reconnect (SM fix) (#3459)
+   - Wait for async writers to flush messages in tests (#3466)
+   - Inbox flaky tests fixes (#3630)
+
+### Other
+- Faster, more stable CircleCI on docker with DB backends (#3401, #3519, #3530)
+- Use OTP 24.3.4 for CircleCI (#3661)
+- Updating dependencies and cleaning the rebar.lock file (#3425, #3430, #3465, #3490, #3520, #3531, #3533, #3573, #3657, #3669)
+- Document low-level MAM options (#3329)
+- Document MUC Light cache affiliations (#3509)
+- Minor documentation improvements (#3369, #3400, #3404, #3412, #3438, #3532, #3545, #3668)
+- Update CONTRIBUTING.md (#3622)
+
+## Commits, merged PRs and closed issues
+- [List of merged PRs](https://github.com/esl/MongooseIM/pulls?q=is%3Apr+is%3Amerged+milestone%3A5.1.0)
+
+- [List of closed issues](https://github.com/esl/MongooseIM/issues?q=is%3Aissue+is%3Aclosed+closed%3A2021-10-08..2022-06-08)
+
+- [Repository history for this release](https://github.com/esl/MongooseIM/graphs/contributors?from=2021-10-08&to=2022-06-08&type=c)
+
+[List of merged PRs based on merge date]: # (https://github.com/esl/MongooseIM/pulls?q=is%3Apr+base%3Amaster+merged%3A%222021-10-08..2022-06-08%22+sort%3Acreated-asc+)
+
+## Special thanks to our contributors:
+- [@kianmeng](https://github.com/kianmeng) Fix typos in doc (#3545)
+- [@amccormick93](https://github.com/amccormick93) Fix a registration timeout cleanup (#3601)
+- [@ColaCheng](https://github.com/ColaCheng) Fix restart RabbitMQ connection (#3631)
+
 # [MongooseIM 5.0.0](https://github.com/esl/MongooseIM/releases/tag/5.0.0) - 2021-10-07
 
 ## Highlights

README.md

@@ -62,6 +62,7 @@ Check out our test results:
 See the documentation for the latest releases:
 
 * [Master](https://esl.github.io/MongooseDocs/latest/)
+* [5.1.0](https://esl.github.io/MongooseDocs/5.1.0/)
 * [5.0.0](https://esl.github.io/MongooseDocs/5.0.0/)
 * [4.2.0](https://esl.github.io/MongooseDocs/4.2.0/)
 * [4.1.0](https://esl.github.io/MongooseDocs/4.1.0/)

VERSION

@@ -1 +1 @@
-5.0.0
+5.1.0

doc/index.md

@@ -79,6 +79,7 @@ Check out our test results:
 See the documentation for the latest releases:
 
 * [Master](https://esl.github.io/MongooseDocs/latest/)
+* [5.1.0](https://esl.github.io/MongooseDocs/5.1.0/)
 * [5.0.0](https://esl.github.io/MongooseDocs/5.0.0/)
 * [4.2.0](https://esl.github.io/MongooseDocs/4.2.0/)
 * [4.1.0](https://esl.github.io/MongooseDocs/4.1.0/)