Sync_Include: Add a button to synchronize owners / moderators in Web interface

[aepli]

Expected Behavior

As with members, owners and moderators can be added dynamically from data sources.

Now that the management of members, owners and moderators are in the same menu.

Could you add a button to synchronize the owners / moderators in the same way as the "Synchronize members with data sources" button of the members (action_sync_include) ?

Current Behavior

Synchronization is performed once when the data source is added and then only using a scheduled task.

Context

We have many of lists with dynamic owners and / or moderators. This functionality would be of great help when deleting rights, since user synchronization is only daily.

Thank you in advance for your help.

Regards.

[dverdin]

I might be wrong but, if I remember correctly, the owner/editor synchronization is triggered along with users synchronization when clicking the button.

But maybe the button is missing when only owners.editors are included from data sources, and not users?

[aepli]

Hello,

I have tested the workaround of @dverdin, but without success.

Here is the test protocol:

• a list.incl file in the data_sources folder, for list inclusions
• a Bar list with two members
• a Foo list which uses the Bar list to dynamically add owners with or without members adding dynamically
• deleting one member in the Bar list
• using the "Synchronize members with data sources" button in the menu: Users > Subscribers
• the owner is not removed
• forced execution of the synchronization task
• the owner is deleted!

Command to force the synchronization task to run after one minutes:

$ mv var/spool/sympa/task/{*,$(date -d "$(date) 1 minutes" +'%s')}.INIT.sync_include.foo@<my_domain>

Note: don't forget to replace <my_domain> with your email domain and "foo" with your list name

list.incl:

include_list [% param.0 %]

Log when adding the Bar list:

Feb  6 13:01:52 lunimlsdev4 wwsympa[52636]: info main::do_edit_list_request(data_source) [robot listesdev.unige.ch] [session 53245847640687] [client xxxx::xxxx] [user philippe.aepli@unige.ch] [list foo]
Feb  6 13:02:05 lunimlsdev4 wwsympa[52636]: info main::do_edit_list() [robot listesdev.unige.ch] [session 53245847640687] [client xxxx::xxxx] [user philippe.aepli@unige.ch] [list foo]
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::List <context=foo@listesdev.unige.ch;id=03321335;role=owner;name=include_list bar>: 1 included, 0 deleted, 0 updated, 0 kept
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>: 2 included, 0 deleted, 0 updated
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=editor>
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: err main::#1544 > main::do_edit_list#10635 > Sympa::List::sync_include_admin#4881 > Sympa::Spindle::spin#95 > Sympa::Request::Handler::include::_twist#188 > Sympa::Request::Handler::include::_update_inclusion_table#552 > Sympa::Database::do_prepared_query#382 Unable to execute SQL statement "INSERT INTO inclusion_table (target_inclusion, role_inclusion, source_inclusion, update_epoch_inclusion) VALUES (?, ?, ?, ?)": (23000) Duplicate entry 'bar@listesdev.unige.ch-owner-foo@listesdev.unige.ch' for key 'PRIMARY'
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: err main::#1544 > main::do_edit_list#10635 > Sympa::List::sync_include_admin#4881 > Sympa::Spindle::spin#95 > Sympa::Request::Handler::include::_twist#188 > Sympa::Request::Handler::include::_update_inclusion_table#573 Unable to update list Sympa::List <foo@listesdev.unige.ch> in database
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::List <context=foo@listesdev.unige.ch;id=03321335;role=owner;name=include_list bar>: 0 included, 0 deleted, 0 updated, 1 kept
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>: 0 included, 0 deleted, 0 updated
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=editor>
Feb  6 13:02:07 lunimlsdev4 wwsympa[52636]: info main::do_edit_list_request(data_source) [robot listesdev.unige.ch] [session 53245847640687] [client xxxx::xxxx] [user philippe.aepli@unige.ch] [list foo]

Log when using the "Synchronize members with data sources" button:

Feb  6 13:12:15 lunimlsdev4 wwsympa[52636]: info main::do_sync_include(foo) [robot listesdev.unige.ch] [session 53245847640687] [client xxxx::xxxx] [user philippe.aepli@unige.ch] [list foo]
Feb  6 13:12:15 lunimlsdev4 wwsympa[52636]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=member>
Feb  6 13:12:15 lunimlsdev4 wwsympa[52636]: info main::do_review() [robot listesdev.unige.ch] [session 53245847640687] [client xxxx::xxxx] [user philippe.aepli@unige.ch] [list foo]

Log when forced execution of the synchronization task:

Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessTask::_execute() Running task Sympa::Task <date=1580991472;label=INIT;model=sync_include;context=foo@listesdev.unige.ch>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=member>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::List <context=foo@listesdev.unige.ch;id=03321335;role=owner;name=include_list bar>: 0 included, 0 deleted, 1 updated, 0 kept
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=owner>: 0 included, 1 deleted, 1 updated
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=foo@listesdev.unige.ch;role=editor>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessTask::do_next() line 5 of sync_include: next(1581077888, INIT)
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spool::store() Sympa::Task <date=1581077888;label=INIT;model=sync_include;context=foo@listesdev.unige.ch> is stored into Sympa::Spool::Task as <1581077888.INIT.sync_include.foo@listesdev.unige.ch>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessTask::do_next() --> new task Sympa::Task <date=1581077888;label=INIT;model=sync_include;context=foo@listesdev.unige.ch>
Feb  6 13:18:08 lunimlsdev4 task_manager[76955]: notice Sympa::Spindle::ProcessTask::_execute() The task Sympa::Task <date=1580991472;label=INIT;model=sync_include;context=foo@listesdev.unige.ch> is now useless. Removing it

I have two comments:

• With the button, only the members synchronization action is executed, unlike the task which synchronizes all the roles.
• I do not understand the error message when adding the data source. Two inclusions at following ?

Regards.

[ikedas]

Hi @aepli ,

Could you please apply this patch and check if the feature you requested will work properly?

• Note that this patch includes the patch I presented at issue Cannot remove owners/editors when their external data source as removed #858.

[aepli]

Hi @ikedas,

I only tested this patch, as it contains the two fixes.

The features work properly, I just note a few points.

• The owners and moderators synchronization buttons are at the end of the form unlike the members.

• Adding members is not immediate with the owner_include option (use of the synchronization button / task execution) unlike the include_sympa_list option. But I think this is either not recommended or deprecated.

• When removing the data source, suppression of the members is not immediate as opposed to owners and moderators. You need wait for execution of the scheduled task.

• If no more data source is present, the last execution of the scheduled task does not generate a new task, it is therefore removed.

Regards.

[ikedas]

I only tested this patch, as it contains the two fixes.

The features work properly, I just note a few points.

Thanks!

• The owners and moderators synchronization buttons are at the end of the form unlike the members.

I can't decide on which side these buttons would be better to be placed.

• Adding members is not immediate with the owner_include option (use of the synchronization button / task execution) unlike the include_sympa_list option. But I think this is either not recommended or deprecated.
• When removing the data source, suppression of the members is not immediate as opposed to owners and moderators. You need wait for execution of the scheduled task.

IMHO updating parameters for data sources need not adding/removing those users immediately. Because if (one or more) big data sources were updated, web interface would stall for a long time.

I'll add a message describing "Members (or owners, moderators) will be updated soon (or later)".

• If no more data source is present, the last execution of the scheduled task does not generate a new task, it is therefore removed.

Such tasks will add no more users, but they have to remove users who have been included.

[aepli]

Hi @ikedas,

Thank you for this clarifications.

Regards.

[ikedas]

@aepli, I added some messages to describe update will be made later. As changes has been merged into current sympa-6.2 branch and will be included in the next release, it's grateful that you will check it and suggest more improvements.
Thanks for your suggestions!