Synchronize members with data source and task manager crash randomly

[aepli]

Hello,

Updating lists' members randomly fails with the inclusion of members from another local list.

I have noticed this on a list that still uses the include_list directive of Sympa 6.1.x and not the new include_sympa_list parameter.

On our test environment, we use Dumper on the %{$include_member} variable in the _load_list_members_from_include function of the List.pm module to try to understand what was happening.

As you can see in the extract of our log file (at the end of this message), the content of the include_member hashtable is not ordered!

reformatted content of the include_member variable:

%{$include_member} = {
    'include_sympa_list' => [
        {
            'filter' => undef,
            'name' => 'include_list include-list-full',
            'listname' => 'include-list-full'
        }
    ],
    'defaults' => {}
};
%{$include_member} = {
    'defaults' => {},
    'include_sympa_list' => [
        {
            'filter' => undef,
            'listname' => 'include-list-empty',
            'name' => 'include_list include-list-empty'
        }
    ]
};

Looking through the code, I found that initialization of @defs array uses the value related to the first key of the %{$include_member} hashtable.

This can create a problem if defaults for the key is the first, because it provides an empty hastable.

bin/Sympa/List.pm (without Dumper)

...
5975
5976         if ($include_member and %$include_member) {
5977             my @types = keys %{$include_member};
5978             my $type  = $types[0];                    #FIXME: Gets random key?
5979             my @defs  = @{$include_member->{$type}};
5980             my $def   = $defs[0];
5981             push @{$sources->{$type}}, $def;
5982         }
...

I do not know exactly how to fix the problem, but I used a workaround by replacing the include_list directive with the new include_sympa_list parameter on all our lists and I did not have any more crash.

Regards.

---

/var/log/sympa/sympa.log:

Jan 16 11:07:31 lunimlsdev3 wwsympa[8695]: info main::do_sync_include(test-include-listes) [robot listesdev.unige.ch] [session 11186343395275] [client fd69:620:600:16::cafe:192] [user philippe.aepli@unige.ch] [list test-include-listes]
Jan 16 11:07:31 lunimlsdev3 wwsympa[8695]: info Sympa::List::_load_list_members_from_include() Contenu du dictionnaire include_member: $VAR1 = {#012          'include_sympa_list' => [#012                                    {#012                                      'filter' => undef,#012                                      'name' => 'include_list include-list-full',#012                                      'listname' => 'include-list-full'#012                                    }#012                                  ],#012          'defaults' => {}#012        };
Jan 16 11:07:31 lunimlsdev3 wwsympa[8695]: info Sympa::List::_load_list_members_from_include() Contenu du dictionnaire include_member: $VAR1 = {#012          'defaults' => {},#012          'include_sympa_list' => [#012                                    {#012                                      'filter' => undef,#012                                      'listname' => 'include-list-empty',#012                                      'name' => 'include_list include-list-empty'#012                                    }#012                                  ]#012        };
Jan 16 11:07:31 lunimlsdev3 wwsympa[8695]: err main::#1638 > main::do_sync_include#16729 > Sympa::List::sync_include#6878 > Sympa::List::_load_list_members_from_include#5983 DIED: Not an ARRAY reference at /home/sympa/bin/Sympa/List.pm line 5983.

---

_Debian 9.3
Perl 5.24.1
exim4 4.89
mariadb 15.1
apache2 2.4.25
sympa 6.2.22

[ikedas]

Hi @aepli,
I can't understand what the code you showed is doing, too. Can you apply this patch and see if the problem is solved? If this change is promising, I'll submit PR later.

(Also, if possible, please check the case with multiple include_* paragraphs).

[aepli]

Hello @ikedas ,

I tested with one, two and three lists and the problem has not reappeared.
It seems corrected.
Thanks.

Do you need more tests?

Greetings.

[ikedas]

@aepli, thanks for testing. It also seems working well on my side.

If you are ok, please close this issue.