fix: remove caching in fetchListOfGroups

[kesselb]

• Resolves: [Bug]: v28 - Receiving erroneous emails about removal from LDAP groups #42195

Summary

When using nested groups without a memberof overlay, then fetchListOfGroups is called from getGroupsByMember without applying the group filter.

In some setups, the "unfiltered" result is then written back to the group mapping table. That might cause random "An administrator removed you from group" activities.

I was unable to replicate it locally, but we got the feedback that the random activities stopped with the patch applied.

Ref: #42195

TODO

• CI & Tests
• Review

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[come-nc]

Is that fixing the same issue as #45364?

[kesselb]

Is that fixing the same issue as #45364?

Looks related.

server/apps/user_ldap/lib/Access.php

Lines 949 to 966 in dde0b48

	$groupRecords = $this->searchGroups($filter, $attr, $limit, $offset);
	$listOfDNs = array_reduce($groupRecords, function ($listOfDNs, $entry) {
	$listOfDNs[] = $entry['dn'][0];
	return $listOfDNs;
	}, []);
	$idsByDn = $this->getGroupMapper()->getListOfIdsByDn($listOfDNs);
	array_walk($groupRecords, function (array $record) use ($idsByDn) {
	$newlyMapped = false;
	$gid = $idsByDn[$record['dn'][0]] ?? null;
	if ($gid === null) {
	$gid = $this->dn2ocname($record['dn'][0], null, false, $newlyMapped, $record);
	}
	if (!$newlyMapped && is_string($gid)) {
	$this->cacheGroupExists($gid);
	}
	});

Our working theory was that the above block maps LDAP groups that do not match the group filter. That can happen with nested groups, where we cannot use the group filter for the query right away.

I think it should work to combine this pr with #45364: Keep the caching logic, but turn auto-mapping off.

If @blizzz and you think we should keep it, then let's do it. But please keep in mind that the amount of caching logic makes the code really difficult to understand and debug, and therefore removing the cache block improves the maintainability a bit.

[come-nc]

Quite frankly I think we can remove this logic it looks like it tries to be too smart.
@kesselb Could you test how many extra LDAP queries are fired, if any, with this change when listing the groups of a user or the members of a group?

[kesselb]

I tried gathering you some reliable data, but's the caching layers, having the mapping in the db already yes/no, .... makes it horrible to get reproducible results.

Login with LDAP user:

ldapCacheTTL: 0

login master, 39 ldap calls, WU1eZOvwNdoB5eGtXzmg
login branch, 39 ldap calls, xH5v3gZjD1DC978P19s8


ldapCacheTTL: 600

(extra login was done before to warmup the cache)

login master, 6 ldap calls, RCeguhBIPMUV5367Y3EP
login branch, 15 ldap calls, aZrSU6ja9xgU3k85USB2

Calls RCeguhBIPMUV5367Y3EP (master)

"ldap_explode_dn"
"ldap_connect"
"ldap_set_option"
"ldap_set_option"
"ldap_set_option"
"ldap_bind"

Calls aZrSU6ja9xgU3k85USB2 (this branch)

"ldap_explode_dn"
"ldap_connect"
"ldap_set_option"
"ldap_set_option"
"ldap_set_option"
"ldap_bind"
"ldap_search"
"ldap_errno"
"ldap_get_entries"
"ldap_parse_result"
"ldap_connect"
"ldap_set_option"
"ldap_set_option"
"ldap_set_option"
"ldap_bind"

[jgrocha]

I may be missing something, but I did not solve the issue on my side with this fix. I've commented out the code on my side, but the problem persists. I'm still getting those emails "An administrator removed you from group ..."

How should I test this? Should I clear something from the BD? Reconfigure LDAP?

[come-nc]

I may be missing something, but I did not solve the issue on my side with this fix. I've commented out the code on my side, but the problem persists. I'm still getting those emails "An administrator removed you from group ..."

How should I test this? Should I clear something from the BD? Reconfigure LDAP?

You should double-check your LDAP configuration in Nextcloud for group-member association.

[jgrocha]

I may be missing something, but I did not solve the issue on my side with this fix. I've commented out the code on my side, but the problem persists. I'm still getting those emails "An administrator removed you from group ..."
How should I test this? Should I clear something from the BD? Reconfigure LDAP?

You should double-check your LDAP configuration in Nextcloud for group-member association.

Upgrading to the recent 29.0.8 version fixed the problem. This same code is still there (the code commented out in this PR on apps/user_ldap/lib/Access.php) and works as it used to work.

[jgrocha]

I may be missing something, but I did not solve the issue on my side with this fix. I've commented out the code on my side, but the problem persists. I'm still getting those emails "An administrator removed you from group ..."
How should I test this? Should I clear something from the BD? Reconfigure LDAP?

You should double-check your LDAP configuration in Nextcloud for group-member association.

Upgrading to the recent 29.0.8 version fixed the problem. This same code is still there (the code commented out in this PR on apps/user_ldap/lib/Access.php) and works as it used to work.

Upgrading to 30.0.1 restores this issue!

A lot of new email messages with An administrator removed you from group xxxx

[joshtrichards]

Issue #29832 may or may not also contain some related reports this covers.

[come-nc]

I merged the other PR, but I still think we should merge this one.
The only possible performance impact is for new groups in the LDAP, which is a rare situation.
It avoids potential complicated bugs and ease debugging such issues in the future.

[come-nc]

CI failing on Scenario: Search only with group members - allowed, so I suppose it is related. It’s surprising though.

[come-nc]

CI failing on Scenario: Search only with group members - allowed, so I suppose it is related. It’s surprising though.

This was because the test changes configuration and does not wait for group mapping to sync, so it was relying on the automapping that this PR is removing.
I added a manual sync of the concerned group state in the test.

[come-nc]

/backport to stable31

[come-nc]

/backport to stable30

[come-nc]

/backport to stable29

[brianjmurrell]

I applied this patch to my local NC 30 instance but my Activity log is still showing a battling removal and addition of groups:

Brian J Murrell removed you from group brian February 27, 2025 12:49 PM 28 minutes ago

An administrator added you to group brian February 27, 2025 12:48 PM 29 minutes ago

Brian J Murrell removed you from group brian February 27, 2025 11:48 AM 88 minutes ago

An administrator added you to group brian February 27, 2025 11:45 AM 2 hours ago

Brian J Murrell removed you from group brian February 27, 2025 10:48 AM 2 hours ago

An administrator added you to group brian February 27, 2025 10:43 AM 3 hours ago

Brian J Murrell removed you from group brian February 27, 2025 9:44 AM 4 hours ago

An administrator added you to group brian February 27, 2025 9:41 AM 4 hours ago