Add LDAP group sync to Teams, fixes #1395

[svenseeberg]

• Add setting for a JSON that maps LDAP groups to Org Teams. Format:

{"cn=MyGroup,cn=groups,dc=example,dc=org": {"MyGiteaOrganization": ["MyGiteaTeam1", "MyGiteaTeam2", ...], ...}, ...}

• Sync is being run on login and periodically.
• Existing group filter settings are reused.
• Removal of users from Teams can optionally be enabled.
• Teams that are not listed in the mapping JSON are not touched.

fixes #1395

[6543]

@svenseeberg can you resolve conflicts :)

PS: since its a pull from an org we maintainer cant apply code suggestions or resolve conflicts. if you need help just tell us.

[svenseeberg]

@svenseeberg can you resolve conflicts :)

PS: since its a pull from an org we maintainer cant apply code suggestions or resolve conflicts. if you need help just tell us.

Right, should not be an issue. We will take care of rebasing on the current main branch.

[svenseeberg]

I'll also look into the linting errors.

[6543]

make fmt

[6543]

you dont have to hurry we are currently in feature-freeze ... :)

& we need some tests for this code

[svenseeberg]

we need some tests for this code

Are there any specific requirements for the tests? Mocking an LDAP server is somewhat complicated ;-) However, we could easily test the functions that do not interact with the LDAP server.

[6543]

well we have unit tests who test selve contained functions or easy to moke on ...
example: https://github.com/go-gitea/gitea/blob/main/modules/auth/openid/discovery_cache_test.go

and we have integration tests:
example: https://github.com/go-gitea/gitea/blob/main/integrations/auth_ldap_test.go

and we have a running ldap to test against (https://drone.gitea.io/go-gitea/gitea/41501/2/5)

[svenseeberg]

I guess we can work with that :-)

[zeripath]

We have some LDAP tests already which use a docker container for the LDAP.

[codecov-commenter]

Codecov Report

Merging #16299 (a75516d) into main (212e81f) will increase coverage by 0.39%.
The diff coverage is 70.12%.

❗ Current head a75516d differs from pull request most recent head 0d402cc. Consider uploading reports for the commit 0d402cc to get more accurate results

@@            Coverage Diff             @@
##             main   #16299      +/-   ##
==========================================
+ Coverage   45.74%   46.14%   +0.39%     
==========================================
  Files         831      839       +8     
  Lines       92178    92563     +385     
==========================================
+ Hits        42171    42712     +541     
+ Misses      43249    43066     -183     
- Partials     6758     6785      +27     

Impacted Files	Coverage Δ
build/codeformat/formatimports.go	66.35% <0.00%> (ø)
cmd/admin.go	0.00% <0.00%> (ø)
cmd/admin_auth_ldap.go	75.49% <0.00%> (-2.59%)	⬇️
cmd/cert.go	0.00% <0.00%> (ø)
cmd/doctor.go	0.00% <0.00%> (ø)
cmd/dump.go	0.89% <0.00%> (ø)
cmd/dump_repo.go	0.00% <0.00%> (ø)
cmd/manager.go	0.00% <ø> (ø)
cmd/serv.go	2.46% <0.00%> (ø)
cmd/web.go	0.00% <0.00%> (ø)
... and 321 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f910924...0d402cc. Read the comment docs.

[svenseeberg]

@6543 we updated our pull request an included tests. Can you please review?

[melegiul]

Thank you for your review @6543
I'm sorry for the long response time, but now I have removed the funk package in 3a032cc
Unfortunately I could not find existing functions to use instead, so I added some to the utils module for these three use cases:

1. get all keys from a map
2. get the intersection of two slices
3. get the difference of two slices

Thanks in advance for your feedback

[6543]

@svenseeberg can you resolve conflicts :)

(just do a merge of main and resolve - rebase or squash not required since pull's are squash-merged anyway)

[svenseeberg]

@svenseeberg can you cherry-pick netzbegruenung@6ef197e ? I thinkt that's the cleanset solution ... & merge upstream in first

Oh sorry, I read this comment after I merged the PR.

I added @wxiaoguang and @6543 as collaborators to our repo.

[wxiaoguang]

Since the main branch has changed a lot (including the lint rules), there are some new work to do. Give me some more time.

[wxiaoguang]

Hmm... CI fails, maybe I did something wrong during the refactoring. @svenseeberg could you help to take a look?

Fixing ...

[6543]

@wxiaoguang yes you did not adjust the tests appropriate

[6543]

ok two things need to be done:

1. test if this pull works in current state for you @svenseeberg @gtudan
2. update inteagration tests

@svenseeberg thanks - that will help :)

[gtudan]

Just tried the latest build and I can confirm that it works

[6543]

tests PASS localy

[wxiaoguang]

🚀

[svenseeberg]

Thank you all for your work on this!

[scs-kno]

Is there a way to debug this feature? It doesn't create any Organizations for me with the latest Docker image.

[melegiul]

Is there a way to debug this feature? It doesn't create any Organizations for me with the latest Docker image.

Actually this feature is not included, orgs and teams have to be created manually
But should be not much effort to automatically generate them

[scs-kno]

Has anybody written a short manual/tutorial on how to use this feature?

[melegiul]

Has anybody written a short manual/tutorial on how to use this feature?

This file contains an example for the Team Group Map field with the expected JSON format
https://github.com/go-gitea/gitea/blob/main/services/auth/source/ldap/README.md
Please note that you have to properly set the fields "Group Attribute Containing List Of Users" and "User Attribute Listed In Group" to detect the users groups

[svenseeberg]

To elaborate a little on the previous answer: You need to have created the organizations and teams before you can sync LDAP users into them. To create the config and JSON you need to make sure that you have the following information:

• the org and team names
• the DN (distinguished name) of your group, for example cn=mygroup,dc=groups,dc=example,dc=com
• the attribute that contains the membership information within your LDAP group object (usually something like member)
• You need to know which user attribute is contained in the group membership attribute (for example the "username" cn or the full dn)

[janosmiko]

Hi @svenseeberg !

First of all, thank you for your awesome work on this! 👏

Is it possible somehow to dynamically map LDAP nested groups as organizations and teams?

Eg:

LDAP:                   | Gitea group:
- group1                | Organization 1
  - subgroup1           | Team 1
    - member 1          | Gitea user 1

And later if I create a new main group, and a subgroup (in both LDAP and Gitea) and a new member in LDAP attach that user to the team in Gitea?

[svenseeberg]

I don't think that this can currently be done. The options for configuring the LDAP queries are very limited.

[janosmiko]

Hi @svenseeberg !

First of all, thank you for your awesome work on this! 👏

Is it possible somehow to dynamically map LDAP nested groups as organizations and teams?

Eg:

LDAP:                   | Gitea group:
- group1                | Organization 1
  - subgroup1           | Team 1
    - member 1          | Gitea user 1

And later if I create a new main group, and a subgroup (in both LDAP and Gitea) and a new member in LDAP attach that user to the team in Gitea?

If anyone's interested, I implemented an external solution to do this:
https://github.com/janosmiko/gitea-ldap-sync