Export feature for user and group list

[bpcurse]

Is your feature request related to a problem? Please describe.
There seems to be no simple way to export a list of all registered users. Especially on larger instances an external user overview with comments and task automation (like identifying long inactive users and notifying the admin to clean up) can be very handy.

Describe the solution you'd like
A simple button in the user management view, that exports the list to e.g. a csv file (or better: asks which format to export to and which columns should be included).
The same could be done to export a list of all existing groups, especially as the group management features in nextcloud are insufficient thus far.

Additional context
As the count of groups the individual user belongs to can differ a lot, a suitable export format should be chosen.

[drkmccy]

I'm surprised nobody has commented on this. A user list export feature is something pretty basic and necessary.

[bpcurse]

Me too... Due to this missing basic function I've experienced quite an embarrassing situation some time ago. A group manager of one of our user groups needed an overview of all group members.
It resulted in taking several screenshots of the user management view 🙈

If I would have had more time, a database export might have been possible, but a professional UI shouldn't lack this feature. I wonder how large Nextcloud subscription users with several 1000 users handle this?

[ledufakademy]

+1, i need to emailm all user ... so create a list for Thunderbird and no Export user list function.

[kesselb]

I wonder how large Nextcloud subscription users with several 1000 users handle this?

LDAP

i need to emailm all user ... so create a list for Thunderbird and no Export user list function.

<?php

$nextcloudUrl = 'https://nextcloud.test';
$adminUsername = 'admin';
$adminPassword = 'admin';

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $nextcloudUrl . '/ocs/v1.php/cloud/users');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERPWD, $adminUsername . ':' . $adminPassword);
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'OCS-APIRequest: true',
    'Accept: application/json'
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$data = json_decode(curl_exec($ch), true);

$emails = [];
if (isset($data['ocs']['data']['users'])) {
        $users = $data['ocs']['data']['users'];
        foreach($users as $userId) {
                curl_setopt($ch, CURLOPT_URL, $nextcloudUrl . '/ocs/v1.php/cloud/users/' . $userId);
                $userData = json_decode(curl_exec($ch), true);
                if(isset($userData['ocs']['data']['email'])) {
                        $emails[] = $userData['ocs']['data']['email'];
                }
        }
}

echo implode(',', $emails);

[bpcurse]

@kesselb Thank you!
Although it's quite slow (more than a minute for ~200 users) it does the job.

I did some minor changes because I don't like saving an unhashed admin password on the server and I needed a vertical list instead of the comma separated one.

Calling this through external sites app is quite convenient 👍

<?php

if(isset($_POST['submit']))
{
	$nextcloudUrl = $_POST['url'];
 	$adminUsername = $_POST['user'];
 	$adminPassword = $_POST['password'];
	 
 	if(isset($adminPassword) && isset($adminUsername) && isset($nextcloudUrl))
  	{
		$ch = curl_init();
		curl_setopt($ch, CURLOPT_URL, $nextcloudUrl . '/ocs/v1.php/cloud/users');
		curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
		curl_setopt($ch, CURLOPT_USERPWD, $adminUsername . ':' . $adminPassword);
		curl_setopt($ch, CURLOPT_HTTPHEADER, [
			'OCS-APIRequest: true',
    			'Accept: application/json'
		]);
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

		$data = json_decode(curl_exec($ch), true);

		$emails = [];
		if (isset($data['ocs']['data']['users'])) {
			$users = $data['ocs']['data']['users'];
        		foreach($users as $userId) {
			curl_setopt($ch, CURLOPT_URL, $nextcloudUrl . '/ocs/v1.php/cloud/users/' . $userId);
                	$userData = json_decode(curl_exec($ch), true);
                	if(isset($userData['ocs']['data']['email'])) {
                 	    $emails[] = strtolower($userData['ocs']['data']['email']);
                	}
       		}
	}

	echo "<ul><li>".implode('</li><li>', $emails)."</li></ul>";
	$adminPassword = ''; 
	}
}
?>

<form action="#" method="post">
    <input type="text" name="url" placeholder="https://cloud.example.com">
	<input type="text" name="user" placeholder="Admin user name">
	<input type="password" name="password" placeholder="Admin user password">
    <input type="submit" name="submit" value="submit">
</form>

[kesselb]

I still think we should add a basic export feature sometimes. But in the meantime calling the api should do the trick.

Although it's quite slow (more than a minute for ~200 users) it does the job.

Yep. Curl is able to do multiple request at once. Probably not 200 but 5 or 10: https://www.php.net/manual/en/function.curl-multi-init.php

[bpcurse]

@kesselb I would like to enhance the code further, add some options to make it more versatile and publish it on github in a repository.
Would you mind if I use your initial code (see above) as a basis?

[kesselb]

@bpcurse I don't mind. There is probably another (python based) script somewhere. Some people at help.nextcloud.com shared it.

[bpcurse]

The nextcloud-userexport script I made from the initial code provided above can be found here:
https://github.com/bpcurse/nextcloud-userexport
@kesselb Thanks for the support

I hope that it will be useful for some admins.
@derekblankmccoy @ledufakademy

Feedback, feature and pull requests are welcome. Happy holidays!

[bpcurse]

I have just released an enhanced version (v1.0.0) of the before mentioned script.
https://github.com/bpcurse/nextcloud-userexport/releases/tag/v1.0.0

As it helps as a workaround I hope this isn't seen as hijacking this issue.

[ZeikoFr]

Thanks for your script it is reallly usefull.

Hope we could get this feature integrated in Nextcloud in the future

[szaimen]

There is an integration into nextcloud now, afaics.
https://github.com/bpcurse/nextcloud-userexport#nextcloud-integration

Also I'd say the feature described in this issues sounds like a good idea for a dedicated app. Feel free to post it in our "App ideas" section in the forum, to find a developer that wants to work on it:
https://help.nextcloud.com/c/apps/app-ideas/21

Cheers

[ahcm]

Btw. just in case someone has a benefit I use:
occ user:list -i | yq -c '.[]| [.[][1,2][], .[][5].groups[][]]|@csv' | sed 's/\"/"/g;s/^"//;s/"$//'

[Lionel-cl]

Btw. just in case someone has a benefit I use: occ user:list -i | yq -c '.[]| [.[][1,2][], .[][5].groups[][]]|@csv' | sed 's/"/"/g;s/^"//;s/"$//'

It looks great but it doesn't work with me.
Error: unknown shorthand flag: 'c' in -c
I am using YQ version 4.16.2
Can you confirm your expression?

[ocroquette]

The "Users¨ page is really not great to get an overview about users, their groups, etc. Looking for a way to export the data so that I can render it in a better way, I found this issue. Unfortunately, there is still no export function in the web interface, but if your use case is just better readability, and you have shell access to the server, you can use:

occ user:list --info
occ group:list

The output of user:list looks like YAML, but actually it is not. If you need to parse it to render it differently , the following Python script might be helpful as a basis:

#!/usr/bin/env python3
#
# occ-users-overview.py
#
# Parse the output of occ user:list and generate a human-readable overview
# of the users and the groups they belong to.
#
# Usage:
#   sudo -u www-data php occ user:list --info | occ-users-overview.py
#
# See also: https://github.com/nextcloud/server/issues/14715
#

import yaml
import sys
import re
import string

# The output format of user:list --info looks like YAML, but it is not YAML.
# 
#  - username:
#    - user_id: username
#    - display_name: Display Name
#    - email: ...
#    - cloud_id: ...
#    - enabled: true
#    - groups:
#      - 0: ...
#      - 1: ...
#    - quota: 0 B
#    - last_seen: 1970-01-01T00:00:00+00:00
#    - user_directory: ...
#    - backend: ...
#
# The following transformation make it useable for a YAML parser:

input_string = sys.stdin.read()
input_string = re.sub(r"^  - ", "", input_string, flags=re.MULTILINE)
input_string = re.sub(r"^    - ", "  ", input_string, flags=re.MULTILINE)
input_string = re.sub(r"^      - \d+: ", "  - ", input_string, flags=re.MULTILINE)

full_data = yaml.safe_load(input_string)

print("\nGroups of each user:\n")

for user_name in sorted(full_data.keys()):
    print(user_name)
    for group_name in full_data[user_name]["groups"]:
        print("  - ", group_name)