Test Results for 2.0.1 release

[phil-davis]

Test Plan Password policy

Areas to test

1. web UI

1.1 admin creates a user
1.2 admin changes password of a user
1.3 user changes own password in personal user settings
1.4 user sets own password on lost password page
1.5 user creates a public share (this is using the share API, so no need to test by itself if the share API is tested)

2. User Provisioning API

2.1 users / adduser
2.2 users / edituser

3. share API

3.1 create a new share
3.2 update share

4. occ

4.1 user:add
4.2 user:resetpassword

5. guests app

[phil-davis]

Testing functionality

0 No special password requirements / no link expiration requirements

These tests make sure the app does not block anything if all its settings are disabled
Test Setup:
disable all requirements checks

Test Case	Steps	Expected Result	Results	Comments
very simple passwords work	set "aaa" as password	The intended action is implemented	1.1 ✅ 1.2 ✅ 1.3 ✅ 1.4 ✅ 2.1 ✅ 2.2 ✅ 3.1 ✅ 3.2 ✅ 4.1 ✅ 4.2 ✅
public links without expire date & password work	Create/update a public link with a password and no expire date	Link is created/updated	3.1 ✅ 3.2 ✅
public links without expire date & no password work	Create/update a public link without a password and no expire date	Link is created/updated	3.1 ✅ 3.2 ✅
public links with an expire date far in the future & password work	Create/update a public link with a password and the expire date set to today +1 year	Link is created/updated	3.1 ✅ 3.2 ✅
public links with an expire date far in the future & no password work	Create/update a public link without a password and the expire date set to today +1 year	Link is created/updated	3.1 ✅ 3.2 ✅

[phil-davis]

1 Minimum characters required

Test Setup:

1. enable "minimum characters" check
2. set "minimum characters" requirement to "10"

Test Case	Steps	Expected Result	Results	Comments
Minimum characters required, too short password set	set a password of less than 8 chars	User is notified that the password isn't long enough, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum characters required, long enough password set	set a password of 8 chars	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Enforcement of password protection for shares overrides zero characters requirement	1. set "minimum characters" requirement to "0" 2. enable "Enforce password protection" in sharing settings 3. As user share a folder with an empty password.	Password is not set and link is not created	1.5 ✅
Passwords with a length of 0 are not possible	1. set "minimum characters" requirement to "0" 2. set an empty password.	The intended action is not implemented	1.1 ✅ 1.2 ✅ 1.3 ✅ 1.4 ✅ 2.1 ✅ 2.2 ✅ 4.1 ✅ 4.2 ✅
letters not allowed in "minimum characters" field	set "minimum characters" requirement to "a"	Hint is shown, saying that a number has to be entered	✅
negative numbers are not allowed in "minimum characters" field	set "minimum characters" requirement to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
"0" is not allowed in "minimum characters" field	set "minimum characters" requirement to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
empty "minimum characters" field is not allowed	clear "minimum characters" requirements field	Hint is shown, saying that a number has to be entered	✅	Fake saving, refreshing the page a zero appears

[phil-davis]

2 Lowercase letters required

Test Setup:

1. enable "lowercase letters" check
2. set "lowercase letters" requirement to "3"

Test Case	Steps	Expected Result	Results	Comments
Minimum amount of lowercase letters required, given password does not contain enough lowercase letters	set a password containing less than 3 lowercase letters	User is notified that the password does not contain enough lowercase letters, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum amount of lowercase letters required, given password contains enough lowercase letters	set a password containing 3 lowercase letters	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
letters not allowed in "lowercase letters" field	set "lowercase letters" requirement to "a"	Hint is shown, saying that a number has to be entered	✅
negative numbers are not allowed in "lowercase letters" field	set "lowercase letters" requirement to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
"0" is not allowed in "lowercase letters" field	set "lowercase letters" requirement to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
empty "lowercase letters" field is not allowed	clear "lowercase letters" field	Hint is shown, saying that a number has to be entered	✅	Fake saving, refreshing the page a zero appears

3 Uppercase letters required

Test Setup:

1. enable "uppercase letters" check
2. set "uppercase letters" requirement to "3"

Test Case	Steps	Expected Result	Results	Comments
Minimum amount of uppercase letters required, given password does not contain enough uppercase letters	set a password containing less than 3 uppercase letters	User is notified that the password does not contain enough uppercase letters, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum amount of uppercase letters required, given password contains enough uppercase letters	set a password containing 3 uppercase letters	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
letters not allowed in "uppercase letters" field	set "uppercase letters" requirement to "a"	Hint is shown, saying that a number has to be entered	✅
negative numbers are not allowed in "uppercase letters" field	set "uppercase letters" requirement to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
"0" is not allowed in "uppercase letters" field	set "uppercase letters" requirement to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
empty "uppercase letters" field is not allowed	clear "uppercase letters" field	Hint is shown, saying that a number has to be entered	✅	Fake saving, refreshing the page a zero appears

[phil-davis]

4 Numerals required

Test Setup:

1. enable "numbers" check
2. set "numbers" requirement to "3"

Test Case	Steps	Expected Result	Results	Comments
Minimum amount of numerals required, given password does not contain enough numerals	set a password containing less than 3 numerals	User is notified that the password does not contain enough numbers, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum amount of numerals required, given password contains enough numerals	set a password containing 3 numerals	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
letters not allowed in "numbers" field	set "numbers" requirement to "a"	Hint is shown, saying that a number has to be entered	✅
negative numbers are not allowed in "numbers" field	set "numbers" requirement to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
"0" is not allowed in "numbers" field	set "numbers" requirement to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
empty "numbers" field is not allowed	clear "numbers" field	Hint is shown, saying that a number has to be entered	✅	Fake saving, refreshing the page a zero appears

[phil-davis]

5 Special characters required

Test Setup:

1. enable "special characters" check
2. set "special characters" requirement to "3"

Test Case	Steps	Expected Result	Results	Comments
Minimum amount of special characters required, given password does not contain enough special characters	set a password containing less than 3 special characters	User is notified that the password does not contain enough special characters, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum amount of special characters required, given password contains enough special characters	set a password containing 3 special characters	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
letters not allowed in "special characters" field	set "special characters" requirement to "a"	Hint is shown, saying that a number has to be entered	✅
empty "special characters" field is not allowed	clear "special characters" field	Hint is shown, saying that a number has to be entered	✅
"0" is not allowed in "special characters" field	set "special characters" requirement to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
negative numbers are not allowed in "special characters" field	set "special characters" requirement to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
Minimum amount of special characters required, only given special characters are allowed, given password contains forbidden special characters	1. enable "Define special characters" checkbox 2. Define "^$/" as special characters. 3. set a password not containing 3 of the special characters, e.g. "^$#"	User is notified that the password contains invalid special characters, setting the password is refused. The intended action is not implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Minimum amount of special characters required, only given special characters are allowed, given password contains one of the allowed special characters	1. enable "Define special characters" checkbox 2. Define "^$/" as special characters. 3. set a password containing the 3 special characters, e.g. "$^$"	Password is set to the correct value. The intended action is implemented	1.1 🤖 1.2 🤖 1.3 🤖 1.4 🤖 2.1 🤖 2.2 🤖 3.1 🤖 3.2 🤖 4.1 🤖 4.2 🤖
Emojis in the list of allowed special characters	1. Place an emoji in the list of allowed special characters 2. use that emoji in a password	Password is set to the correct value. The intended action is implemented	1.1 ✅ 1.2 ✅ 1.3 ✅ 1.4 ✅ 2.1 ✅ 2.2 ✅ 3.1 ✅ 3.2 ✅ 4.1 ✅ 4.2 ✅	See issue #141 for discussion of character counts for multi-byte Unicode characters

Note: issue #41 is still outstanding. You can check "Restrict to these special characters" and leave the list empty. Effectively the "check" makes no difference.

[phil-davis]

6. Public Link expiration

Test Case	Steps	Expected Result	Results	Comments
empty "days until link expires if password is set" field is not allowed	1. Enable "days until link expires if password is set" check. 2. clear "days until link expires if password is set" field	Hint is shown, saying that a number has to be entered	❌	Issue #41
"0" is not allowed in "days until link expires if password is set" field	1. Enable "days until link expires if password is set" check. 2. set "days until link expires if password is set" field to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
negative numbers are not allowed in "days until link expires if password is set" field	1. Enable "days until link expires if password is set" check. 2. set "days until link expires if password is set" field to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
empty "days until link expires if password is not set" field is not allowed	1. Enable "days until link expires if password is not set" check. 2. clear "days until link expires if password is not set" field	Hint is shown, saying that a number has to be entered	❌	Issue #41
"0" is not allowed in "days until link expires if password is not set" field	1. Enable "days until link expires if password is not set" check. 2. set "days until link expires if password is not set" field to "0"	Hint is shown saying that values less than 1 are not allowed	❌	Issue #41
negative numbers are not allowed in "days until link expires if password is not set" field	1. Enable "days until link expires if password is not set" check. 2. set "days until link expires if password is not set" field to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
Limit max. link expiration if password is set. Try to create a link without expire date.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and no expire date	Notification is shown, saying that the expiration date is required. Link is not created	1.5 ✅
Limit max. link expiration if password is set. Try to create a link with invalid date.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and expire date of today+7 days	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not created	1.5 ✅
Limit max. link expiration if password is set. Try to create a link with a date inside of max. expire date	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and expire date of today+4 days	Link is created	1.5 ✅
Limit max. link expiration if password is not set. Try to create a link without expire date.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and no expire date	Notification is shown, saying that the expiration date is required. Link is not created	1.5 ✅
Limit max. link expiration if password is not set. Try to create a link with invalid date.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and expire date of today+7 days	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not created	1.5 ✅
Limit max. link expiration if password is not set. Try to create a link with a date inside of max. expire date	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and expire date of today+4 days	Link is created	1.5 ✅
Limit max. link expiration if password is set. Delete the expire date of an existing link with password.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and expire date of today+4 days 4. Edit the newly created public link and delete the expire date	Notification is shown, saying that the expiration date is required. Link is not updated	1.5 ✅
Limit max. link expiration if password is set. Change the expire date of an existing link with password to an invalid date.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and expire date of today+4 days 4. Edit the newly created public link and set the expire date to today+7 days	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not updated	1.5 ✅
Limit max. link expiration if password is set. Change the expire date of an existing link with password to an valid date.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link with a password and expire date of today+4 days 4. Edit the newly created public link and set the expire date to today+5 days	Link is updated	1.5 ✅
Limit max. link expiration if password is not set. Delete the expire date of an existing link without a password.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and expire date of today+4 days 4. Edit the newly created public link and delete the expire date	Notification is shown, saying that the expiration date is required. Link is not updated	1.5 ✅
Limit max. link expiration if password is not set. Change the expire date of an existing link without a password to an invalid date.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and expire date of today+4 days 4. Edit the newly created public link and set the expire date to today+7 days	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not updated	1.5 ✅
Limit max. link expiration if password is not set. Change the expire date of an existing link without a password to an valid date.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link without a password and expire date of today+4 days 4. Edit the newly created public link and set the expire date to today+5 days	Link is updated	1.5 ✅
Limit max. link expiration if password is not set. Remove the password of an existing link without an expire date.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link with a password and no expire date 4. Edit the newly created public link and remove the password	Notification is shown, saying that the expiration date is required. Link is not updated	1.5 ✅
Limit max. link expiration if password is not set. Remove the password of an existing link with an expire date too far in the future.	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link with a password and a expire date of today+14 days 4. Edit the newly created public link and remove the password	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not updated	1.5 ✅
Limit max. link expiration if password is not set. Remove the password of an existing link with an expire date within the limits of "link expiration if password is not set"	1. Enable "days until link expires if password is not set" check. 2. Set "days until link expires if password is not set" to "5" 3. Create a public link with a password and a expire date of today+4 days 4. Edit the newly created public link and remove the password	Link is updated	1.5 ✅
Limit max. link expiration if password is set. Add a password to an existing link without an expire date.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link without a password and no expire date 4. Edit the newly created public link and set a password	Notification is shown, saying that the expiration date is required. Link is not updated	1.5 ✅
Limit max. link expiration if password is set. Add a password to an existing link with an expire date too far in the future.	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link without a password and a expire date of today+14 days 4. Edit the newly created public link and add a password	Notification is shown, saying that the expiration date cannot exceed 5 days. Link is not updated	1.5 ✅
Limit max. link expiration if password is set. Add a password to an existing link with an expire date within the limits of "link expiration if password is set"	1. Enable "days until link expires if password is set" check. 2. Set "days until link expires if password is set" to "5" 3. Create a public link without a password and a expire date of today+4 days 4. Edit the newly created public link and add a password	Link is updated	1.5 ✅

[phil-davis]

7. User Password policies

Test Case	Steps	Expected Result	Results	Comments
empty "days until user password expires" field is not allowed	1. Enable "days until user password expires" check. 2. clear "days until user password expires" field	Hint is shown, saying that a number has to be entered	🚫	Issue #41
"0" is not allowed in "days until user password expires" field	1. Enable "days until user password expires" check. 2. set "days until user password expires" field to "0"	Hint is shown saying that values less than 1 are not allowed	🚫	Issue #41
negative numbers are not allowed in "days until user password expires" field	1. Enable "days until user password expires" check. 2. set "days until user password expires" field to "-1"	Hint is shown saying that values less than 1 are not allowed	✅
User Password expires after the days set passed	1. Enable "days until user password expires" check. 2. set "days until user password expires" field to "3" 3. Modify server's date to +4 days	Check that the password has expired	✅	e.g. sudo date --set="2018-10-02 11:30"

[phil-davis]

8. Force change on first login

Test Case	Steps	Expected Result	Results	Comments
Password changing is mandatory after the 1st login	1. Enable "Force change on first login" check. 2. Log in with a user for the 1st time	Check you are requested to change the password	✅	details
Password changing is no longer mandatory after disabling the option	1. Disable "Force change on first login" check. 2. Log in with a user for the 1st time	Check you are no longer requested to change the password	✅	details

[phil-davis]

9 Number of last passwords that should not be used

Test Case	Steps	Expected Result	Results	Comments
Password cannot be used again within the last 3 password changes	1. Enable "last passwords should not be used" check. 2. set "last passwords should not be used" field to "3" 3. create a new user with password "aaa" 4. Log in as the new user and go to the settings page 5. change password "aaa" to "bbb" 6. change password "bbb" to "ccc" 7. change password "ccc" to "ddd" 8. change password "ddd" to "ddd" 9. change password "ddd" to "ccc" 10. change password "ddd" to "bbb"	Password changes at steps 8, 9 and 10 are not accepted	✅
Password can be used again after 3 password changes	1. Enable "last passwords should not be used" check. 2. set "last passwords should not be used" field to "3" 3. create a new user with password "aaa" 4. Log in as the new user and go to the settings page 5. change password "aaa" to "bbb" 6. change password "bbb" to "ccc" 7. change password "ccc" to "ddd" 8. change password "ddd" to "aaa"	Password changes are all accepted.	✅

Issue #57 is fixed.

[phil-davis]

Using occ command to expire passwords.

1. Create groups xx yy and zz

2. Create users yy1 yy2 zz1 zz2 notingroup

3. Put yy1 yy2 into group yy

4. Put zz1 zz2 into group zz

5. Login as yy1 and logout

6. Login as zz1 and logout

7. Enable user password policy "3 days until password expires"

8. occ user:expire-password --uid=notingroup
   Login as notingroup - the password is expired.

9. occ user:expire-password --uid=notingroup
   Login as notingroup - the password is expired.
   Login as zz1 - the password is NOT expired.

10. occ user:expire-password --group=yy
    Login as yy1 - the password is expired.
    Login as yy2 - the password is expired.
    Login as zz1 - the password is NOT expired.
    Login as zz2 - the password is NOT expired.
    Login as notingroup - the password is NOT expired.

11. occ user:expire-password --group=yy --group=zz
    Login as yy1 - the password is expired.
    Login as yy2 - the password is expired.
    Login as zz1 - the password is expired.
    Login as zz2 - the password is expired.
    Login as notingroup - the password is NOT expired.

12. occ user:expire-password --group=xx --group=yy
    Login as yy1 - the password is expired.
    Login as yy2 - the password is expired.
    Login as zz1 - the password is NOT expired.
    Login as zz2 - the password is NOT expired.
    Login as notingroup - the password is NOT expired.

13. occ user:expire-password --all
    Login as yy1 - the password is expired.
    Login as yy2 - the password is expired.
    Login as zz1 - the password is expired.
    Login as zz2 - the password is expired.
    Login as notingroup - the password is expired.
    Login as admin - the password is expired.

The above passes with the code fixes in PR #144 - ref issue #143 for what was going slightly wrong.

That PR has been merged - good.

[phil-davis]

Issue #53

Password History is cleaned up:

• create a user, login, change the password a few times

mysql> select * from oc_user_password_history;

• see that there are entries for the user
• delete the user

mysql> select * from oc_user_password_history;

• see that there are NO entries for the user

Works.

Notifications are cleaned up:
this was a general notifications problem, fixed by owncloud/notifications#221
It is in the notifications app release for 10.0.10 and is tested there - because the issue applies to any app that has created pending notifications, not just password policy.

[phil-davis]

Issue #101

Failing unit tests on Oracle are now passing, which "proves" that the backend code works with Oracle.

@PVince81 is this enough?

[PVince81]

@phil-davis enough for Oracle, yes

[phil-davis]

I realised that we can easily run the current automated acceptance tests on Oracle - PR #145 demonstrates that it passes. So we know Oracle is OK at least for the parts automated so far.

(this is another advantage of automation - components can be switched in/out and automated tests can verify behaviour)

[phil-davis]

Tarball checks out OK