lang('Auth.errorPasswordEmpty') fails sometime

[kenjis]

From #71 (comment)

There was 1 failure:

1) Tests\Unit\PasswordsTest::testEmptyPassword
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'A Password is required.'
+'Auth.errorPasswordEmpty'

/home/runner/work/shield/shield/tests/Unit/PasswordsTest.php:22

https://github.com/codeigniter4/shield/runs/6222433930?check_suite_focus=true

Ref lonnieezell/myth-auth#494 (comment)

[MGatner]

While we're collecting evidence I will add that I see this occasionally across many repos that use the framework. Not sure if it is some testing behavior or a bug in the repo itself.

[kenjis]

I see this error every day lately.
But I have no clue. After the error occurs, the test passes normally and it never happens twice in a row.

[MGatner]

I know, it's a pain. It happens elsewhere so definitely not something specific to this repo.

Here's what I feel fairly certain about: the error only happens erratically because we run tests in random order, therefor the underlying issue has something to do with ordering of service interactions.

That said, because it is random and rare it is very hard to reproduce the scenario that caused the failure to troubleshoot. My hunch is that it is related to Services.

[kenjis]

executionOrder="random"!
I didn't know it.

[MGatner]

It's a good practice. With so many moving pieces it is easy to write a test that passes accidentally because of some "bleed over" from another test. Sometimes it can complicate things like debugging though!

We could disable the random order, but I doubt that would help fix the underlying issue. And long-term I would prefer to keep the tests random.

[kenjis]

I got the cause.

This test case shows the cause.

--- a/tests/Unit/PasswordsTest.php
+++ b/tests/Unit/PasswordsTest.php
@@ -6,6 +6,7 @@
 use CodeIgniter\Shield\Config\Auth as AuthConfig;
 use CodeIgniter\Shield\Entities\User;
 use CodeIgniter\Test\CIUnitTestCase;
+use Config\Services;
 
 /**
  * @internal
@@ -14,6 +15,9 @@ final class PasswordsTest extends CIUnitTestCase
 {
     public function testEmptyPassword()
     {
+        Services::reset();
+        d(Services::autoloader()->getNamespace());
+
         $passwords = new Passwords(new AuthConfig());
 
         $result = $passwords->check('', new User());

$ vendor/bin/phpunit tests/Unit/PasswordsTest.php b/tests/Unit/PasswordsTest.php
PHPUnit 9.5.20 #StandWithUkraine

Runtime:       PHP 8.0.18
Configuration: /Users/kenji/work/codeigniter/codeigniter-shield/phpunit.xml
Random Seed:   1653307776

F                                                                   1 / 1 (100%)
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Services::autoloader()->getNamespace()                                                                                                         │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
array (0) []
══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Called from <ROOT>/tests/Unit/PasswordsTest.php:19 [d()]


Time: 00:00.069, Memory: 12.00 MB

There was 1 failure:

1) Tests\Unit\PasswordsTest::testEmptyPassword
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'A Password is required.'
+'Auth.errorPasswordEmpty'

/Users/kenji/work/codeigniter/codeigniter-shield/tests/Unit/PasswordsTest.php:26

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

[kenjis]

@MGatner The random order is fine. But it requires all test methods to be independent, or with proper @depends.
At least all devs should know we are using random.

Services::reset() behavior seems to be for rare cases. I propose to change the default value.
codeigniter4/CodeIgniter4#6020