`Prompt::fake()` incompatible with `PendingCommand` created by `$this->artisan()`?

[jesseleite]

Laravel Prompts Version

0.1.24

Laravel Version

11.19.0

PHP Version

8.3.7

Operating System & Version

macOS 14.2

Terminal Application

WezTerm

Description

Working on some tests for Statamic... I did a code dive and found your Prompt::fake() helper, which is genious! 🔥

However, I've noticed an issue where tests containing Prompt::fake() pass when run in isolation...

But they fail when the whole test case and/or suite is ran...

We're hoping to use Prompt::fake() in our Statamic core test suite, but I can't get it to play nicely with other tests where we use $this->artisan().

It seems that PendingCommand is leaking mock expectations into other tests which use Prompt::fake(). I'm not sure how to solve, or if there's a way to improve Prompt::fake() for packages who wish to use it? 🤔

I've created a repo with an example test case to make it easier to reproduce, if that helps!

Any thoughts? 🙏

Steps To Reproduce

• Clone https://github.com/jesseleite/laravel-prompts-fake-issue
• Run composer install
• Run phpunit
  • Notice that if you run each of the three sample tests in isolation, they all pass
  • But if you run the whole test case/suite, the tests with Prompt::fake() will fail

[jesseleite]

A coworker and I found the issue, which in hindsight I guess makes sense given that stack trace....

When $this->artisan() is run in a test, the ConfiguresPrompts trait sets fallbacks on all the different prompt types, storing those fallbacks via static property on the Prompt class, which leak between tests at runtime because of the static property.

As a hacky workaround, we were able to use late static binding to reset the $shouldFallback boolean, to trick Prompt::fake() from trying to use those fallbacks.

I'm open to creating a PR for laravel/prompts here, but not sure the best approach... For example, we could create a Prompt::resetFallbacks() for people to use in their own test suites? Or maybe we could prevent Prompt::fake() from ever trying to use fallbacks altogether?

Curious on your thoughts?

[jesseleite]

Hey @driesvints! Re: that needs more info label, is there something specific you are looking for?

You can see the actual PR where we're trying to use Prompt::fake() mentioned above, but I've distilled what we're doing into a more grokkable gist here.

Happy to help with a PR, just curious on @jessarcher's thoughts before I dig any further (this is NOT a rush by any means though!) ❤️

[jessarcher]

Hey @jesseleite!

Prompt::fake() never reached a point where I felt it was good enough for folks to use in their own test suites. It's essentially a low-level internal testing helper.

From a consuming app's POV, I don't know how useful it is to queue up fake keypresses and assert against rendered output. Generally, you just want to test the logic of your own command when a user chooses a specific response without worrying about which keys they pressed to get there.

At the moment, the recommended way to test a command that uses Prompts is to use Laravel's built-in test helpers. E.g.

// TestCommand.php

public function handle()
{
    $result = text('Enter some text');

    $this->info($result);
}

// TestCommandTest.php

$this->artisan('statamic:test-command')
    ->expectsQuestion('Enter some text', 'Test command successful!')
    ->expectsOutput('Test command successful!')
    ->assertExitCode(0);

I have wanted to build some better userland test helpers into Prompts for a while, but probably following a similar API to the above where you're really just testing that your command prompted the user and that it handles the response.

I'm open to ideas, though! And if you still feel like Prompt::fake() is useful outside of internal testing, then I'm open to making it easier to work with.

[jesseleite]

Yeah @jessarcher, I think you are absolutely right... Maybe I should be worrying about the expected questions/answers, rather than the exact keys they've pressed via Prompt::fake(). I appreciate your guidance here ❤️

[emmadesilva]

Hey @jesseleite!

Prompt::fake() never reached a point where I felt it was good enough for folks to use in their own test suites. It's essentially a low-level internal testing helper.

I have wanted to build some better userland test helpers into Prompts for a while, but probably following a similar API to the above where you're really just testing that your command prompted the user and that it handles the response.

I'm open to ideas, though! And if you still feel like Prompt::fake() is useful outside of internal testing, then I'm open to making it easier to work with.

Hi @jessarcher! Great work on this package. I'd love it if we could revisit opening up Prompt::fake() and other parts of the internal APIs for more userland features. For HydePHP (hydephp/develop#2062) we're tapping into some customizations and it would be really helpful for us to be able to properly fake keypresses.

For example, we have a custom feature to toggle all settings using a checkbox, by using event listeners. So it would be really great if we could test this logic. That's currently being blocked by the same problem in this issue: static::$shouldFallback being sticky and there no being a way to reset it.

A simple first step would be to either add a helper to reset the state, or remove the sticky logic and simply allow any assignments:

public static function fallbackWhen(bool $condition): void
{
    static::$shouldFallback = $condition;
}

I'm more than happy to discuss these things further if you want to talk about ideas and our use cases, and of course I'll gladly PR whatever I can if it's something that fits with your vision.