basic tests of the public facing HTTP interface. refs #16, #113, #342…

[mro]

…, https://github.com/mro/ShaarliOS/issues/1

Transferred from https://github.com/mro/Shaarli-API-test, could become a git module for version-independant compatibility tests.

[ArthurHoaro]

It's very good! But we can't disable PHP 5.3 since we still support it.
Maybe you could test php -v in run.sh and return 0 if the version is less than 5.4?

Ping @virtualtam for CI tools stuff.

[mro]

while I'm a big fan of conservative and long-term backward compatibility (as you know) – 5.3 had it's EOL years ago.

For how long do you think will shaarli continue to officially support 5.3?

P.S.: [ "PHP 5.3." != "$(php --version | head -n 1 | cut -c 1-8)" ] || { echo "PHP 5.3 doesn't bring a proper webserver to run the tests." && exit 0; }

[mro]

The last question goes to @ArthurHoaro, @nodiscc and @virtualtam

[ArthurHoaro]

Hem... there is actually a discussion #250 where we decided "OK! let's drop PHP 5.3 compatibility." and then merged a PR for PHP 5.3 compatibility. ¯_(ツ)_/¯

[mro]

I argue for unconditional tests.

[virtualtam]

Theoretically, I'd go towards dropping support for any PHP version that has reached its end-of-life period (currently 5.3 and 5.4).
Though, a lot of Shaarli users are stuck with older versions, due to their shared hosting featuring obsolete PHP versions sometimes missing essential extensions (php-gd, php-intl, etc.).

For now, the development efforts to support older PHP versions is minimal (mostly array declaration syntax) but might grow as times goes. When we'll start using more recent core features (e.g. namespaces) and third-party libraries for templating, testing and other purposes, we'll need to drop support to ageing platforms.

As far as tests are concerned, we should rather update the Travis matrix and start using PHPUnit's test suite features, to distinguish unitary and functional tests and run them separately. This will allow for a more selective test coverage (currently, 5.5, 5.6 and 7.x matter).

[ArthurHoaro]

See mro/Shaarli#2

[mro]

this PR ages. Mind https://github.com/mro/Shaarli-API-test/commit/9ddfc884ea45 due to changed HTTP GET parameters on failed logins in 52ccf0d.

[mro]

@ArthurHoaro is there anything that prevents a soon merge?

[virtualtam]

Once introduced, test suites tend to grow quite fast.

By experience, Bash scripts are just a pain in the arse to scale and maintain, as things quickly turn either tricky or unreadable (e.g. booleans, variable scope, string operations).

There's still a long way to go before having a clean codebase (see #324), I'd rather not have to maintain both legacy code and a blob of scripts (and so might other maintainers).

There are more suitable community frameworks dedicated to running browser test suites over web applications, such as:

• PHPUnit + HTTP client (e.g. Guzzle), which would integrate well as Shaarli is written in PHP, which most maintainers and contributors are at ease with
• Selenium, which would allow for more proper UI/UX in-browser testing, has a PHP driver for bindings, as well as basic PHPUnit support

[ArthurHoaro]

I agree with @virtualtam. While it's great to have your tests around, I'd rather not have to maintain them. I think functional tests will be more pertinent when most of the refactoring will be completed.

[mro]

While it's great to have your tests around, I'd rather not have to maintain them.

@ArthurHoaro thank you for your kind words, but this is quite a disappointment, especially given your initial reaction #536 (comment)

In essence this sounds to me like there's at this time no interest in an even minimal stable HTTP interface (check login state, post a link). This is IMO a disastrous signal to all 3rd party tool authors.

[nodiscc]

I'd rather not have to maintain both legacy code and a blob of scripts (and so might other maintainers).

👍 @mro if you want to maintain this test suite in a repo under the Shaarli org please let us know