Prepare the 'php72' branch

[scara]

PHP 7.2 is now RC5: time to prepare a branch/image for it.

Will look for the extensions required here, spare time permitted.

[scara]

Note: PHP 7.2 (now still rc, php:7.2-rc-apache) is based on Debian 9 i.e. follow microsoft/msphpsql#484 for workaround and final solution (no ETA).

[andrewnicols]

Latest update is:

No updates but it is in the plan to support it in the next RTW.

When that lands, I'll prepare the 72 branch.

[stronk7]

Ref: solr & php72: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224078

[stronk7]

With solr disabled because of the link above... I've ended with this one, all databases working ok so far (have run unit tests in mysql, pgsql, mariadb, oracle and sqlsrv).

Surely not final... but hope it will save you some time (finding libraries and other details). In fact it has helped to find https://tracker.moodle.org/browse/MDL-61240.

Just running a few more tests...

[stronk7]

TODO: Can switch to newer instaclient 12.2... (running 12.1.. now). But don't want to get distracted...

[stronk7]

(and got it passing after updating oci to 12.2 and setting library path properly)

[scara]

Great! If this will be merged:

• should it be a separate 7.2 branch i.e. still defaulting to 7.1?
• should a new issue about solr be filed to track that missing?

TIA,
Matteo

[stronk7]

should it be a separate 7.2 branch i.e. still defaulting to 7.1?

In this repo (in all in general), I'm against using the 😱 "latest" tag (de facto "default" one). So I'd keep current behavior of having to, always, specify a tag.
And I imagine that.. until solr is fixed and 7.2 works with more moodle branches... the effective default for moodle-docker compositions, can continue being 7.1 without any problem. 👍
So, summary... no changes are needed to achieve that and 7.1 will continue being picked by moodle-docker.
Only point it that I'd suggest to push this both to master (dev docker tag) and 7.2 (7.2 docker tag) branches so it becomes the base for next branches / fixes.

should a new issue about solr be filed to track that missing?

+💯

[scara]

Ref: solr & php72: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224078

That's strange: Remi repo for CentOS has that extension available since months, https://rpms.remirepo.net/enterprise/7/php72/x86_64/repoview/php-pecl-solr2.html.

I found just a mention about Stretch long time ago, oerdnj/deb.sury.org#664 (comment): what about dropping a new issue there to know where we actually are in Debian 9 for solr and PHP 7.2?

Edit: it is there as a package, https://packages.sury.org/php/dists/stretch/main/binary-amd64/Packages:

Package: php-solr
Version: 2.4.0-5+0~20170908051553.2+stretch~1.gbpd2bc7d
Architecture: amd64
Maintainer: Debian PHP PECL Maintainers <pkg-php-pecl@lists.alioth.debian.org>
Installed-Size: 1271
Depends: php-curl, php-common (>= 1:7.0+33~), phpapi-20170718 | phpapi-20160303 | phpapi-20151012 | phpapi-20131226, libc6 (>= 2.14), libcurl3 (>= 7.16.2), libxml2 (>= 2.7.4)
Provides: php5.6-solr, php7.0-solr, php7.1-solr, php7.2-solr
Homepage: http://pecl.php.net/package/solr
Priority: optional
Section: php
Filename: pool/main/p/php-solr/php-solr_2.4.0-5+0~20170908051553.2+stretch~1.gbpd2bc7d_amd64.deb
Size: 170352
SHA256: bdbde09baa6ed2669d1307c71a54b0a05bff436ae90e2cafbbf27dcf13ca3617
SHA1: 4b38caf5c85ccad488ab573eda56cd92457fca99

Nothing has been filed into https://github.com/docker-library/php - found the fix by Remi: https://git.php.net/?p=pecl/search_engine/solr.git;a=commitdiff;h=744e32915d5989101267ed2c84a407c582dc6f31 (php/pecl-search_engine-solr@744e329) - and if I give it a try on an plain 7.2-apache container instance it breaks on something that looks like already fixed by Remi but not in the main repo, https://bugs.php.net/bug.php?id=75631.

Edit^2: added a proposal in stronk7@aaf7d05#r27038188, to enable solr and then track the availability of a new official pecl version in a separate issue, here.

HTH,
Matteo

[stronk7]

Great findings, Matteo!

As commented @ the commit... 100% welcome a new .sh for adding manually compiled solr till there is a pecl one with the needed fixes. +1 !

Tell me where to pick it from (or if you want me to try it), let's do some tests and, if everything is working... surely we can convert this into a proper PR a let it advance.

Thanks, ciao :-)

[stronk7]

(have created #19 about to monitor the availability of a proper php-solr extension for PHP 7.2)

[scara]

Hi @stronk7,
here is the extra commit, scara@e985b11: it passes the test, https://travis-ci.org/scara/moodle-php-apache/builds/333494629, and I'm going to launch the Moodle unit tests.

Edit, here is the result quite interesting since we need to disable redis since MDL-60951 and maybe working at solr related code as done w/ sqlsrv in MDL-61240:

# export MOODLE_DOCKER_WWWROOT=<path_to_moodle_master_branch>
# export MOODLE_DOCKER_DB=pgsql
# export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=true                                             
# bin/moodle-docker-compose up -d
# bin/moodle-docker-wait-for-db
# bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/init.php
# bin/moodle-docker-compose exec -T webserver vendor/bin/phpunit
PHPUnit test environment setup complete.
[root@vm-docker01 moodle-docker]# bin/moodle-docker-compose exec -T webserver vendor/bin/phpunit
Moodle 3.5dev (Build: 20180125), a4f914b54dbafcaf1cc2bf1cce8bde30cc69db57
Php: 7.2.1, pgsql: 9.6.6, OS: Linux 3.10.0-693.11.6.el7.x86_64 x86_64
PHPUnit 6.4.4 by Sebastian Bergmann and contributors.
...
..........................................                    8765 / 8765 (100%)

Time: 13.92 minutes, Memory: 385.00MB

There were 15 errors:

1) core_session_redis_testcase::test_normal_session_start_stop_works
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:95
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

2) core_session_redis_testcase::test_session_blocks_with_existing_session
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:111
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

3) core_session_redis_testcase::test_session_is_destroyed_when_it_does_not_exist
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:144
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

4) core_session_redis_testcase::test_session_is_destroyed_when_we_have_it_open
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:152
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

5) core_session_redis_testcase::test_multiple_sessions_do_not_interfere_with_each_other
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:162
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

6) core_session_redis_testcase::test_multiple_sessions_work_with_a_single_instance
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:204
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

7) core_session_redis_testcase::test_session_exists_returns_valid_values
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:224
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

8) core_session_redis_testcase::test_kill_sessions_removes_the_session_from_redis
session_set_save_handler(): Cannot change save handler when headers already sent

/var/www/html/lib/classes/session/redis.php:155
/var/www/html/lib/tests/session_redis_test.php:240
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit core_session_redis_testcase lib/tests/session_redis_test.php

9) search_solr_engine_testcase::test_search with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/engine/solr/tests/engine_test.php:207
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

10) search_solr_engine_testcase::test_delete with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/engine/solr/tests/engine_test.php:285
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

11) search_solr_engine_testcase::test_alloweduserid with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/engine/solr/tests/engine_test.php:332
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

12) search_solr_engine_testcase::test_highlight with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/engine/solr/tests/engine_test.php:381
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

13) search_solr_engine_testcase::test_solr_filling with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/engine/solr/tests/engine_test.php:649
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

14) search_solr_engine_testcase::test_get_query_total_count with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/engine/solr/tests/engine_test.php:710
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

15) search_solr_engine_testcase::test_manager_paged_search with data set "file-indexing-off" (0)
count(): Parameter must be an array or an object that implements Countable

/var/www/html/search/engine/solr/classes/engine.php:246
/var/www/html/search/engine/solr/classes/engine.php:158
/var/www/html/search/classes/manager.php:695
/var/www/html/search/classes/manager.php:599
/var/www/html/search/engine/solr/tests/engine_test.php:744
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

--

There were 10 failures:

1) search_solr_engine_testcase::test_search with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:208
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

2) search_solr_engine_testcase::test_delete with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:285
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

3) search_solr_engine_testcase::test_alloweduserid with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:333
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

4) search_solr_engine_testcase::test_highlight with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:382
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

5) search_solr_engine_testcase::test_reindexing_files
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:488
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

6) search_solr_engine_testcase::test_index_filtered_file
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:571
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

7) search_solr_engine_testcase::test_solr_filling with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 75.

/var/www/html/search/engine/solr/tests/engine_test.php:650
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

8) search_solr_engine_testcase::test_get_query_total_count with data set "file-indexing-on" (1)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:711
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

9) search_solr_engine_testcase::test_manager_paged_search with data set "file-indexing-on" (1)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:745
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

10) search_solr_engine_testcase::test_context_restriction
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    0 => 'C1F1'
-    1 => 'C1F2'
-    2 => 'C2F'
-    3 => 'Course 1'
-    4 => 'Course 2'

/var/www/html/search/engine/solr/tests/engine_test.php:846
/var/www/html/search/engine/solr/tests/engine_test.php:794
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

ERRORS!
Tests: 8765, Assertions: 2596717, Errors: 15, Failures: 10, Skipped: 15.

HTH,
Matteo

[scara]

Pinging @dmonllao, to kindly ask for a check about the above errors being an issue in compiling solr vs an issue in the Moodle code.

Guessing the first i.e. my commit should not be added: @stronk7 please do not just install the redis extension commenting it out w/ a ref to the related Tracker issue.

HTH,
Matteo

[dmonllao]

Hi @scara, I will look at it with great pleasure. I am guessing that it is because of http://php.net/manual/en/migration72.incompatible.php#migration72.incompatible.warn-on-non-countable-types and response->docs not being countable. I am verifying its type now.

[dmonllao]

Confirmed that they are caused by a problem in moodle code: https://tracker.moodle.org/browse/MDL-61281 (up for peer review)

[scara]

Hi @dmonllao,
TNX a ton for your prompt reply 👍 😄 !

Will run the tests again this weekend and comment in the Tracker.

[scara]

Hi @stronk7 and @dmonllao,
updated Moodle code w/ the commit from MDL-61281, then added the following two commits on top of Eloy's proposal:

• scara@439d3ae, to disable the redis extension
• scara@b040a8a, to re-enable solr, installing it via source code

Here is the result of the run:

# bin/moodle-docker-compose exec -T webserver vendor/bin/phpunit
Moodle 3.5dev (Build: 20180125), 0f08178e6d0a3af3035f882a3d62e874e11e9258
Php: 7.2.1, pgsql: 9.6.6, OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64
PHPUnit 6.4.4 by Sebastian Bergmann and contributors.

.............................................................   61 / 8765 (  0%)
...
..........FFFFFFFF..FF.FFFFFFF............................... 8357 / 8765 ( 95%)
............................................................. 8418 / 8765 ( 96%)
............................................................. 8479 / 8765 ( 96%)
............................................................. 8540 / 8765 ( 97%)
............................................................. 8601 / 8765 ( 98%)
............................................................. 8662 / 8765 ( 98%)
............................................................. 8723 / 8765 ( 99%)
..........................................                    8765 / 8765 (100%)

Time: 13.18 minutes, Memory: 385.00MB

There were 17 failures:

1) search_solr_engine_testcase::test_search with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:208
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

2) search_solr_engine_testcase::test_search with data set "file-indexing-off" (0)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:208
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

3) search_solr_engine_testcase::test_delete with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:285
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

4) search_solr_engine_testcase::test_delete with data set "file-indexing-off" (0)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:285
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

5) search_solr_engine_testcase::test_alloweduserid with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:333
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

6) search_solr_engine_testcase::test_alloweduserid with data set "file-indexing-off" (0)
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/search/engine/solr/tests/engine_test.php:333
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

7) search_solr_engine_testcase::test_highlight with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:382
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

8) search_solr_engine_testcase::test_highlight with data set "file-indexing-off" (0)
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:382
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

9) search_solr_engine_testcase::test_reindexing_files
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:488
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

10) search_solr_engine_testcase::test_index_filtered_file
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/search/engine/solr/tests/engine_test.php:571
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

11) search_solr_engine_testcase::test_solr_filling with data set "file-indexing-on" (1)
Failed asserting that actual size 0 matches expected size 75.

/var/www/html/search/engine/solr/tests/engine_test.php:650
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

12) search_solr_engine_testcase::test_solr_filling with data set "file-indexing-off" (0)
Failed asserting that actual size 0 matches expected size 75.

/var/www/html/search/engine/solr/tests/engine_test.php:650
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

13) search_solr_engine_testcase::test_get_query_total_count with data set "file-indexing-on" (1)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:711
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

14) search_solr_engine_testcase::test_get_query_total_count with data set "file-indexing-off" (0)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:711
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

15) search_solr_engine_testcase::test_manager_paged_search with data set "file-indexing-on" (1)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:745
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

16) search_solr_engine_testcase::test_manager_paged_search with data set "file-indexing-off" (0)
Failed asserting that 0 matches expected 30.

/var/www/html/search/engine/solr/tests/engine_test.php:745
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

17) search_solr_engine_testcase::test_context_restriction
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    0 => 'C1F1'
-    1 => 'C1F2'
-    2 => 'C2F'
-    3 => 'Course 1'
-    4 => 'Course 2'

/var/www/html/search/engine/solr/tests/engine_test.php:846
/var/www/html/search/engine/solr/tests/engine_test.php:794
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:
 vendor/bin/phpunit search_solr_engine_testcase search/engine/solr/tests/engine_test.php

FAILURES!
Tests: 8765, Assertions: 2596623, Failures: 17, Skipped: 28.

No more errors from PHP 7.2 and Countable as expected, but same failures which I need to explore if due to compiling solr from scratch or due to PHP 7.2 and some changes in the response.

At the end:

• @stronk7 please pick scara@439d3ae and I think that even @andrewnicols will be fine with the 7.2 initial support
• @dmonllao will you plan to check for these failures too?

TIA,
Matteo

[scara]

@dmonllao for the record, quickly debugging that test case, selecting just one failed test:

bin/moodle-docker-compose exec webserver php vendor/bin/phpunit --filter test_search search_solr_engine_testcase search/engine/solr/tests/engine_test.php

show me that the response coming from the search engine is actually empty i.e. w/ no matching.
While I'm seeing many hits in the log of the solr container I'm still wondering if my way of compiling the solr extension is "wrong".
I'll try:

1. to make test to validate the binary being compiled
2. to create a debug container where the PHP setup is not coming from "source" but from "supported" - i.e. working - packages, including the solr extension

HTH,
Matteo

[dmonllao]

Hi Matteo, I didn't compile it myself. search_solr php unit runs with MDL-61281 patch in are passing for me.

`➜ master git:(MDL-61281_master) ✗ vendor/bin/phpunit -c search
Moodle 3.5dev (Build: 20180125), 93b226723942a471d75e98e3424a9cf2929f9544
Php: 7.2.1.1.16.04.1.1, pgsql: 9.5.10, OS: Linux 4.4.0-112-generic x86_64
PHPUnit 6.4.4 by Sebastian Bergmann and contributors.

............................................................... 63 / 63 (100%)

Time: 52.74 seconds, Memory: 82.50MB

OK (63 tests, 504 assertions)
`

[scara]

TNX for your superfast feedback 👍.

I already filed an issue into Docker PHP about solr extension and its outdated v2.4.0 (docker-library/php#571).
Will look at a way to get solr into Moodle PHP 7.2 docker toolbox 😉.

[stronk7]

Well, this is what I've got (1st attempt!):

$ bin/moodle-docker-compose exec webserver vendor/bin/phpunit search_solr_engine_testcase
Moodle 3.8dev (Build: 20190520), a866c5c23866fe527282429384ddbcb902c41c14
Php: 7.3.5, pgsql: 9.6.7, OS: Linux 4.9.125-linuxkit x86_64
PHPUnit 7.5.9 by Sebastian Bergmann and contributors.

.............................                                     29 / 29 (100%)

Time: 30.53 seconds, Memory: 71.00 MB

OK (29 tests, 235 assertions)

And what I did (manually for now, within the 7.3 container, committing it, was (exactly what macports does):

1. Download the old 2.4.0 tarball.
2. Patch exactly with the very-same macports patchfile.
3. phpize, configure, make (needed to apt-get install curl, but that's ok, make install)
4. create the solr.ini

And moodle tests passed pretty well!

I must confess that I also ran solr tests (make test) and got a number of failures... but haven't looked if they were because cannot access to any solr instance or what.

So the question here is... no matter it's using an old codebase and patch, and some own extension tests are failing... as far as it makes moodle happy (and really works, I've it enabled in my dev sites and use it a number of times)... should we go for it (until the problem is fixed upstream and we process it at #19 ) ?

I feel myself inclined to go for it, and have it working ASAP. Later we can look if there are newer combinations also working but, for now, I'd consider this (imperfect but) good enough.

Thoughts?

[stronk7]

Just have seen this, from April: http://git.php.net/?p=pecl/search_engine/solr.git;a=commitdiff;h=98a8bf540bcb4e9b2e1378cce2f3a9bf6cd772b8

Going to give it a run...

[scara]

Hi @stronk7,
great news! 🎉

I feel myself inclined to go for it, and have it working ASAP. Later we can look if there are newer combinations also working but, for now, I'd consider this (imperfect but) good enough.

+1 for me 👍

For the record: the patch you used matches php/pecl-search_engine-solr@744e329 which is about ten commits before the commit I used for my first tests... I still wonder why I had those failures given the strange HTTP Response pattern.
BTW a tech struggle among many others of mine, for my curiosity sake 😉.

I must confess that I also ran solr tests (make test) and got a number of failures... but haven't looked if they were because cannot access to any solr instance or what.

They are both unit and integration tests.
You need to use its own solr instance, omars/solr53, and then configure the Host and Port to access that instance via https://github.com/php/pecl-search_engine-solr/blob/50397192abc8860388244cf4270b0ac8fc8d94ff/tests/test.config.inc#L17.
At the time of my runs via make test, I had one failure that should be somewhat addressed now.

Going to give it a run...

🤞 😉

Matteo

P.S.: time for me to buy a Mac? Looking for a sponsor 😆

[stronk7]

Just to confirm that using current master (php/pecl-search_engine-solr@98a8bf5), it's a total disaster:

Moodle 3.8dev (Build: 20190520), a866c5c23866fe527282429384ddbcb902c41c14
Php: 7.3.5, pgsql: 9.6.7, OS: Linux 4.9.125-linuxkit x86_64
PHPUnit 7.5.9 by Sebastian Bergmann and contributors.

...FFFFFFFF..FF.FFFFFFFFF..FF                                     29 / 29 (100%)

Time: 23.31 seconds, Memory: 64.50 MB

There were 21 failures:
....
....

Not sure how it can be so bad, it's crazy!

[stronk7]

Ok, so what if:

• we use 2.4.0 + macports patch to build a working version.
• we apply it to the 72 and 73 images.
• we wait till they release something and verify if can move to it (Monitor PECL's php-solr releases towards standard php 7.2/7.3 support #19).

Don't think it's worth the effort looking at the details when there is at very least one combination working. I really hate mysteries, because they usually hide some interesting truth to learn from (maybe we are doing something wrong in our engine implementation?), but right now I'd prioritize to have them working. Then investigations can continue (here or @ #19).

I'm going to prepare a patch for 72 and 73 (stretch only). Will link once available... feel free to comment anything...ciao :-)

[stronk7]

They are both unit and integration tests.
You need to use its own solr instance

Yeah, I saw that there were request failures and friends and imagined that I was missing something like that (a server instance). But was so impatient to verify if the combination was going to work that didn't look further, heh.

And yes, as said above, agree with you... it doesn't make much sense that they have broken it so badly... so maybe there is some config/setting on connection or whatever that we are missing and cause all the surprising failures... so we should keep investigating, yeah.

[scara]

TNX @stronk7 !

because they usually hide some interesting truth to learn from

Great: you take my point 😉!

Matteo

[stronk7]

BTW, I just tried a bisect session here, starting from php/pecl-search_engine-solr@744e329 just to discover that the upstream patch doesn't lead to a working (for Moodle) version, grrr. So whatever killed it, was between 2.4.0 and that patch.

Maybe later will continue the bisect section to see where the hell things became broken (maybe that will give us a clue of the reasons).

Back to the 2.4.0 + patch preparations...

[stronk7]

Ok, have created #57 (php73) and #58 (php72) to get solr working there. And, in #19 we'll do any next research about it, where did it became broken and so on.

So, right now, in order to have the images 100% ok... we are still missing:

• mongodb (logstore): MDL-54592 - was reimplemented for Moodle 3.7 so we'll have to enable the extension and, conditionally, add the configuration in config.php.template (coz it continues not working in older Moodle versions).
• analytics python backend: Requirements for mlbackend_python tests missing #9 - we need to analyze if that's the way... also what happens with other stuff we may want to be available (say git, say nvm/nodejs...)

Tomorrow more, have got enough of docker for today :-)

[stronk7]

Ok, created #60 , #61 , #62 and moodlehq/moodle-docker#102.

With them we can consider this 100% achieved but #9 (analytics python backend) that can be handled apart.

Once everything is in place... will run some tests to verify that there aren't other leftovers...

Ciao :-)

[stronk7]

If nobody is against, I'm going to close this. Only remaining issues (standard solr #19 and mlbackend #9, #35) already have their place and the 72/73 images are being used since some months ago.... 3... 2... 1...

[scara]

+1

[stronk7]

Sold, thanks all! Let's continue in related issues!