-
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HHVM errors from Travis ci #10220
Comments
When the HHVM php7 compatibility flag is on, the errors greatly increase. Currently tests die at about 12% Click here to see an example build... This is the error that seems to be killing the HHVM_with_PHP7_compatibility flag unitTests Methods with the same name as their class will not be constructors in a future version of PHP; Cache_Lite has a deprecated constructor: 1x
1x in JCacheStorageTest::casesGetInstance |
That should only be a deprecation notice and not a failure. That or HHVM has a behavior change compared to PHP 7. |
According to HHVM when the flag Seems HHVM with php7 mode may be far more strict about some of the PHP7 changes. There is a PR for Cache_Lite that will resolve the PHP7 deprecated constructor issue. |
At some point a decision about what HHVM LTS versions we want to test against will be needed. There is a feature request and a PR for Travis CI that will allow Trusty available HHVM LTS versions If/when that PR gets merged we will have the following options 3.3, 3.6, 3.9, 3.12, and when it's released in July hhvm 3.15. Currently, we only have the option to test against |
Allow HHVM LTS version specification #733 has been implemented and documented. we now have access to the following
note: hhvm is latest (not always an LTS version), and hhvm-nightly is current nightly dev of HHVM I suggest not testing against LTS versions older than 3.12since that is the first LTS with an option for php7 mode That would suggest a maximum HHVM build matrix option as follows - php: hhvm-3.15
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="no" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
- php: hhvm-3.15
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="yes" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
- php: hhvm
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="no" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
- php: hhvm
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="yes" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
- php: hhvm-nightly
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="no" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
- php: hhvm-nightly
sudo: true
dist: trusty
group: edge # until the next update
addons:
apt:
packages:
- mysql-server-5.6
- mysql-client-core-5.6
- mysql-client-5.6
services:
- mysql
- postgresql
- memcache
- memcached
- redis-server
env: HHVMPHP7="yes" INSTALL_APCU="no" INSTALL_APCU_BC_BETA="no" # Disabled items that currently do not work in travis-ci hhvm
allow_failures:
- php: hhvm-3.15
- php: hhvm
- php: hhvm-nightly
before_script:
- composer install
- if [[ $HHVMPHP7 == "yes" ]]; then echo hhvm.php7.all=1 >> /etc/hhvm/php.ini; fi This travis build matrix will build 6 versions of HHVM
|
HHVM postgresql driver issue the This missing function causes See test build on hhvm-3.12 with hhvm pgsql.so |
|
In reviewing the |
JDocumentHtmlTest::testEnsureMergeHeadDataReturnsThisObject Array to string conversion failure is due to an intentional difference in the HHVM
|
@mbabker In checking with HHVM on the I tried looking into what might need to be done with our PHPUnit tests to deal with the PHP errors so that the test will pass, but I'm more lost now than before on how to solve this testing issue. I also think that the PHPUnit converting notices to exceptions is also related to the php cache lite test failure on the notice Do you have any thoughts on how to deal with this? |
I believe #11565 fixes the memcache issues in HHVM. |
I'm not sure we are going to be able to get PHP unit and HHVM to work together. The owner/maintainer of PHPunit doesn't care about HHVM anymore and is likely to drop HHVM from their travis builds Maybe this is a dead road in the project now (=_=) |
sorry to ear that :( shall we remove the hhvm tests on travis them? |
(Stupid phone, closing things because of small buttons and big fingers) @andrepereiradasilva I'm trying to see if I can find out if PHP Unit is going to drop the possibility of HHVM for sure, or if they are just currently annoyed with the past state of the project. If PHP unit is definitely dropping support for HHVM then we need to figure out if we are going to find a different unit test option for HHVM, or if we are going to kill HHVM support development. IMO That's more of a PLT call |
yes, i agree |
I got access to the "alpha way to initiate a debug job via an API call on travis" and tried to figure out what was going on with the memory issue. After a few tests I had to "hack" the I have edited the list of tests that error or fail to reflect the existing issues that would need to be addressed for the unit tests to pass. |
I got acknowledgment from HHVM of the bug that was causing our "Fatal error: request has exceeded memory limit" issue. They have implemented a patch that should fix the issue in the next release (I assume 3.15.1) I'm still working with HHVM to get a fix to the HHVM-nightly packages so I can test the patch and verify that the memory issue caused by |
So the bug in HHVM causing the "Fatal error: request has exceeded memory limit" issue has been fixed in HHVM 3.16.0-dev (hhvm-nightly) unfortunately the patch got missed in the 3.15.1 LTS release. hopefully it gets included in the 3.15.2 LTS release. |
nice |
HHVM testing note with the complete test run: A number of the Errors are new and due to a newly introduced connection issue with Postgresql "PDOException: [112]: could not translate host name "localhost port=5432 dbname=joomla_ut" to address: Name or service not known" |
The memory issue is in part the size of the test suite, in part our own configurations and code structure. If you look at the last HHVM builds for staging and 4.0-dev (where about half the test suite has been dumped thanks to dropping code for the Framework stack and relying on its tests) the HHVM build runs in full on both but the memory runs out at the end of the cycle on staging but no such issue with 4.0. |
Maybe not a full solution.... the underlying error for the failure of At least we know a little more. |
Maybe adjust it to |
Here is the PR for the suggested change to the Postgresql driver #12359 |
Hey @photodude thanks for all your work on this. I've looked into this, too, and you already saw one of my fixes. The array_unique() issue is unfortunate, to say the least. I think the behavior of HHVM here is wrong, but anyway. Regarding the JDatabaseDriverPostgresqlTest::testReleaseTransactionSavepoint error, this seems like a bigger issue, since we don't seem to be using the Postgres functions correctly... I still think that keeping compatible to HHVM would be good. |
is fixed in PR #12013 |
@csthomas That's awesome glad to see the fix, I hope it's merged soon. |
The following error will not be fixed in HHVM as noted in this issue facebook/hhvm#1071 3) JUserHelperTest::testGetCryptedPassword
Password is hashed to crypt-blowfish without salt
Failed asserting that false is true.
/tests/unit/suites/libraries/joomla/user/JUserHelperTest.php:443 This is returning |
|
I additionally suggest that the |
I believe this HHVM issue facebook/hhvm#2060 is the cause of the failure in our |
Looks like we have PR's or have discovered HHVM bugs for all of the test issues and errors with the singular exception of the Anyone have suggestions on how to proceed with solving those? |
I made a small change to one of the JInstallerAdapterTest tests from Fatal error: Call to undefined method
PHPUnit_Framework_MockObject_InvocationMocker::getTableColumns() in
/libraries/joomla/table/table.php on line 241 |
@mbabker I don't know much about the Again sorry for my lack of experience and knowledge on autoloaders. |
I honestly have no clue why it was marked private, but ever since the auto On Tuesday, October 18, 2016, Walt Sorensen notifications@github.com
|
Thanks @mbabker my only assumption is it got marked private by habit since in php 4 there was a preference to mark methods private or protected with leading underscores. In this instance, it seems like marking this private is the completely wrong thing to do. (it's questionable why it even works in php facebook/hhvm#959 (comment) ) |
We now have solutions in proposed PRs, Merged PRs, or HHVM bug fixes for all known issues except the Once everything is merged, and the few errors remaining are fixed, it should be reasonable to get people attempting to live test on HHVM again (something I don't think people have done since J3.3 and J3.4 due to the errors we had to fix and the bugs in HHVM prior to HHVM 3.15.2) |
Biggest block to live testing with HHVM is now just the failures with the JInstallerAdapterTest |
The more I try to dig into the JInstallerAdapterTest failures the more this is looking like an issue with PHPUnit 4.x and HHVM. My best guess is there is an issue with the mocking of I'm at a bit of a loss on how to solve that issue. Best case scenario is this is only a phpunit/hhvm issue and the J3.7.0 nightly build for staging should be good to go for real world testing on HHVM 3.15.2 with mysql or hhvm 3.16.0-dev nightly with PostgreSQL. |
The fix for the JInstallerAdapter failure on hhvm is included in PR #12990 There are some new PDOMySql failures in the HHVM tests but that is due to a Travis configuration issue. I'm still waiting for Travis to fix that error since it's due to some change they did in the Trusty testing images. Once we merge PR #12990 and |
Not acceptable. Whether an API is used in core or not, deprecated or not, it should still function as expected until the day it's removed. Likewise, if the class has tests, those should continue to function to validate the behavior. |
Ok I'll close #12668 as not a valid solution to the testing issue. |
@photodude if everything here is finally resolved with PR #12990 and PR #12428 then it can be closed here? This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/10220. |
@brianteeman Just waiting for PR #12990 to reach RTC or merged. With that PR everything is finally resolved with the HHVM tests. |
PR was merged. Nice work @photodude Still i notice two things tht can be improved in hhvm tests:
But maybe this issue should be closed a new one open for that issues. |
@andrepereiradasilva, I'm not sure about the slow tests, there seems to be about 6 tests that are significantly slower than the others. As for the "invalid configuration directive" this seems to be just a notice from hhvm. There is an issue open on it with hhvm |
ok @photodude close this. i will open a new issue for the hhvm slow tests |
Closing as the HHVM tests are now passing with all the PRs having been merged. @andrepereiradasilva this is the open issue with hhvm on the "invalid configuration directive" notice facebook/hhvm#7279 |
This is a list of the tests which are known to be failing or have errors in the HHVM phpunit testing (allowed failures)
Unfortunately, due to an out memory limit error in Travis-ci I can only retrieve the names of the tests that errored or failed (test listed below)I was able to get some of the test failure details by filtering and only running one test at a time.
you can see the results of my first test here https://travis-ci.org/photodude/joomla-cms/jobs/127702652
There were
1190 errors:1) JInstallerAdapterTest::testCheckExistingExtensionForExistingExtensionUnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:103
2) JInstallerAdapterTest::testCheckExistingExtensionForExtensionThatDoesNotExist
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:148
3) JInstallerAdapterTest::testDiscoverInstall
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:371
4) JInstallerAdapterTest::testDiscoverInstallWithNoDescription
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:439
5) JInstallerAdapterTest::testInstall
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:827
6) JInstallerAdapterTest::testInstallOnUpdateRoute
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:901
7) JInstallerAdapterTest::testInstallAbortsWhenSetupUpdatesThrowsException
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:980
8) JInstallerAdapterTest::testInstallWithNoDescription
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:1052
9) JInstallerAdapterTest::testParseQueriesWithUpdateRouteAndParsingReturningTrueCallsParseSchemaUpdatesCorrectly
UnexpectedValueException: No columns found for #__extensions table
/libraries/joomla/table/table.php:245
/libraries/joomla/table/table.php:162
/libraries/joomla/table/extension.php:30
/tests/unit/suites/libraries/cms/installer/JInstallerAdapterTest.php:1330
See proposed PR #12990
10) JDocumentHtmlTest::testEnsureMergeHeadDataReturnsThisObjectArray to string conversion
/tests/unit/core/helper/helper.php:52
/libraries/joomla/document/html.php:222
/tests/unit/suites/libraries/joomla/document/html/JDocumentHTMLTest.php:134
See merged PR #12013
11) JDatabaseDriverPostgresqlTest::testReleaseTransactionSavepointpg_result_error_field() expects parameter 1 to be resource, boolean given
/home/travis/build/photodude/joomla-cms/tests/unit/core/helper/helper.php:52
/home/travis/build/photodude/joomla-cms/libraries/joomla/database/driver/postgresql.php:1513
/home/travis/build/photodude/joomla-cms/libraries/joomla/database/driver/postgresql.php:742
/home/travis/build/photodude/joomla-cms/libraries/joomla/database/driver/postgresql.php:1384
/home/travis/build/photodude/joomla-cms/tests/unit/suites/database/driver/postgresql/JDatabaseDriverPostgresqlTest.php:1205
See merged PR #12359
There were
32 failuresFatal error: Call to undefined function pg_set_error_verbosity() in /home/travis/build/photodude/joomla-cms/libraries/joomla/database/driver/postgresql.php on line 140fixed in the release of HHVM 3.15.21) JInstallerAdapterTest::testParseQueriesWithUpdateRouteAndParsingReturningFalseReturnsExceptionSee proposed PR #12990Expectation failed for method name is equal to string:parseSchemaUpdates when invoked 1 time(s).
Method was expected to be called 1 times, actually called 0 times.
2) JLoaderTest::testSetupWithoutPrefixesFailed asserting that true is false.
/tests/unit/suites/libraries/joomla/JLoaderTest.php:621
See facebook/hhvm#2060 for related issue to JLoaderTest
See PR #12478 for a merged internal solution
3) JUserHelperTest::testGetCryptedPasswordPassword is hashed to crypt-blowfish without salt
Failed asserting that false is true.
/tests/unit/suites/libraries/joomla/user/JUserHelperTest.php:443
See merged PR #12428 for a salted fix
At the conclusion of testing memory failsFixed in hhvm 3.16.0 and 3.15.2Fatal error: request has exceeded memory limit in /libraries/vendor/phpunit/php-timer/src/Timer.php on line 97
Invalid configuration directive notice
'JLanguageTest::testParse' This gives a notice but no failure or error
The text was updated successfully, but these errors were encountered: