Weird errors after updating to latest terminatur version

[ibuildit]

When running drush pullsite sitename.dev or drush pd sitename.dev I get this crash report. Using the recommended versions of Vagrant and Virtualbox from the Kalastack project site.

The function terminus_api_backup_catalog exists via an included file so I can't understand why it can't find the function. How ever, when I tested to run the same commands as sudo, I don't get the error (?!?!?) but then the entire alias etc process breaks...

PHP Fatal error: Call to undefined function terminus_api_backup_catalog() in /usr/share/drush/commands/terminatur/vendor/pantheon-systems/terminus/terminus.drush.inc on line 2693
PHP Stack trace:
PHP 1. {main}() /usr/share/drush/drush.php:0
PHP 2. drush_main() /usr/share/drush/drush.php:16
PHP 3. _drush_bootstrap_and_dispatch() /usr/share/drush/drush.php:61
PHP 4. drush_dispatch() /usr/share/drush/drush.php:92
PHP 5. call_user_func_array() /usr/share/drush/includes/command.inc:175
PHP 6. drush_command() /usr/share/drush/includes/command.inc:0
PHP 7. _drush_invoke_hooks() /usr/share/drush/includes/command.inc:208
PHP 8. call_user_func_array() /usr/share/drush/includes/command.inc:334
PHP 9. drush_terminatur_data() /usr/share/drush/includes/command.inc:0
PHP 10. _terminatur_get_data_bucket() /usr/share/drush/commands/terminatur/terminatur.drush.inc:257
PHP 11. _terminatur_choose_data_bucket() /usr/share/drush/commands/terminatur/terminatur.database.inc:216
PHP 12. _terminatur_data_make_backup() /usr/share/drush/commands/terminatur/terminatur.database.inc:246
PHP 13. terminus_latest_bucket() /usr/share/drush/commands/terminatur/terminatur.database.inc:74
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Call to undefined function terminus_api_backup_catalog() in
/usr/share/drush/commands/terminatur/vendor/pantheon-systems/terminus/terminus.drush.inc, line
2693

[pirog]

My guess is that you actually have two versions of terminatur installed. One in ~/.drush and one in /usr/share/drush/commands. Can you confirm?

[ibuildit]

Yes, that's correct.

[pirog]

you may be getting this error because you have an outdated version of terminatur in ~/.drush. I would make sure you have it installed only once and that it is the latest version.

When you are done make sure you clear the drush cache (drush cache-clear drush). Then rerun drush pauth, drush ta and try again. If that doesnt work you can try manually clearing the cache by deleting ~/.drush/cache and then running pauth and ta before pullsite.

[ibuildit]

Thank you for your quick reply. I had indeed tried to install terminus and terminatur manually as I didn't know they were already in place. I destroyed the box and spinned it up again.

This time I got the same error again. How ever, after manually deleting the .drush/cache directory it started to work!

I tried the same thing twice btw, still had to manually trash the cache directory. You might want to look into that.

Excellent, thank you very much!!

[pirog]

We aren't sure if this is an upstream issue or not but we might be looking into it :) Thanks @ibuildit

[jdleonard]

I also encountered this error. In my case, I had Kalabox Beta 1 installed and overwrote it with Kalabox Beta 2 (dragged Beta 2 to my Applications directory). Removing .drush/cache solved the issue for me. I didn't rebuild the VM.

[pirog]

Weird... you may have run this before we updated kalastack with the new "purge" commands for terminatur.

Will keep this open for now.

[whastings]

I noticed that when I ran into this problem, I had to remove ~/.drush/cache, not just ~/.drush/cache/default, in order to fix it. I think our Kalastack code is just removing ~/.drush/cache/default. Unfortunately, if we start removing ~/.drush/cache, the user will have to reauthenticate with Pantheon, but I suppose that's a minor inconvenience.

[fluxsauce]

Can you remove from ~/.drush/cache all but the pantheon / pantheon-persist directories?

[thiagodemellobueno]

Also experiencing this, does seem that ~/.drush/cache works.

[pirog]

I'd really like to figure out a way to resolve this without removing the entire cache directory. Going to look more into how drush does all these things later today.

[whastings]

@pirog I've been looking into it today, and I'm thinking now that it's not a Drush cache issue. I think Terminus's include files aren't getting loaded because the function that loads them, terminus_bootstrap(), isn't getting called under certain circumstances (which I've yet to figure out how to replicate). Our Terminatur bootstrap function is always called from our drush commands via _terminatur_get_site(). In following the flow of our bootstrap routine, you can see that we run terminus_bootstrap() via _terminatur_get_session() from various places. But if you already have an ssh key loaded and aliases downloaded, you're probably missing any of the logic that triggers Terminus's Bootstrap, which results in the file with terminus_api_backup_catalog() not being loaded.

I think the reason cache clearing has worked temporarily is that it clears Terminatur's cache of which ssh keys work with Pantheon, causing it to invoke Terminus's bootstrap in querying for the list of keys.

My working theory is that to prevent this problem, we just need to add a call to _terminatur_get_session() to the top of _terminatur_get_data_bucket(), but I'd like to figure out how to reliably reproduce the bug before committing a fix. I'll continue my investigation.

[whastings]

@pirog, looks like my theory was correct. The latest commit should resolve this issue.