Deployment getting stuck #333

axeloz · 2017-03-08T18:10:45Z

Before submitting your issue, please make sure that you've checked all of the checkboxes below.

You're running the latest release version of Deployer.
Ensure that you're running at least PHP 5.6.4, you can check this by running php -v
You've ran composer install --no-dev from the root of your installation.
You've ran npm install --production from the root of your installation.
You've ran rm -rf bootstrap/cache/* from the root of your installation.
You have restarted the queue listener and node socket server.

Expected behaviour

The deployer should run properly

Actual behaviour

The deployer seems to run VERY slowly, so slowly that I don't know if it still runs or is stuck.
I have been waiting very long (1h or more). It is getting stuck on the "npm install" task.

This project was working yesterday and the days before. I have upgraded Node in the meantime. I have deleted the node_modules folder and run npm install --production again.

Steps to reproduce

No idea

Environment info

Operating System: Ubuntu 14
PHP Version: 7.1
Node Version: 7.7.1
Database System: MySQL
Database Version: 5.5.50-38.0

Logs (see storage/logs/) or other output that would be helpful

(If logs are large, please upload as attachment).

The text was updated successfully, but these errors were encountered:

axeloz · 2017-03-09T09:03:13Z

Here is some log I found into the failed jobs table.

Illuminate\Queue\MaxAttemptsExceededException: A queued job has been attempted too many times. The job may have previously timed out. in /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:357
Stack trace:
#0 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(285): Illuminate\Queue\Worker->markJobAsFailedIfAlreadyExceedsMaxAttempts('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), 1)
#1 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(258): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), Object(Illuminate\Queue\WorkerOptions))
#2 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(110): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\BeanstalkdJob), 'beanstalkd', Object(Illuminate\Queue\WorkerOptions))
#3 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(100): Illuminate\Queue\Worker->daemon('beanstalkd', 'deployer-high,d...', Object(Illuminate\Queue\WorkerOptions))
#4 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(83): Illuminate\Queue\Console\WorkCommand->runWorker('beanstalkd', 'deployer-high,d...')
#5 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#6 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(28): call_user_func_array(Array, Array)
#7 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Support/helpers.php(912): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#8 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(86): value(Object(Closure))
#9 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(30): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#10 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/Container.php(524): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#11 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Illuminate\Container\Container->call(Array)
#12 /var/www/deployer/vendor/symfony/console/Command/Command.php(262): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#13 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(167): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#14 /var/www/deployer/vendor/symfony/console/Application.php(826): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/deployer/vendor/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/deployer/vendor/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(123): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/deployer/artisan(45): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 {main}

12h later, the deployment is still running. I thought there were a timeout after which the task is cancelled.
Thanks again

axeloz · 2017-03-09T10:43:33Z

Doing more testing, I reverted back to NodeJS 5.8.0.
I ran a deployment on another project which always worked.
It's getting stuck too. I have a failed job :

+-----+------------+---------------+-----------------------------------------+---------------------+
| ID  | Connection | Queue         | Class                                   | Failed At           |
+-----+------------+---------------+-----------------------------------------+---------------------+
| 218 | beanstalkd | deployer-high | REBELinBLUE\Deployer\Jobs\DeployProject | 2017-03-09 11:32:10 |
+-----+------------+---------------+-----------------------------------------+---------------------+

The task he's getting stuck on is basically a gulp build command. If I run the same command directly in the release folder on the server with the same unix user, it works just fine.

If you have any idea, it would be great.
Thanks

axeloz · 2017-03-09T11:30:41Z

More debug log, hope it helps:

[2017-03-09 12:28:21] production.DEBUG: ### Run script over SSH
ssh -o CheckHostIP=no \
    -o IdentitiesOnly=yes \
    -o StrictHostKeyChecking=no \
    -o PasswordAuthentication=no \
    -o IdentityFile=/var/www/deployer/storage/app/tmp/keyPOPdrK \
    -p 22 deploy@127.0.0.1 'bash -s' << 'EOF'
        # Turn on quit on non-zero exit
        set -e
        ### Custom script - 215
cd /var/www/maison/www/releases/20170309122725/user/themes/maison
npm=`which npm`
$npm install
EOF

[2017-03-09 12:28:35] production.ERROR: Pheanstalk\Exception\ServerException: Server reported NOT_FOUND in /var/www/deployer/vendor/pda/pheanstalk/src/YamlRe
sponseParser.php:36
Stack trace:
#0 /var/www/deployer/vendor/pda/pheanstalk/src/Connection.php(142): Pheanstalk\YamlResponseParser->parseResponse('NOT_FOUND', NULL)
#1 /var/www/deployer/vendor/pda/pheanstalk/src/Pheanstalk.php(396): Pheanstalk\Connection->dispatchCommand(Object(Pheanstalk\Command\StatsJobCommand))
#2 /var/www/deployer/vendor/pda/pheanstalk/src/Pheanstalk.php(312): Pheanstalk\Pheanstalk->_dispatch(Object(Pheanstalk\Command\StatsJobCommand))
#3 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Jobs/BeanstalkdJob.php(91): Pheanstalk\Pheanstalk->statsJob(Object(Pheanstalk\Job))
#4 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(377): Illuminate\Queue\Jobs\BeanstalkdJob->attempts()
#5 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(321): Illuminate\Queue\Worker->markJobAsFailedIfWillExceedMaxAttempts('beanstal
kd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), 1, Object(Illuminate\Queue\MaxAttemptsExceededException))
#6 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(295): Illuminate\Queue\Worker->handleJobException('beanstalkd', Object(Illumina
te\Queue\Jobs\BeanstalkdJob), Object(Illuminate\Queue\WorkerOptions), Object(Illuminate\Queue\MaxAttemptsExceededException))

REBELinBLUE · 2017-03-10T14:05:49Z

"A queued job has been attempted too many times" hmm, what is your supervisor config setup? It should only try a job once

REBELinBLUE · 2017-03-10T14:07:55Z

Oh actually looking at the call to markJobAsFailedIfAlreadyExceedsMaxAttempts in the log it has only tried once. So, are you saying it has timed out but hasn't marked it as failed?

You can clean it up with php artisan deployer:cleanup, I will investigate further

axeloz · 2017-03-10T14:16:54Z

Thanks @REBELinBLUE, I have cleaned up many times and retried in different conditions.

On the main project's page, I can see that the deployment is "Failed".
On the deployment details page, I can see the first 2 steps are "Completed" (green) and the 3rd is still "Running" (orange), 24h hours later. All the steps after are "Cancelled" (red).

REBELinBLUE · 2017-03-10T14:27:37Z

hmm, OK I'll have to look, guess I have broken

deployer/app/Console/Commands/ClearStalledDeployment.php

Lines 59 to 84 in 70779a6

    
               /** 
        
                * Cleans up any stalled deployments in the database. 
        
                * 
        
                * @todo Maybe readd pending to the queue if possible? 
        
                */ 
        
               public function cleanupDeployments() 
        
               { 
        
                   // Mark any pending steps as cancelled 
        
                   ServerLog::where('status', '=', ServerLog::PENDING) 
        
                            ->update(['status' => ServerLog::CANCELLED]); 
        
                   // Mark any running steps as failed 
        
                   ServerLog::where('status', '=', ServerLog::RUNNING) 
        
                            ->update(['status' => ServerLog::FAILED]); 
        
                   // Mark any running/pending deployments as failed 
        
                   Deployment::whereIn('status', [Deployment::DEPLOYING, Deployment::PENDING]) 
        
                             ->update(['status' => Deployment::FAILED]); 
        
                   // Mark any aborting deployments as aborted 
        
                   Deployment::whereIn('status', [Deployment::ABORTING]) 
        
                             ->update(['status' => Deployment::ABORTED]); 
        
                   // Mark any deploying/pending projects as failed 
        
                   Project::whereIn('status', [Project::DEPLOYING, Project::PENDING]) 
        
                          ->update(['status' => Project::FAILED]);

with the refactoring https://github.com/REBELinBLUE/deployer/blob/master/app/Console/Commands/ClearStalledDeployment.php#L103-L120

REBELinBLUE · 2017-03-10T18:34:55Z

hmm, can you stop the queue listener/supervisor, run the cleanup command, run php artisan queue:flush and then start supervisor again and see if that marks the "running" stage as failed. Can't find any reason it wouldn't mark it as failed so wondering if the process has hanged so keeps restarting it

REBELinBLUE · 2017-03-10T18:35:58Z

Which version of node & npm is on the server you are trying to deploy to?

axeloz · 2017-03-13T10:40:52Z

Hello @REBELinBLUE I did clean the queue many times, restarted supervisor and beanstakld but nothing did it.
npm : 3.8.3
nodejs : 5.10.1 but tried with 5.8.0 and 7.7.1 (switching using "n")

Still stuck I don't get it . Thanks

REBELinBLUE · 2017-03-13T21:47:54Z

I'm still trying to recreate this, you can fix the one which is marked as "running" by editing the status column in the serverlogs table to 3

something like (assuming you don't have any currently running deployments

UPDATE server logs SET status = 3 WHERE status = 2;

REBELinBLUE · 2017-03-13T21:48:48Z

Are you sure your npm install command isn't prompting for input? What exactly is the command you have configured for that step

axeloz · 2017-03-14T09:52:59Z

Hello,

The exact command is:

cd {{ release_path }}/user/themes/maison
npm=`which npm`
$npm install

I don't know whether there is a prompt or not. The deployer doesn't display the output button. Nothing in the server_log table either (NULL).

Running the same command manually works of course...

REBELinBLUE · 2017-03-14T10:18:54Z

Can you post your packages.json?

axeloz · 2017-03-14T10:43:42Z

Sure

{
  "name": "maison",
  "version": "1.0.0",
  "description": "The **Maison** Theme is for [Grav CMS](http://github.com/getgrav/grav).  This README.md file should be modified to describe the features, installation, configuration, and general usage of this theme.",
  "main": "gulpfile.js",
  "dependencies": {},
  "devDependencies": {
    "get-folder-size": "^1.0.0",
    "gulp": "^3.9.1",
    "gulp-image-resize": "^0.12.0",
    "gulp-imagemin": "3.1.1",
    "gulp-rename": "^1.2.2",
    "gulp-sass": "^3.1.0",
    "gulp-sourcemaps": "^2.4.0",
    "gulp-util": "^3.0.8",
    "gulp-watch": "^4.3.11"
  },
  "author": "",
  "license": "ISC"
}

axeloz · 2017-03-15T16:05:35Z

I'm getting really confused now : I got rid of the NPM install step and it's getting stuck sooner now, on the composer install:

Loading composer repositories with package information
Installing dependencies from lock file
Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. Run update to update them.
  - Installing antoligy/dom-string-iterators (v1.0.0)
    Loading from cache

  - Installing doctrine/cache (v1.6.1)
    Loading from cache

  - Installing donatj/phpuseragentparser (v0.6.0)
    Loading from cache

  - Installing erusev/parsedown (dev-master 4b64939)
    Loading from cache

  - Installing erusev/parsedown-extra (0.7.1)
    Loading from cache

  - Installing filp/whoops (2.1.5)
    Loading from cache

  - Installing gregwar/cache (v1.0.12)
    Loading from cache

  - Installing gregwar/image (dev-master 72568cf)
    Loading from cache

  - Installing seld/cli-prompt (1.0.2)
    Loading from cache

  - Installing league/climate (3.2.1)
    Loading from cache

  - Installing matthiasmullie/path-converter (1.1.0)
    Loading from cache

  - Installing matthiasmullie/minify (1.3.43)
    Loading from cache

  - Installing symfony/polyfill-mbstring (v1.3.0)
    Loading from cache

  - Installing symfony/var-dumper (v2.8.17)
    Loading from cache

  - Installing psr/log (1.0.2)
    Loading from cache

  - Installing maximebf/debugbar (1.13.1)
    Loading from cache

  - Installing monolog/monolog (1.22.0)
    Loading from cache

  - Installing symfony/yaml (v2.8.17)
    Loading from cache

  - Installing symfony/event-dispatcher (v2.8.17)
    Loading from cache

  - Installing pimple/pimple (v3.0.2)
    Loading from cache

  - Installing rockettheme/toolbox (1.3.3)
    Loading from cache

  - Installing symfony/debug (v3.0.9)
    Loading from cache

  - Installing symfony/console (v2.8.17)
    Loading from cache

  - Installing symfony/polyfill-iconv (v1.3.0)
    Loading from cache

  - Installing twig/twig (v1.31.0)

It's been 20min, still stuck on Twig.

axeloz · 2017-03-15T16:12:51Z

Confirmed into the failed_jobs table :

Illuminate\Queue\MaxAttemptsExceededException: A queued job has been attempted too many times. The job may have previously timed out. in /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:357

@REBELinBLUE what should I do? I don't understand why everything fails now. It used to work really fine before.

REBELinBLUE · 2017-03-15T16:19:15Z

argh, what the actual hell! Looking at the composer output, could you try loading a url via CURL on the server

Something like curl --verbose https://packagist.org/packages/list.json

I notice all the packages it installs are loaded from cache so maybe it is getting stuck when it needs to download something (which could also happen with npm as well)

axeloz · 2017-03-15T16:21:59Z

Thanks. CURL did download the JSON file, no problem here.

REBELinBLUE · 2017-03-15T16:32:56Z

Could you try editing resources/scripts/deploy/steps/InstallComposerDependencies.sh adding --verbose to the command, restart the queue listener and try another deploy

REBELinBLUE · 2017-03-26T14:14:29Z

Any luck?

axeloz · 2017-03-27T08:58:00Z

Hello @REBELinBLUE, my bad, I did not get your last message so I didn't try this.
Unfortunately, I could not remain stuck that long at work without a deployer so I developed my own headless deployer last week using Laravel.

REBELinBLUE · 2017-03-27T13:43:20Z

Sorry to hear that, I just really don't know what is happening without more output.

Only other thing I could think is that it is related to #327 but there would be an error in the log about the job being too big

roshangautam · 2017-03-27T18:54:58Z

I have the exact same issue.

[2017-03-27 13:50:20] production.ERROR: exception 'Pheanstalk\Exception\ServerException' with message 'Server reported NOT_FOUND' in /var/www/deployer/vendor/pda/pheanstalk/src/YamlResponseParser.php:36
Stack trace:
#0 /var/www/deployer/vendor/pda/pheanstalk/src/Connection.php(142): Pheanstalk\YamlResponseParser->parseResponse('NOT_FOUND', NULL)
#1 /var/www/deployer/vendor/pda/pheanstalk/src/Pheanstalk.php(396): Pheanstalk\Connection->dispatchCommand(Object(Pheanstalk\Command\StatsJobCommand))
#2 /var/www/deployer/vendor/pda/pheanstalk/src/Pheanstalk.php(312): Pheanstalk\Pheanstalk->_dispatch(Object(Pheanstalk\Command\StatsJobCommand))
#3 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Jobs/BeanstalkdJob.php(91): Pheanstalk\Pheanstalk->statsJob(Object(Pheanstalk\Job))
#4 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(377): Illuminate\Queue\Jobs\BeanstalkdJob->attempts()
#5 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(322): Illuminate\Queue\Worker->markJobAsFailedIfWillExceedMaxAttempts('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), 1, Object(Illuminate\Queue\MaxAttemptsExceededException))
#6 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(295): Illuminate\Queue\Worker->handleJobException('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), Object(Illuminate\Queue\WorkerOptions), Object(Illuminate\Queue\MaxAttemptsExceededException))
#7 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(258): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), Object(Illuminate\Queue\WorkerOptions))
#8 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(110): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\BeanstalkdJob), 'beanstalkd', Object(Illuminate\Queue\WorkerOptions))
#9 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(101): Illuminate\Queue\Worker->daemon('beanstalkd', 'deployer-high,d...', Object(Illuminate\Queue\WorkerOptions))
#10 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(85): Illuminate\Queue\Console\WorkCommand->runWorker('beanstalkd', 'deployer-high,d...')
#11 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#12 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(29): call_user_func_array(Array, Array)
#13 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Support/helpers.php(912): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#14 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(86): value(Object(Closure))
#15 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(30): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#16 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/Container.php(524): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#17 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Illuminate\Container\Container->call(Array)
#18 /var/www/deployer/vendor/symfony/console/Command/Command.php(262): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#19 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(168): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#20 /var/www/deployer/vendor/symfony/console/Application.php(826): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/deployer/vendor/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/deployer/vendor/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(123): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 /var/www/deployer/artisan(46): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 {main}

And jobs failing with the following error message

exception 'Illuminate\Queue\MaxAttemptsExceededException' with message 'A queued job has been attempted too many times. The job may have previously timed out.' in /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:357
Stack trace:
#0 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(286): Illuminate\Queue\Worker->markJobAsFailedIfAlreadyExceedsMaxAttempts('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), 1)
#1 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(258): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), Object(Illuminate\Queue\WorkerOptions))
#2 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(110): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\BeanstalkdJob), 'beanstalkd', Object(Illuminate\Queue\WorkerOptions))
#3 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(101): Illuminate\Queue\Worker->daemon('beanstalkd', 'deployer-high,d...', Object(Illuminate\Queue\WorkerOptions))
#4 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(85): Illuminate\Queue\Console\WorkCommand->runWorker('beanstalkd', 'deployer-high,d...')
#5 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#6 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(29): call_user_func_array(Array, Array)
#7 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Support/helpers.php(912): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#8 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(86): value(Object(Closure))
#9 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(30): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#10 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Container/Container.php(524): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#11 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Illuminate\Container\Container->call(Array)
#12 /var/www/deployer/vendor/symfony/console/Command/Command.php(262): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#13 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Console/Command.php(168): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#14 /var/www/deployer/vendor/symfony/console/Application.php(826): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/deployer/vendor/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/deployer/vendor/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/deployer/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(123): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/deployer/artisan(46): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 {main}

This started happening since I added following commands before activating a new release

cd {{ release_path }}
npm install
php artisan queue:restart

Other projects without npm are working fine.

REBELinBLUE · 2017-03-27T19:54:29Z

Could you try switching to master and see if that fixes it (running composer install --no-dev and npm install --production and php artisan app:update as if you were installing a new version)?

roshangautam · 2017-03-28T13:33:48Z

The new version is working fine. Thanks @REBELinBLUE

REBELinBLUE · 2017-03-28T20:08:28Z

great to hear. Interesting that it was causing different output in the log for you (and presumably it was the same thing for @axeloz), everyone else it was giving a message saying JOB_TOO_BIG which is why I didn't think it was the same issue 🙈

axeloz · 2017-03-29T09:31:25Z

Oops, well ... My mistake, I have been too impatient :)
My own deployer works well, I did spend quite some time on it, I have not the courage to throw it away.
:)

REBELinBLUE · 2017-03-29T19:15:41Z

Ha ha, no problem of course, when you need something to work you need it to work. Surprised Laravel doesn't catch the error to be honest, may have to look at seeing if I can catch it myself as there is no reason it should cause a fatal error.

Silly thing it, it only came up because I fixed a bug. Previously if you were viewing the server log and it changed you didn't see the change without closing the window even though you were supposed to, I think the broadcasting so that correctly sent the change but in the process the log often ended up being bigger than the default limit for a beanstalkd job and it appears my dev server has it set higher than the default so it didn't happen for me.

REBELinBLUE added the Waiting: Needs Feedback label Mar 10, 2017

REBELinBLUE added Can't Reproduce Hide from CHANGELOG and removed Waiting: Needs Feedback labels Mar 27, 2017

REBELinBLUE closed this as completed Mar 27, 2017

REBELinBLUE added the Waiting: Needs Feedback label Mar 27, 2017

REBELinBLUE reopened this Mar 27, 2017

REBELinBLUE closed this as completed Mar 28, 2017

dgpro mentioned this issue Jul 18, 2017

Deployments timing out on PHP 7.1 #348

Closed

6 tasks

kamioftea mentioned this issue Apr 27, 2018

NPM install causes deploy to cancel with exception 'Pheanstalk\Exception\ServerException' with message 'Server reported NOT_FOUND' #372

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deployment getting stuck #333

Deployment getting stuck #333

axeloz commented Mar 8, 2017

axeloz commented Mar 9, 2017

axeloz commented Mar 9, 2017

axeloz commented Mar 9, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

axeloz commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

axeloz commented Mar 13, 2017

REBELinBLUE commented Mar 13, 2017

REBELinBLUE commented Mar 13, 2017

axeloz commented Mar 14, 2017 •

edited

Loading

REBELinBLUE commented Mar 14, 2017

axeloz commented Mar 14, 2017

axeloz commented Mar 15, 2017 •

edited

Loading

axeloz commented Mar 15, 2017

REBELinBLUE commented Mar 15, 2017

axeloz commented Mar 15, 2017

REBELinBLUE commented Mar 15, 2017

REBELinBLUE commented Mar 26, 2017

axeloz commented Mar 27, 2017

REBELinBLUE commented Mar 27, 2017

roshangautam commented Mar 27, 2017

REBELinBLUE commented Mar 27, 2017

roshangautam commented Mar 28, 2017

REBELinBLUE commented Mar 28, 2017 •

edited

Loading

axeloz commented Mar 29, 2017

REBELinBLUE commented Mar 29, 2017

Deployment getting stuck #333

Deployment getting stuck #333

Comments

axeloz commented Mar 8, 2017

Expected behaviour

Actual behaviour

Steps to reproduce

Environment info

Logs (see storage/logs/) or other output that would be helpful

axeloz commented Mar 9, 2017

axeloz commented Mar 9, 2017

axeloz commented Mar 9, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

axeloz commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

REBELinBLUE commented Mar 10, 2017

axeloz commented Mar 13, 2017

REBELinBLUE commented Mar 13, 2017

REBELinBLUE commented Mar 13, 2017

axeloz commented Mar 14, 2017 • edited Loading

REBELinBLUE commented Mar 14, 2017

axeloz commented Mar 14, 2017

axeloz commented Mar 15, 2017 • edited Loading

axeloz commented Mar 15, 2017

REBELinBLUE commented Mar 15, 2017

axeloz commented Mar 15, 2017

REBELinBLUE commented Mar 15, 2017

REBELinBLUE commented Mar 26, 2017

axeloz commented Mar 27, 2017

REBELinBLUE commented Mar 27, 2017

roshangautam commented Mar 27, 2017

REBELinBLUE commented Mar 27, 2017

roshangautam commented Mar 28, 2017

REBELinBLUE commented Mar 28, 2017 • edited Loading

axeloz commented Mar 29, 2017

REBELinBLUE commented Mar 29, 2017

axeloz commented Mar 14, 2017 •

edited

Loading

axeloz commented Mar 15, 2017 •

edited

Loading

REBELinBLUE commented Mar 28, 2017 •

edited

Loading