Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keep releases keeps adding directories (named 1, 1.1, 1.1.2, 1.1.2.3) #1004

Closed
filipponeri opened this issue Feb 9, 2017 · 29 comments
Closed

Keep releases keeps adding directories (named 1, 1.1, 1.1.2, 1.1.2.3) #1004

filipponeri opened this issue Feb 9, 2017 · 29 comments
Labels
bug research

Comments

@filipponeri
Copy link

Q A
Issue Type Bug
Deployer Version 4.2
Local Machine OS Ubuntu 16.10
Remote Machine OS Ubuntu 16.10

Description

the set('keep_releases', 3); command does not appear to work.
Deployer keeps adding directories (with names and in this order: '1', '2' , '1.1' , '1.1.2' , '1.1.2.3' ) without deleting any previous release directories. This results in quickly filling up the disk space.

Apart for this problem, Deploy deploys correctly the new software on the remote web server and does not output any error.
Thank you for your help.
Best regards, Filippo

Running deploy

Steps to reproduce

dep deploy.php

Content of deploy.php


namespace Deployer;
//require_once 'recipe/common.php';
require 'recipe/laravel.php';

//use function Deployer\{server, task, run, set, get, add, before, after};

set('ssh_type', 'native');
set('ssh_multiplexing', true);

set('repository', 'git@gitlab.com:repository');

// Laravel writable dirs
add('shared_files', []);
add('writable_dirs', ['storage', 'vendor']);

server('staging', 'hosting', 22232)
    ->user('usename')
    ->password('pwd')
    ->set('branch', 'master')
    ->set('deploy_path', '/home/provrtsp/public_html/promarket11.com')
    ->pty(true);

set('keep_releases', 3);

//->identityFile('~/.ssh/id_rsa.pub', '~/.ssh/id_rsa')

/**
 * Setup the environment file in the new release
 */
task('environment', function () {
    run('cp /home/provrtsp/shared/env {{release_path}}/.env');
   
})->desc('Environment setup');

task('composer_rerun', function () {
    run('cd /home/provrtsp/public_html/promarket11.com/current; php ~/composer.phar install ');
})->desc('Environment setup');

// Laravel writable dirs
set('writable_dirs', ['storage', 'vendor']);

// to be added to task call below 'deploy:symlink',

/**
 * Main task
 */
task('deploy', [
    'deploy:prepare',
    'deploy:release',
    'deploy:update_code',
    'deploy:vendors',
    'deploy:symlink',
    'cleanup',
    'environment',
])->desc('Deploy your project');

after('deploy', 'success');
after('success', 'composer_rerun');

Output log

With enabled option for verbose output -vvv.

(this run creates the release directory named 1.1.2.3 )

$ dep deploy -vvv
➤ Executing task deploy:prepare
[staging] > echo $0
[staging] < bash
[staging] > if [ ! -d /home/provrtsp/public_html/promarket11.com ]; then mkdir -p /home/provrtsp/public_html/promarket11.com; fi
[staging] > if [ ! -L /home/provrtsp/public_html/promarket11.com/current ] && [ -d /home/provrtsp/public_html/promarket11.com/current ]; then echo true; fi
[staging] > cd /home/provrtsp/public_html/promarket11.com && if [ ! -d .dep ]; then mkdir .dep; fi
[staging] > cd /home/provrtsp/public_html/promarket11.com && if [ ! -d releases ]; then mkdir releases; fi
[staging] > cd /home/provrtsp/public_html/promarket11.com && if [ ! -d shared ]; then mkdir shared; fi
• done on [staging]
✔ Ok [5s 306ms]
➤ Executing task deploy:release
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -h release ]; then echo 'true'; fi)
[staging] > cd /home/provrtsp/public_html/promarket11.com && ([ -d releases ] && [ "$(ls -A releases)" ] && echo "true" || echo "false")
[staging] < true
[staging] > cd /home/provrtsp/public_html/promarket11.com && (cd releases && ls -t -d /)
[staging] < 1.1.2/ 1.1/ 2/ 1/
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -f .dep/releases ]; then echo "true"; fi)
[staging] < true
[staging] > cd /home/provrtsp/public_html/promarket11.com && (tail -n 11 .dep/releases)
[staging] < 20170209064323,1
[staging] < 20170209064710,2
[staging] < 20170209065719,1.1
[staging] < 20170209070156,1.1.2
[staging] < 20170209071015,1.1.2
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -d /home/provrtsp/public_html/promarket11.com/releases/1 ]; then echo 'true'; fi)
[staging] < true
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -d /home/provrtsp/public_html/promarket11.com/releases/1.1 ]; then echo 'true'; fi)
[staging] < true
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -d /home/provrtsp/public_html/promarket11.com/releases/1.1.2 ]; then echo 'true'; fi)
[staging] < true
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [ -d /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3 ]; then echo 'true'; fi)
[staging] > cd /home/provrtsp/public_html/promarket11.com && (date +"%Y%m%d%H%M%S")
[staging] < 20170209071307
[staging] > cd /home/provrtsp/public_html/promarket11.com && (echo '20170209071307,1.1.2.3' >> .dep/releases)
[staging] > cd /home/provrtsp/public_html/promarket11.com && (mkdir /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3)
[staging] > cd /home/provrtsp/public_html/promarket11.com && (if [[ "$(man ln)" =~ "--relative" ]]; then echo "true"; fi)
[staging] < bash: man: command not found
[staging] > cd /home/provrtsp/public_html/promarket11.com && (ln -nfs /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3 /home/provrtsp/public_html/promarket11.com/release)
• done on [staging]
✔ Ok [8s 97ms]
➤ Executing task deploy:update_code
[staging] > which git
[staging] < /usr/local/bin/git
[staging] > /usr/local/bin/git version
[staging] < git version 2.10.0
[staging] > if [ -h /home/provrtsp/public_html/promarket11.com/release ]; then echo 'true'; fi
[staging] < true
[staging] > readlink /home/provrtsp/public_html/promarket11.com/release
[staging] < /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3
[staging] > /usr/local/bin/git clone -b master --recursive -q git@gitlab.com:nerifil/mp.git /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3 2>&1
• done on [staging]
✔ Ok [17s 415ms]
➤ Executing task deploy:vendors
[staging] > if hash composer 2>/dev/null; then echo 'true'; fi
[staging] < true
[staging] > which composer
[staging] < /opt/cpanel/composer/bin/composer
[staging] > cd /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3 && /opt/cpanel/composer/bin/composer install --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader
[staging] < Loading composer repositories with package information
[staging] < Installing dependencies from lock file
[staging] < Dependency resolution completed in 0.001 seconds
[staging] < Analyzed 135 packages to resolve dependencies
[staging] < Analyzed 328 rules to resolve dependencies
[staging] < - Installing symfony/process (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing knplabs/knp-snappy (0.4.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing vlucas/phpdotenv (v2.4.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/polyfill-mbstring (v1.3.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/var-dumper (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/translation (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/routing (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/http-foundation (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/event-dispatcher (v3.2.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing psr/log (1.0.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/debug (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/http-kernel (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/finder (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/console (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing swiftmailer/swiftmailer (v5.4.5)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing paragonie/random_compat (v2.0.4)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing ramsey/uuid (3.5.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing nikic/php-parser (v3.0.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing jakub-onderka/php-console-color (0.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing jakub-onderka/php-console-highlighter (v0.3.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing dnoegel/php-xdg-base-dir (0.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing psy/psysh (v0.8.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing nesbot/carbon (1.22.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing mtdowling/cron-expression (v1.2.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing monolog/monolog (1.22.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing league/flysystem (1.0.34)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/polyfill-util (v1.3.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/polyfill-php56 (v1.3.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing jeremeamia/superclosure (2.3.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing doctrine/inflector (v1.1.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing classpreloader/classpreloader (3.1.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing laravel/framework (v5.3.30)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing barryvdh/laravel-snappy (v0.3.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/yaml (v3.2.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/dom-crawler (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/css-selector (v3.1.10)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing symfony/browser-kit (v3.2.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/diff (1.4.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/recursion-context (2.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/exporter (2.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/comparator (1.2.4)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/version (2.0.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/resource-operations (1.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/object-enumerator (2.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/global-state (1.1.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/environment (2.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/php-text-template (1.2.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing doctrine/instantiator (1.0.5)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/phpunit-mock-objects (3.4.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/php-timer (1.0.8)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/php-file-iterator (1.4.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing sebastian/code-unit-reverse-lookup (1.0.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/php-token-stream (1.4.9)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/php-code-coverage (4.0.5)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing webmozart/assert (1.2.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpdocumentor/reflection-common (1.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpdocumentor/type-resolver (0.2.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpdocumentor/reflection-docblock (3.1.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpspec/prophecy (v1.6.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing myclabs/deep-copy (1.6.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing phpunit/phpunit (5.7.12)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing psr/http-message (1.0.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing guzzlehttp/psr7 (1.3.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing guzzlehttp/promises (v1.3.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing guzzlehttp/guzzle (6.2.2)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing facebook/webdriver (1.3.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing behat/gherkin (v4.4.5)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing codeception/codeception (2.2.9)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing h4cc/wkhtmltoimage-amd64 (0.12.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing h4cc/wkhtmltopdf-amd64 (0.12.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing laravelcollective/html (v5.3.1)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing pusher/pusher-php-server (2.6.3)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < - Installing stripe/stripe-php (v4.4.0)
[staging] < Loading from cache
[staging] < Extracting archive
[staging] <
[staging] < Generating optimized autoload files
[staging] < Warning: Ambiguous class resolution, "App\Exceptions\Handler" was found in both "/home/provrtsp/public_html/promarket11.com/releases/1.1.2.3/app/Exceptions/HandlerNO.php" and "/home/provrtsp/public_html/promarket11.com/releases/1.1.2.3/app/Exceptions/Handler.php", the first will be used.
[staging] < > post-install-cmd: Illuminate\Foundation\ComposerScripts::postInstall
[staging] < > post-install-cmd: php artisan optimize
[staging] < Generating optimized class loader
[staging] < Compiling common classes
• done on [staging]
✔ Ok [110s 895ms]
➤ Executing task deploy:symlink
[staging] > if [[ "$(man mv)" =~ "--no-target-directory" ]]; then echo "true"; fi
[staging] < bash: man: command not found
[staging] > cd /home/provrtsp/public_html/promarket11.com && ln -nfs /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3 current
[staging] > cd /home/provrtsp/public_html/promarket11.com && rm release
• done on [staging]
✔ Ok [1s 637ms]
➤ Executing task cleanup
[staging] > cd /home/provrtsp/public_html/promarket11.com && if [ -e release ]; then rm release; fi
[staging] > cd /home/provrtsp/public_html/promarket11.com && if [ -h release ]; then rm release; fi
• done on [staging]
✔ Ok [919ms]
➤ Executing task environment
[staging] > cp /home/provrtsp/shared/env /home/provrtsp/public_html/promarket11.com/releases/1.1.2.3/.env
• done on [staging]
✔ Ok [5s 43ms]
➤ Executing task success
✔ Ok [0ms]
➤ Executing task composer_rerun
[staging] > cd /home/provrtsp/public_html/promarket11.com/current; php ~/composer.phar install
[staging] < Loading composer repositories with package information
[staging] < Installing dependencies (including require-dev) from lock file
[staging] < - Installing pimple/pimple (v3.0.2)
[staging] < Loading from cache
[staging] <
[staging] < - Installing phpseclib/phpseclib (2.0.4)
[staging] < Loading from cache
[staging] <
[staging] < - Installing psr/cache (1.0.1)
[staging] < Loading from cache
[staging] <
[staging] < - Installing symfony/cache (v3.2.3)
[staging] < Loading from cache
[staging] <
[staging] < - Installing symfony/expression-language (v3.2.3)
[staging] < Loading from cache
[staging] <
[staging] < - Installing evenement/evenement (v2.0.0)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/stream (v0.4.6)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/promise (v2.5.0)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/event-loop (v0.4.2)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/socket (v0.4.6)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/cache (v0.4.1)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/dns (v0.4.3)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/socket-client (v0.4.6)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/http-client (v0.4.15)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/http (v0.4.2)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/child-process (v0.4.1)
[staging] < Loading from cache
[staging] <
[staging] < - Installing react/react (v0.4.2)
[staging] < Loading from cache
[staging] <
[staging] < - Installing elfet/pure (v2.0.0)
[staging] < Loading from cache
[staging] <
[staging] < - Installing deployer/deployer (v4.2.1)
[staging] < Loading from cache
[staging] <
[staging] < - Installing deployer/phar-update (v2.0.0)
[staging] < Loading from cache
[staging] <
[staging] < - Installing fzaninotto/faker (v1.6.0)
[staging] < Loading from cache
[staging] <
[staging] < - Installing hamcrest/hamcrest-php (v1.2.2)
[staging] < Loading from cache
[staging] <
[staging] < - Installing mockery/mockery (0.9.7)
[staging] < Loading from cache
[staging] <
[staging] < phpseclib/phpseclib suggests installing ext-libsodium (SSH2/SFTP can make use of some algorithms provided by the libsodium-php extension.)
[staging] < symfony/cache suggests installing symfony/polyfill-apcu (For using ApcuAdapter on HHVM)
[staging] < react/event-loop suggests installing ext-event (~1.0)
[staging] < react/event-loop suggests installing ext-libev ()
[staging] < react/event-loop suggests installing ext-libevent (>=0.1.0)
[staging] < react/react suggests installing ext-event (Allows for use of a more performant event-loop implementation.)
[staging] < react/react suggests installing ext-libev (Allows for use of a more performant event-loop implementation.)
[staging] < react/react suggests installing ext-libevent (Allows for use of a more performant event-loop implementation.)
[staging] < Generating autoload files
[staging] < > Illuminate\Foundation\ComposerScripts::postInstall
[staging] < > php artisan optimize
[staging] < Generating optimized class loader
[staging] < Compiling common classes
• done on [staging]
✔ Ok [18s 103ms]
Successfully deployed!
@antonmedv antonmedv changed the title error: keep releases not working - keeps adding directories (named 1, 1.1, 1.1.2, 1.1.2.3) without deleting previous release directories Keep releases keeps adding directories (named 1, 1.1, 1.1.2, 1.1.2.3) Feb 9, 2017
@antonmedv
Copy link
Member

Try to clear releases dir (leave only last release) and delete .dep dir. Then deploy again.

@filipponeri
Copy link
Author

filipponeri commented Feb 9, 2017 via email

@antonmedv
Copy link
Member

antonmedv commented Feb 9, 2017 •
edited
Loading

This dir names indecates what deployer does not recognize 1as release. Please add this to deploy.php and show output:

task('debug', function () {
    writeln(json_encode(get('releases_list')));
});
after('deploy:release', 'debug');

@filipponeri
Copy link
Author

filipponeri commented Feb 9, 2017 via email

@antonmedv
Copy link
Member

Looks like deployer cant recognize releases list. Can you debug this part of code to see how releases list parsed?

I can't reproduce this.

@filipponeri
Copy link
Author

filipponeri commented Feb 10, 2017 via email

@filipponeri
Copy link
Author

I believe that I have found the bug in the code responsible for the unexpected behavior. I am however not sure where to fix the code.

Here is my understanding of the problem: the undesired behavior is caused by the result produced by toArray() at line 40 in file recipe/deploy/release.php, the relative snippet of the code is:

''set('releases_list', function () {
cd('{{deploy_path}}');
// If there is no releases return empty list.
if (!run('[ -d releases ] && [ "$(ls -A releases)" ] && echo "true" || echo "false"')->toBool()) {
return [];
}
// Will list only dirs in releases.
$list = run('cd releases && ls -t -d */')->toArray(); // <<<<<< line 40 producing an unexpected result
// Prepare list.
$list = array_map(function ($release) {
return basename(rtrim($release, '/'));
}, $list);

rest of the code
})

the toArray() function does not produce the expected result because the delimiter used is '\n' instead of a white space delimiter to break the string (please note that this simple fix does not work however as duplicate white spaces have to be removed as well).

As an example:
if the run() function at line 40 returns: '3/ 2/ 1.1.2.3.4/ 1.1/ 1/ 1.1.2.3/ 1.1.2/'
the variable $list at line 40 will contain an array of 1 element containing the string
'3/ 2/ 1.1.2.3.4/ 1.1/ 1/ 1.1.2.3/ 1.1.2/'

The expected result should be, I assume, that the array in $list would have 7 elements each one being the name of a directory in the input.
Because of the wrong content in $list than the rest of the code fails to perform as due.

Now the question is how/where to fix the code. I do not know well enough deployer to suggest to modify the toString() function as any modification there will also affect all the other calls in any part of the code. This is really a matter to be decided by Deployer's author.

I simply report here a local code change which consists in replacing line 40 with the following lines:

$list = run('cd releases && ls -t -d */');
$list = $list->toString();
$list = preg_replace('/\s+/', ' ',$list);    //to trim all white spaces in the string. 
//Note that php rtrim() or trim() are not enough because run() returns a

// string containing two or more consecutive white spaces
$list = explode(' ', $list);

With the proposed modification, assuming that run() returns
'3/ 2/ 1.1.2.3.4/ 1.1/ 1/ 1.1.2.3/ 1.1.2/'
then after the last line $list will contain the array:
[0] => 3/
[1] => 2/
[2] => 1.1.2.3.4/
[3] => 1.1/
[4] => 1/
[5] => 1.1.2.3/
[6] => 1.1.2/

which I assume to be the correct result.

What I still do not know in order to propose how to modify deployer's code:
a) does the toArray() function need to be modified?
b) does the uncorrect behavior of toArray() in release.php affect other parts of the code base or not?
c) why the decision to use '\n' in toArray() as delimiter? I presume that in other parts of the code this choice produces the correct behavior
d) and so on.
which is why I have to defer to the deployer's author to make the right decision.
Thank you for your time. Much appreciated.

@antonmedv
Copy link
Member

This is really strange behavior. Please, attache .dep/releases file here.
Also try to run this on server:
echo 'one' >> file
echo 'two' >> file
What the result?

@filipponeri
Copy link
Author

here is the output from the shell after running the two echo commands:

$ echo 'one' >> file
$ echo 'two' >> file
$ more file
one
two
$

here is the uploaded file (zipped)
releases.zip

Anyway the content of the .dep/releases file is:
20170210145402,1
20170210145735,2
20170210152101,3
20170210160211,4
20170211021308,5

Moreover I can confirm that now the set('keep_releases', 3); command produces the desired effect

Finally I want to add that the parsing problem that I observed does not happen when parsing the .dep/releases file but in parsing the output of the shell command called by run('cd releases && ls -t -d */')

@antonmedv
Copy link
Member

Ok, so there may be some system specific in this command: ls -t -d */
This command works fine on a lot of systems. Can you show output of this command?
And version of coreutils from ubuntu.

@filipponeri
Copy link
Author

$ ls -t -d */
5/ 4/ 3/

the output is on one single line

coreutils version is
$ ls --version
ls (GNU coreutils) 8.4
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.

@franciscomemoli
Copy link

I'm having the same problem.
when I run ls -t -d */ y get
1.1.2.3/ 1.1.2/ 1.1/ 2/ 1/
all in a single line.
There are a patch for the moment ?

@filipponeri
Copy link
Author

here is the patch that I made and I am currently using. I have done more than 10 deployments without any issue. So it seems to work.
Use at your own risk as I am not the author of Deployer I cannot provide any guarantee of correct behavior with other functions used by Deployer
release.php.zip

The attached file has to be copied in vendor/deployer/deployer/recipe/deploy/

@antonmedv
Copy link
Member

Will do patch, but why ls act so?

@franciscomemoli
Copy link

I did exactly the same! For now is working we will see in a few months.
Thanks!

@pluseg
Copy link
Contributor

pluseg commented Feb 17, 2017

@filipponeri @franciscomemoli Guys, what php version do you use?
I've figured out that the issue is in lines:

$nextReleaseNumber = 1;
if (count($list) > 0) {
    $nextReleaseNumber = (int)max($list) + 1;
}

In your case the expression (int)max($list) + 1 must equals 1, hence (int)max($list) = 0. This is possible if $list contains only 0, null or non-numeric string (http://php.net/manual/en/function.max.php). Note that even if $list = ['2/ 1/'] (instead of ['2/', '1/']) then my php7.0 returns correct answer (int)max(['2/ 1/']) = 2.

Could you please execute php -r "echo (int)max(['2/ 1/']);" on your servers?

@franciscomemoli
Copy link

franciscomemoli commented Feb 17, 2017 •
edited
Loading

I'm using php 7
PHP 7.0.9 (cli) (built: Oct 14 2016 14:50:46) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.9, Copyright (c) 1999-2016, by Zend Technologies

When I run php -r "echo (int)max(['2/ 1/']);" y get this

user@pc:~$ php -r "echo (int)max(['2/ 1/']);"
2user@pc:~$

@filipponeri
Copy link
Author

filipponeri commented Feb 17, 2017 via email •
edited
Loading

@antonmedv
Copy link
Member

Probliem is not in max func. Its in ls command which return one line instead of multiple lines(each folder on one line) please, try to run ls and get list.

@pluseg
Copy link
Contributor

pluseg commented Feb 18, 2017 •
edited
Loading

@Elfet yeah, it's my fault, the bug is in wrong exploding this string with all folders.
@franciscomemoli @filipponeri guys please try this ls -td1 */ instead of ls -t -d */. It works fine for me.

@filipponeri
Copy link
Author

filipponeri commented Feb 19, 2017 via email

@pluseg
Copy link
Contributor

pluseg commented Feb 19, 2017

@filipponeri Looks fine. Could you please replace ls -t -d */ with ls -td1 */ in recipe/deploy/release.php and try to deploy? Need to clear releases folder on your server before it. Also be sure that release.php doesn't contain your patch above.

ps. sorry for asking it. I can't reproduce it on my environment, so I need your help.

@franciscomemoli
Copy link

franciscomemoli commented Feb 19, 2017 •
edited
Loading

Hi,
I tryed and when I run ls -td1 */ I get something like
2.1/
2/
1/

But when I deploy with this fix it didn't work as I expected.
I was trying many things I and figure out that the problem is the order of the folders.
So, when I change ls -td1 */ for ls -d1 */ all works fine.

@pluseg
Copy link
Contributor

pluseg commented Feb 19, 2017 •
edited
Loading

@franciscomemoli did you try ls -d1 --sort=time */? What do you see in console when run ls -td1 */ and ls -d1 */ manually in release folder?

But when I deploy with this fix it didn't work as I expected.

Did you clear releases dir before new deploy? You should leave only one correct release and clear .dep folder.

@antonmedv
Copy link
Member

Releases order can be grabbed from .dep/releases file.

@antonmedv antonmedv mentioned this issue Feb 21, 2017
Merged
@antonmedv
Copy link
Member

Fixed

@filipponeri
Copy link
Author

filipponeri commented Feb 21, 2017 via email

@crosma
Copy link
Contributor

crosma commented Feb 22, 2017 •
edited
Loading

Huh, didn't even realize the naming 1.2.3.4.5 stuff was a side effect of the bug I fixed, as it was my first time using deployer, I figured that was some naming scheme deployer used.

I should also note that the code that builds the list of releases limits how many of the most recent releases it will read from the CSV to $keepReleases * 2 + 5 and will only prune releases from this list. This can lead to old copies sticking around that you do not want.

For example, if your keep_releases is set to 3, it will only look at the most recent 11 releases in the CSV. If you have 20 releases on your server the oldest 9 will never be deleted.

The easiest workaround is to set your keep_releases to <# of releases on your server> / 2 temporarily, run dep cleanup, reduce keep_releases and run dep cleanup again, repeat until you are down to the number you want to be at and it should work on its own after that.

Now that I think about it, this would be an outright bug for anyone who has lots of releases and later decides they want to reduce them drastically.

@crosma crosma mentioned this issue Feb 22, 2017
Merged
@antonmedv
Copy link
Member

antonmedv commented Feb 22, 2017 •
edited
Loading

The easiest workaround is to set your keep_releases to <# of releases on your server> / 2 temporarily, run dep cleanup, reduce keep_releases and run dep cleanup again, repeat until you are down to the number you want to be at and it should work on its own after that.

There even simple solution. Ssh to server and drop old releases. I'm thinking about increasing this value: $keepReleases * 2 + 5 or may be remove it fully.

Now that I think about it, this would be an outright bug for anyone who has lots of releases and later decides they want to reduce them drastically.

Yes, this is possible bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug research
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@antonmedv @pluseg @crosma @filipponeri @franciscomemoli