Please add cron support

[TomashKhamlai]

ISSUE TYPE

• Feature request

SUMMARY

Cron is often used by CMS

Goal

Magento Upgrade process is possible with cron. If this feature can have negative impact on the whole system then instructions for adding external cron docker in Magento tutorial would be enough to resolve this feature request

[cytopia]

@TomashKhamlai

For production images running Magento I also require cron, however I am using the one from the host system.

Either via docker

0 3 * * * root docker exec devilbox_php_1 php -d memory_limit=-1 /shared/httpd/magento/shell/indexer.php --reindex catalog_url

Or via Docker Compse:

0 3 * * * root cd /path/to/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento/shell/indexer.php --reindex catalog_url

This works well for me. Let me know if this also works for you.

[TomashKhamlai]

@cytopia here how it looks now

*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /shared/httpd/magento2/magento2/var/log/magento.cron.log

*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/update/cron.php >> /shared/httpd/magento2/magento2/var/log/update.cron.log
*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/bin/magento setup:cron:run >> /shared/httpd/magento2/magento2/var/log/setup.cron.log

In the line where I call php bin/magento cron:run cron is not accessible in the container.

So this is not a solution for Magento 2

[cytopia]

@TomashKhamlai
Not sure what you mean by cron is not accessible in the container. The suggestion I proposed above is without using cron from the container.

If you are inside the PHP container, what is the exact command to run the Magento 2 cronjob?

[TomashKhamlai]

Inside of PHP container I run php bin/magento cron:run twice. The first command schedules new changes to MySQL database and executes everything that already has been scheduled and the second one executes the schedule.

To check I use

mysql> select * from magento2.cron_schedule;
Empty set (0.02 sec)

So something wrong with my crontab or Magento 2 application wants to have acces to the cron executable.

[cytopia]

Does this work if you do a apt update && apt install -y cron inside the php container first?

[cytopia]

The next PHP-FPM release will have cron and crontab binaries installed. See above linked PR or here: devilbox/docker-php-fpm#78

[TomashKhamlai]

Today, I tried to do:
git pull && ./update-docker.sh

Then I logged in to PHP container and issued this commands:

devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ php bin/magento cron:install
Error during saving of crontab: sh: 7: crontab: not found
devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ whereis cron
cron: /etc/cron.daily
devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ 

And here what happens when it is installed by hands:

devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ sudo apt install cron
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  anacron logrotate checksecurity
The following NEW packages will be installed:
  cron
0 upgraded, 1 newly installed, 0 to remove and 12 not upgraded.
Need to get 95.4 kB of archives.
After this operation, 257 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 cron amd64 3.0pl1-128+deb9u1 [95.4 kB]
Fetched 95.4 kB in 0s (393 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package cron.
(Reading database ... 37207 files and directories currently installed.)
Preparing to unpack .../cron_3.0pl1-128+deb9u1_amd64.deb ...
Unpacking cron (3.0pl1-128+deb9u1) ...
Setting up cron (3.0pl1-128+deb9u1) ...
Adding group `crontab' (GID 105) ...
Done.
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

I hope this helps.

[cytopia]

So once cron is installed, does the Magento cronjob command work then?

[cytopia]

@TomashKhamlai

Error during saving of crontab: sh: 7: crontab: not found

This has now been fixed the merging the above linked PR. Check out the latest master branch again.

[TomashKhamlai]

@cytopia, what output is after issuing?

crontab -l

I see:

bash: crontab: command not found

I can see the changes in git log also I see the changes in code. Cron is still unavailable for me.

[cytopia]

what container version are you on? And what is your current Devilbox git hash?

[TomashKhamlai]

commit 40d9bd1
Now I am updating docker-images, so my next try wiil be on devilbox/php-fpm:7.1-work-0.65

[TomashKhamlai]

I have checked and I can verify that cron and crontab are not available inside of php container.

[cytopia]

OK, good to hear. Does the Magento cron command now succeed?

[TomashKhamlai]

Nope. No cron, no crontab and php bin/magento cron:... also complains on that.

[cytopia]

It's both available in -0.65

root@php-7.1.25 in /shared/httpd $ cron
cron     crontab
root@php-7.1.25 in /shared/httpd $ which cron
/usr/sbin/cron
root@php-7.1.25 in /shared/httpd $ which crontab
/usr/bin/crontab
root@php-7.1.25 in /shared/httpd $

[SlimDeluxe]

I see support has been added but how do you overcome the fact that the crontab is not persistent? What use is it if it can't survive a restart.
Also, I tried adding a PHP command but it's never executed. The same command works fine if executed directly in CLI.
If try even this basic example
* * * * * php -v >> /shared/httpd/cron.log
nothing is added to cron.log, but if I try
* * * * * date >> /shared/httpd/cron.log
it works.

[SlimDeluxe]

I needed to follow these additional instructions in order to get it working

1. Use absolute binary paths
2. Add the -T parameter to docker-compose exec

So my command in the crontab would look like this:
* * * * * cd /home/omer/devilbox; /usr/local/bin/docker-compose exec -T php /usr/local/bin/php -v >> /home/omer/docker-scheduler.log

[masiorama]

Even following the previous suggestions I'm not able to make cronjob work.
I tried with crontab -e and added 2 * * * * date >> /shared/httpd/cron.log expecting some output but nothing so far.
Any help?

[martin-rueegg]

I found a solution that seems to work:

I created a file $HOME/devilbox/autostart/project-autostart.sh that will

1. create functions to add and remove cron jobs
2. start the cron daemon
3. run all the .sh scripts in $HOST_PATH_HTTPD_DATADIR/*/.devilbox/autostart

In my project directory, I added the file .devilbox/autostart/cron.sh that adds the required cron jobs:

cron_add "* * * * * "$(which php)" /shared/httpd/my_project/htdocs/path/to/script.php some arguments >/dev/null 2>&1"

This can easily be tested with
cron_add "* * * * * date >> /shared/httpd/cron.log"

(which can again be removed later with cron_remove "* * * * * date >> /shared/httpd/cron.log")

I've also created some PRs to support this functionality out-of-the-box:

• Add project-specific startup script support (eg. for cron configuration) devilbox/docker-php-fpm#263
• Improve cron support devilbox/docker-php-fpm#264

Hope this helps anyone.

---

File $HOME/devilbox/autostart/project-autostart.sh

#!/usr/bin/env bash


set -e
set -u
set -o pipefail

DEBUG_LEVEL=$(printenv DEBUG_ENTRYPOINT)
DEBUG_LEVEL=${DEBUG_LEVEL:-0}
export DEBUG_LEVEL

###########################################################
# Functions
############################################################

###
### Execute custom uesr-supplied scripts
###
execute_project_scripts() {
        local script_dir="${1}"
        local debug="${2}"

        if [ ! -d "${script_dir}" ]; then
                run "mkdir -p ${script_dir}" "${debug}"
        fi
        script_files="$( find -L "${script_dir}" -type f -iname '*.sh' | sort -n )"

        # loop over them line by line
        IFS='
        '
        for script_f in ${script_files}; do
                script_name="$( basename "${script_f}" )"
                log "info" "Executing project startup script: ${script_name}" "${debug}"
                if ! bash "${script_f}"; then
                        log "err" "Failed to execute script" "${debug}"
                        exit 1
                fi
        done
}

cron_add() {
  # save the entire line in one variable
  line="$*"

  # check if line already exists in crontab
  crontab -l -u devilbox | grep "$line" > /dev/null
  status=$?

  if [ $status -ne 0 ]
  then
    log "info" "cron: adding line '${line//\*/\*}' ..." "$DEBUG_LEVEL"
    (crontab -l -u devilbox; echo "$line";) | crontab -u devilbox -
  fi

  # make sure the cron service is running
  if ! service cron status >/dev/null
  then
    service cron start
  fi
}
export -f cron_add

cron_remove() {
  # save the entire line in one variable
  line=$*

  # check if line already exists in crontab
  crontab -l -u devilbox | grep "$line" > /dev/null
  status=$?

  if [ $status -eq 0 ]; then
    log "info" "cron: removing line '${line//\*/\*}' ..." "$DEBUG_LEVEL"
    (crontab -l -u devilbox | grep -v "$line";) | crontab -u devilbox -
  fi
}
export -f cron_remove

# this function is required, as long as it is not exported by /docker-entrypoint.d/100-base-libs.sh
log() {
        echo $1 $2
}


for project_dir in /shared/httpd/*
do
        if [ -d $project_dir/.devilbox/autostart ]; then
                project=$(basename "$project_dir")
                log "info" "Evaluating project startup scripts for '$project'" "$DEBUG_LEVEL"
                execute_project_scripts "$project_dir/.devilbox/autostart" "$DEBUG_LEVEL"
        fi
done