Add --exclude-group to cron:run #34117

convenient · 2021-09-17T15:35:00Z

Description

https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-group-run

When you decide to split a single cron group with the --group flag it means you need to do so for every group in your instance.

It's pretty common for third party modules to define their own groups whenever they like, so we have to keep up to date with the defined groups in the instance to keep adding these to our crontab when new groups are added.

One of my instances currently has 11 groups to manage, more will come in the future and its just something I need to mentally keep an eye on, a bit annoying.

In my case I only really want to run the crons like

* * * * * bin/magento cron:run --exclude-group index
* * * * * flock -n index.lock bin/magento cron:run --group index

This will give me better granular control over the indexer crons which can be useful when your store reaches hundreds of thousands of SKUs. Any other random cron groups would then still be able to run by default like in normal operation.

For future proofing I've made the --exclude-group flag stackable so you can define it multiple times like

* * * * * bin/magento cron:run --exclude-group index --exclude-group consumers

Manual testing scenarios (*)

Run cron with no params, all groups run
Run cron with --group, only that group runs
Run cron with --group and that group also in --exclude-group, no groups run
Run cron with --exclude-group, that group is excluded
Run cron with multiple --exclude-group, all defined groups are excluded

Contribution checklist (*)

Pull request has a meaningful description of its purpose
All commits are accompanied by meaningful commit messages
All new or changed code is covered with unit/integration tests (if applicable)
README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
All automated tests passed successfully (all builds are green)

Resolved issues:

resolves [Issue] Add --exclude-group to cron:run #36628: Add --exclude-group to cron:run

https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-group-run When you decide to split a single cron group with the `--group` flag it means you need to do so for every group in your instance. It's pretty common for third party modules to define their own groups whenever they like, so we have to keep up to date with the defined groups in the instance to keep adding these to our crontab when required. One of my instances currently has 11 groups to manage, more will come in the future. In my case I only really want to run the crons like ``` * * * * * bin/magento cron:run --exclude-group index * * * * * flock -n index.lock bin/magento cron:run --group index ``` This will give me better granular control over the indexer crons which can be useful when your store reaches hundreds of thousands of SKUs For future proofing I've made the `--exclude-group` flag stackable so you can define it multiple times like ``` * * * * * bin/magento cron:run --exclude-group index --exclude-group consumers ```

convenient · 2021-09-17T15:35:22Z

@magento run all tests

m2-assistant · 2021-09-17T15:35:34Z

Hi @convenient. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

@magento give me test instance - deploy test instance based on PR changes
@magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

@magento run all tests - run or re-run all required tests against the PR changes
@magento run <test-build(s)> - run or re-run specific test build(s)
For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

Database Compare
Functional Tests CE
Functional Tests EE,
Functional Tests B2B
Integration Tests
Magento Health Index
Sample Data Tests CE
Sample Data Tests EE
Sample Data Tests B2B
Static Tests
Unit Tests
WebAPI Tests
Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

magento-automated-testing · 2021-09-17T15:35:56Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

convenient · 2021-09-17T16:32:05Z

@magento run all tests

magento-automated-testing · 2021-09-17T16:32:09Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

BarnyShergold

Not sure why the sniffer is complaining about the CronCommand.php file as those code elements have been there for sometime.

However the sniffer errors in the test file need to be fixed. If you search the code base for 'array_filter' you will see how the use of 'function' needs to be formatted.

Once the errors are fixed we can give it a proper review :)

convenient · 2021-09-20T15:31:32Z

@BarnyShergold Aye i spotted that as well. I think maybe CronCommand.php hasn't been touched in a while and the static analysis might only run on files changed?

Do you want me to address all static defects in CronCommand.php?

I am on a few days annual leave now, will fix my dodgy array_filters later in the week

BarnyShergold · 2021-09-20T16:24:39Z

@BarnyShergold Aye i spotted that as well. I think maybe CronCommand.php hasn't been touched in a while and the static analysis might only run on files changed?

Do you want me to address all static defects in CronCommand.php?

I am on a few days annual leave now, will fix my dodgy array_filters later in the week

if you can, that would be great - save any aggro :)

convenient · 2021-09-23T08:11:09Z

@magento run all tests

magento-automated-testing · 2021-09-23T08:11:17Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

convenient · 2021-09-23T08:52:58Z

@BarnyShergold Two failures in this test run for the static analysis

PHP Code Sniffer detected 1 violation(s): 

FILE: /var/www/html/app/code/Magento/Cron/Console/Command/CronCommand.php
-------------------------------------------------------------------------
FOUND 2 ERRORS AFFECTING 2 LINES
-------------------------------------------------------------------------
  67 | ERROR | If the @inheritdoc not inline it shouldn’t have braces
 102 | ERROR | {@inheritdoc} does not import parameter annotation
-------------------------------------------------------------------------

As per magento/magento-coding-standard#154 I have no idea what to do with this issue. I've checked a bunch of different core console commands and they all look the same to me. Looking at https://devdocs.magento.com/guides/v2.3/coding-standards/docblock-standard-general.html#inheritdoc it says

The @inheritdoc tag SHOULD NOT be used.

So 🤷 not sure what I can do here. Please advise.

This one seems like a false alert to me, I didn't touch anything remotely near this area so not sure why its popping up.

PHP Copy/Paste Detector has found error(s):
<?xml version="1.0" encoding="UTF-8"?>
<pmd-cpd>
  <duplication lines="28" tokens="92">
    <file path="/var/www/html/app/code/Magento/Quote/Model/Quote/QuantityCollector.php" line="20"/>
    <file path="/var/www/html/app/code/Magento/Quote/Model/Quote/TotalsCollector.php" line="222"/>
    <codefragment>    public function collectItemsQtys(Quote $quote)
    {
        $quoteItems = $quote-&gt;getAllVisibleItems();
        $quote-&gt;setItemsCount(0);
        $quote-&gt;setItemsQty(0);
        $quote-&gt;setVirtualItemsQty(0);

        foreach ($quoteItems as $item) {
            if ($item-&gt;getParentItem()) {
                continue;
            }

            $children = $item-&gt;getChildren();
            if ($children &amp;&amp; $item-&gt;isShipSeparately()) {
                foreach ($children as $child) {
                    if ($child-&gt;getProduct()-&gt;getIsVirtual()) {
                        $quote-&gt;setVirtualItemsQty($quote-&gt;getVirtualItemsQty() + $child-&gt;getQty() * $item-&gt;getQty());
                    }
                }
            }

            if ($item-&gt;getProduct()-&gt;getIsVirtual()) {
                $quote-&gt;setVirtualItemsQty($quote-&gt;getVirtualItemsQty() + $item-&gt;getQty());
            }
            $quote-&gt;setItemsCount($quote-&gt;getItemsCount() + 1);
            $quote-&gt;setItemsQty((float)$quote-&gt;getItemsQty() + $item-&gt;getQty());
        }

</codefragment>
  </duplication>
</pmd-cpd>

Failed asserting that false is true.

Exclude the staging group etc, not relevant for our test

convenient · 2021-09-23T10:33:54Z

@magento run all tests

magento-automated-testing · 2021-09-23T10:33:59Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

convenient · 2021-09-23T12:44:13Z

@magento run Functional Tests CE

magento-automated-testing · 2021-09-23T12:44:20Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

BarnyShergold · 2021-09-23T13:31:28Z

@magento run Static Tests

magento-automated-testing · 2021-09-23T13:31:31Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

convenient · 2021-09-23T16:02:08Z

@BarnyShergold Yeah those two same failures again. what to do with the inheritDoc and that completely random unrelated failure 🤔

engcom-Lima · 2022-12-16T10:59:48Z

✔️ QA Passed

Preconditions:

Install fresh Magento 2.4-develop

Manual testing scenario:

Run cron with no params, all groups run
Run cron with --group, only that group runs
Run cron with --group and that group also in --exclude-group, no groups run
Run cron with --exclude-group, that group is excluded
Run cron with multiple --exclude-group, all defined groups are excluded

Before: ✖️ Not able to run cron using "--exclude-group"

After: ✔️ Now its happening.

==========================================================================================

Additionally tested:
After running the cron executed below scenarios:

Created customer.
Placed order from the customer.
Placed order from the guest.
In admin side -Created new product.
Created invoice, creditmemo, reordered.

No additional manual test case is required as a part of regression.

Since, this branch is having conflicts. Hence, moving this PR to extended testing.

engcom-Echo · 2022-12-16T11:47:12Z

@magento run all tests

magento-automated-testing · 2022-12-16T11:47:17Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

magento-automated-testing · 2022-12-16T12:40:07Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

engcom-Echo · 2022-12-16T13:19:13Z

@magento run Functional Tests CE

magento-automated-testing · 2022-12-16T13:19:18Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

magento-automated-testing · 2022-12-19T05:31:57Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

engcom-Echo · 2022-12-19T12:43:04Z

@magento run all tests

magento-automated-testing · 2022-12-19T12:43:14Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

magento-automated-testing · 2022-12-19T14:37:54Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

magento-automated-testing · 2022-12-19T15:01:58Z

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

magento-engcom-team added the Component: Cron label Sep 17, 2021

magento-engcom-team added Release Line: 2.4 Partner: Ampersand partners-contribution Pull Request is created by Magento Partner labels Sep 17, 2021

Fix unit test

03654f8

BarnyShergold requested changes Sep 20, 2021

View reviewed changes

m2-community-project bot assigned BarnyShergold Sep 20, 2021

m2-community-project bot added the Progress: needs update label Sep 20, 2021

Fix static analysis

52580cc

convenient added 2 commits September 23, 2021 11:29

Update test to run the same on EE and B2B

def2562

Exclude the staging group etc, not relevant for our test

Update test to run the same on EE and B2B

fae148b

Exclude the staging group etc, not relevant for our test

m2-community-project bot added Progress: testing in progress and removed Progress: ready for testing labels Dec 16, 2022

m2-community-project bot added Progress: extended testing and removed Progress: testing in progress labels Dec 16, 2022

Merge 2.4-develop and resolve conflicts

a88182a

engcom-Echo self-assigned this Dec 19, 2022

m2-community-project bot added Progress: ready for testing and removed Progress: extended testing labels Dec 19, 2022

m2-community-project bot added Progress: extended testing and removed Progress: ready for testing labels Dec 19, 2022

Fixed failing integration test

34f9f72

m2-community-project bot added Progress: accept and removed Progress: extended testing labels Dec 20, 2022

magento-devops-reposync-svc merged commit ef91c61 into magento:2.4-develop Dec 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --exclude-group to cron:run #34117

Add --exclude-group to cron:run #34117

convenient commented Sep 17, 2021 •

edited by m2-assistant bot

Loading

convenient commented Sep 17, 2021

m2-assistant bot commented Sep 17, 2021

magento-automated-testing bot commented Sep 17, 2021

convenient commented Sep 17, 2021

magento-automated-testing bot commented Sep 17, 2021

BarnyShergold left a comment

convenient commented Sep 20, 2021

BarnyShergold commented Sep 20, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

BarnyShergold commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

engcom-Lima commented Dec 16, 2022 •

edited by engcom-Echo

Loading

engcom-Echo commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

engcom-Echo commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

magento-automated-testing bot commented Dec 19, 2022

engcom-Echo commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

Add --exclude-group to cron:run #34117

Add --exclude-group to cron:run #34117

Conversation

convenient commented Sep 17, 2021 • edited by m2-assistant bot Loading

Description

Manual testing scenarios (*)

Contribution checklist (*)

Resolved issues:

convenient commented Sep 17, 2021

m2-assistant bot commented Sep 17, 2021

magento-automated-testing bot commented Sep 17, 2021

convenient commented Sep 17, 2021

magento-automated-testing bot commented Sep 17, 2021

BarnyShergold left a comment

Choose a reason for hiding this comment

convenient commented Sep 20, 2021

BarnyShergold commented Sep 20, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

BarnyShergold commented Sep 23, 2021

magento-automated-testing bot commented Sep 23, 2021

convenient commented Sep 23, 2021

engcom-Lima commented Dec 16, 2022 • edited by engcom-Echo Loading

After: ✔️ Now its happening.

engcom-Echo commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

engcom-Echo commented Dec 16, 2022

magento-automated-testing bot commented Dec 16, 2022

magento-automated-testing bot commented Dec 19, 2022

engcom-Echo commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

magento-automated-testing bot commented Dec 19, 2022

convenient commented Sep 17, 2021 •

edited by m2-assistant bot

Loading

engcom-Lima commented Dec 16, 2022 •

edited by engcom-Echo

Loading