Sharing defaults across other yml files

[totalhack]

• Yet Another Cron version: 0.11.2
• Python version: 3..8
• Operating System: Ubuntu

Description

Hi, and thanks for putting this tool together.

I have multiple yacron yml files that I keep separated for organizational purposes. I also have a single defaults.yml that has my web and defaults sections. I had hoped that the defaults would get picked up and used for all other schedules that get picked up in that directory, but it doesn't seem to be working in all cases.

All of the various schedule files get ingested (can see that with /status) and the web section is definitely being read as the web endpoint is up. But it seems like some of the defaults section are being ignored or overridden. These settings worked when I had everything in a single file. But one of my jobs got a "failure" because it saw stderr, with the following message that conflicts with my settings:

has stderr: true; fail_reason: 'failsWhen=producesStderr and stderr is not empty'

So I know it's picking up my defaults.yml because the web interface comes up, but it seems to be not picking up, or overriding some of the other sections. Is this expected?

Note: none of my other files have a defaults section or touch any of these settings.

Example defaults.yml

web:
  listen:
    - http://0.0.0.0:8030

defaults:
  shell: /bin/bash
  utc: false
  timezone: America/New_York
  concurrencyPolicy: Forbid
  executionTimeout: 600
  killTimeout: 10
  captureStderr: true
  captureStdout: true
  failsWhen:
    producesStdout: false
    producesStderr: false
    nonzeroReturn: true
    always: false
  onFailure:
    report:
      mail:
        from: <from email>
        to: <to email>
        smtpHost: smtp.gmail.com
        username: <username>
        password:
          fromEnvVar: <password env var>
        tls: true
        starttls: true
    retry:
      maximumRetries: 1
      initialDelay: 30
      maximumDelay: 60
      backoffMultiplier: 1

[totalhack]

Apologies, missed this note in your README:

Note: if the configuration option is a directory and there are multiple configuration files in that directory, then the defaults section in each configuration file provides default options only for cron jobs inside that same file; the defaults have no effect beyond any individual YAML file.

It would be great to support that, lots of repetition if I try to organize my schedules in different files, but I will work around it for now.

[gjcarneiro]

Yeah. The problem is that the order that yacron includes files in a directory is undefined. And it would be confusing to have the default options change magically depending on what other files exist in the same directory.

If you can work around, great, otherwise we could maybe think of some way to have includes:

defaults:
  ...

includes:
  - file1.yaml
  - file2.haml

Then you could point yacron a the main yaml file. I think that, in this case, it would be OK that included files inherit the defaults set by the parent file. Or something, I'd have to think more about it...

[totalhack]

That could work. Another option, which may cause issues with existing implementations if the name conflicts, would be to look for a special file name to use as a baseline default that is first loaded. So always check for defaults.yml (or something more specific).

[gjcarneiro]

I would prefer to make it explicit, something like:

include:
  - defaults.yaml

jobs:
   ...

But then the tricky part is to make sure that the defaults.yaml doesn't get loaded automatically too, when you give yacron a directory as config. Or maybe it doesn't really matter if it only has a defaults section and empty jobs list... 🤷

I guess it's worth re-opening. It doesn't sound like a complicated feature to implement.

[gjcarneiro]

0.13