Facilitate batch jobs for repetitive calculations

[ischoegl]

Abstract

Research in combustion often requires repetitive calculations for various parameter variations. The goal of this external work is to provide a simple means to automate these tasks. The original Cantera 2.4 based code - ctwrap - has been posted for some time; at the moment, the package is being ported to Cantera 2.5, and now employs Cantera's newly implemented HDF export capabilities.

Motivation

The code is developed with the following objectives in mind. It should:

• create a flexible framework for generic simulations
• provide a command line interface
• be easily scriptable (using YAML configuration files via ruamel.yaml)
• enforce units (via pint)
• avoid clutter (data are saved in HDF containers)
• enable parallel execution of (single-threaded) simulations (via multiprocessing)
• enable simple re-import of simulation results into native Cantera objects

Description

It is easiest to illustrate the objectives with an example: a parallel batch job for adiabatic flame calculations uses the simulation module adiabatic_flame (modified from Cantera's adiabatic_flame.py example) with parameters defined in adiabatic_flame.yaml (for this example, a variation of equivalence ratio with 12 cases). This can be run as:

$ ctwrap adiabatic_flame adiabatic_flame.yaml --parallel

Results are written to a single file adiabatic_flame.h5.

Adapting custom simulation tasks to be used with ctwrap is straight-forward, and mainly involves the creation of a Python module that largely reflects a regular simulation script for a single set of parameter values.

Alternatives

Parameter variations can always be run in custom loops.

References

• microcombustion/ctwrap

[ischoegl]

Work on the code revision is mostly done at this point - the tag v0.2.0 should be fully operational, although we'll be testing for a while (and update the occasional glitch in the documentation).

For the updated version, the YAML configuration of batch jobs was completely revised. The example for the sweep of equivalence ratios mentioned above would use:

# YAML configuration for the `adiabatic_flame` module
strategy:
  sequence:
    upstream.phi: { mode: linspace, limits: [0.4, 2.6], npoints: 12 }
output:
  format: h5
defaults:
  upstream:
    T: 300. kelvin # temperature
    P: 1. atmosphere # pressure
    phi: .55 # equivalence ratio
    fuel: H2
    oxidizer: O2:1,AR:5
  chemistry:
    mechanism: h2o2.yaml
  domain:
    width: 30 millimeter # domain width
ctwrap: 0.2.0

We prepared three examples (minimal example, ignition, and adiabatic flame); in terms of strategies, both sweeps/sequences and test matrices are supported. Beyond, the code is designed to work for arbitrary user-supplied simulations, which can be easily converted to a format compatible with ctwrap.

For further information, see Sphinx documentation.

[bryanwweber]

This is awesome! We have an affiliated projects page on the Cantera website, let me know if you want to add it https://cantera.org/affiliated-packages.html

On a related note, a lifetime ago I wrote a parser for Chemkin input files (for SENKIN, specifically) to do a Cantera simulation (https://github.com/bryanwweber/cansen). I wonder if you'd be interested in taking the parser code to convert into a YAML input file for this wrapper? That seems a far more sustainable path than trying to maintain a separate package.

[ischoegl]

@bryanwweber ... thanks!

[...] We have an affiliated projects page on the Cantera website, let me know if you want to add it https://cantera.org/affiliated-packages.html

I'm aware of it, and am planning to add the project. Before I create a PR I'll probably let this sit for some time. By the time Cantera 2.5 is rolled out, it should be ready though (it does require 2.5 anyways).

On a related note, a lifetime ago I wrote a parser for Chemkin input files (for SENKIN, specifically) to do a Cantera simulation (https://github.com/bryanwweber/cansen). I wonder if you'd be interested in taking the parser code to convert into a YAML input file for this wrapper? That seems a far more sustainable path than trying to maintain a separate package.

I haven't thought about this before and will probably have to think about it. At the moment I have only had a brief look at CanSen: from what I can gather, it seems quite straight-forward to create ctwrap simulation modules (and corresponding YAML configurations) for standard calculations that CanSen supports. I'll definitely look into this some more, as this is an interesting suggestion.

I myself have never used SENKIN, although I have run Chemkin-III 15+ years ago and still have a copy somewhere. So I frankly do not know how many users would benefit from the SENKIN->YAML conversion (I've never used a modern version of Chemkin).

[ischoegl]

@bryanwweber ... thanks for your feedback, which inspired me to implement another 'module' for equilibrium calculations. I ended up with an approach that can handle both single phases (ct.Solution) or multiple phases (ct.Mixture). It's not quite the same as CanSen, but a start. Output is currently to CSV only (via pandas).

A main motivation for this is that I realized that this would be really nice to have for the combustion class I will be teaching in spring, where it can flatten the initial learning curve for Cantera quite a bit. Any feedback is welcome (the current version is tagged v0.3.0-beta).

Here's YAML input for a single phase

# YAML file example for the `equilibrium` module
# file specifes a parameter variation of the equivalence ratio
strategy:
  sequence:
    initial.phi: { mode: linspace, limits: [0.3, 3.5], npoints: 33 }
output:
  format: csv
  force: True
  returns:
    T: T # temperature
    P: P # pressure
    X: X # species mole fractions
defaults:
  initial:
    T: 300. kelvin # temperature
    P: 1. atmosphere # pressure
    phi: .55 # equivalence ratio
  phases:
    gas: # gas phase
      mechanism: gri30.yaml
      moles: 1.
      fuel: CH4
      oxidizer: O2:1.0, N2:3.76
  equilibrate:
    mode: HP
ctwrap: 0.3.0

whereas this one would be for multiple phases:

# YAML file example for the `equilibrium` module
# file specifes a parameter variation of the equivalence ratio
strategy:
  sequence:
    initial.phi: { mode: linspace, limits: [0.3, 3.5], npoints: 33 }
output:
  format: csv
  force: True
  returns:
    T: T # temperature
    P: P # pressure
    species_names: species_moles # moles per species
defaults:
  initial:
    T: 300. kelvin # temperature
    P: 1. atmosphere # pressure
    phi: .55 # equivalence ratio
  phases:
    gas: # gas phase
      mechanism: gri30.yaml
      moles: 1.
      fuel: CH4
      oxidizer: O2:1.0, N2:3.76
    carbon: # solid phase
      mechanism: graphite.yaml
      moles: 0.
      X: C(gr):1.0
  equilibrate:
    mode: HP
    solver: gibbs
    max_steps: 1000
ctwrap: 0.3.0

[bryanwweber]

@ischoegl In going back through the issues to make a comment on #89, I noticed that this also doesn't seem to require any specific changes to Cantera itself. Can it be closed to keep us organized? Thanks!

[ischoegl]

@bryanwweber Absolutely! I have no issue whatsoever with closing this here. I had posted it initially here to solicit feedback when I was overhauling the code late last year. As mentioned elsewhere, I believe using #83 would provide a better venue.

PS: some of the changes I implemented for 2.5.1 were motivated by issues I had run into when running batch jobs. So the 'work-in-progress' was really after the fact (i.e. there were some enhancements to cantera that were due to this work).