SimHarness - Reinforcement Learning for Wildfire Mitigation in Simulated Disaster Environments

SimHarness is a Python-based harness that wraps a SimFire environment to generate effective wildfire mitigation strategy responses via reinforcement learning (RL). Through an easy-to-use API, SimHarness can train one, or multiple, intelligent agents within the SimFire Simulation with a variety of different RL algorithms defined by RLlib.

Documentation is available here.

Conceptual workflow for training an RL model using SimHarness within the SimFire environment.

Table of Contents

• About SimHarness
• Installation
• Getting Started
• License
• Citation

About SimHarness

SimHarness is a Python repository designed to support the training of RLlib RL algorithms within simulated disaster environments defined by SimFire. SimHarness takes as input an instance of the SimFire Simulation class, such as SimFire's FireSimulation, as the training environment. The Simulation object provides an API that allows SimHarness to move agents around the simulated environment and interact with it by placing mitigations. The FireSimulation agents represent firefighters moving through an environment as a wildfire spreads, placing mitigations such as firelines to limit the spread of the fire within the area.

The SimHarness training loop functions similarly to a traditional RL training loop, except it expects the passed-in environment to be a child class of Simulation as opposed to a gymnasium environment. Simulation is currently a class within the SimFire package, but is expected to be moved to a separate, non-disaster-specific package in the future. The simulated environment outputs training signals such as observations and rewards to the SimHarness agent(s) which use the observations to predict optimal actions. The actions produced by the model provide both movement and interaction information. Movements are how the agent is traversing across the environment, such as [nothing, up, down, left, right]. Interactions are how the agent is changing the environment itself. In the case of SimFire, this can be [nothing, fireline, wetline, scratchline]. These actions are relayed back to the simulated environment, which then affects the overall disaster scenario simulated by the environment.

Installation

*Note: SimHarness has only been tested on Ubuntu 18.04 and Python 3.9.18

1. Clone the repository.

   git clone https://github.com/mitrefireline/simharness.git
   cd simharness

2. Setup Pyenv Virtual Environment

   curl https://pyenv.run | bash

   export PYENV_ROOT="$HOME/.pyenv"
   command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
   eval "$(pyenv init -)"

   sudo apt update
   sudo apt install \
   build-essential \
   curl \
   libbz2-dev \
   libffi-dev \
   liblzma-dev \
   libncursesw5-dev \
   libreadline-dev \
   libsqlite3-dev \
   libssl-dev \
   libxml2-dev \
   libxmlsec1-dev \
   llvm \
   make \
   tk-dev \
   wget \
   xz-utils \
   zlib1g-dev

   pyenv install 3.9.18
   pyenv local 3.9.18

3. Install poetry

   curl -sSL https://install.python-poetry.org | python3 -

   Note: Don't forget to add poetry to your path.

   export PATH="$HOME/.local/bin:$PATH"

   poetry config virtualenvs.in-project true
   poetry env use 3.9.18

4. Install required packages

   poetry shell
   poetry install --only main

   Note: To re-enter the environment after this step, run poetry shell.

Getting Started

SimHarness provides a number of tutorials within the tutorials section of the documentation. Documentation is available here.

License

SimHarness is released under the Apache 2.0 license.

Citation

If you use SimHarness for your work, please cite our white paper with the following BibTex entry.

@misc{tapley2023simharness,
      title={Reinforcement Learning for Wildfire Mitigation in Simulated Disaster Environments},
      author={Alexander Tapley and Marissa Dotter and Michael Doyle and Aidan Fennelly and Dhanuj Gandikota and Savanna Smith and Michael Threet and Tim Welsh},
      year={2023},
      booktitle={"Proc. of the Tackling Climate Change with Machine Learning Workshop"},
      series={NeurIPS},
      howpublished={\url{https://arxiv.org/abs/2311.15925}}
}

This project contains content developed by The MITRE Corporation. If this code is used in a deployment or embedded within another project, it is requested that you send an email to opensource@mitre.org in order to let us know where this software is being used.

Copyright ©2022-2023 The MITRE Corporation. ALL RIGHTS RESERVED. Approved for Public Release; Distribution Unlimited. Public Release Case Number 22-3261.