ComStock V1 Public Release.

.gitignore

@@ -0,0 +1,15 @@
+.temp
+.idea
+.DS_Store
+clusters
+data
+local_configuration.*
+AFDD-experiments.*
+localResults
+workflow
+time_series_csv_files
+logs
+server.pid
+temp_data
+*~
+*_cluster.json

LICENSE.txt

@@ -0,0 +1,30 @@
+ComStock™, Copyright (c) 2020 Alliance for Sustainable Energy, LLC. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the
+following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following
+disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
+disclaimer in the documentation and/or other materials provided with the distribution.
+
+3. Redistribution of this software, without modification, must refer to the software by the same designation.
+Redistribution of a modified version of this software (i) may not refer to the modified version by the same
+designation, or by any confusingly similar designation, and (ii) must refer to the underlying software originally
+provided by Alliance as “ComStock™”. Except to comply with the foregoing, the term “ComStock™”, or any confusingly
+similar designation may not be used to refer to any modified version of this software or any modified version of the
+underlying software originally provided by Alliance without the prior written consent of Alliance.
+
+4. The name of the copyright holder(s), any contributors, the United States Government, the United States Department of
+Energy, or any of their employees may not be used to endorse or promote products derived from this software without
+specific prior written permission from the respective party.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) AND ANY CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S), ANY CONTRIBUTORS, THE UNITED STATES GOVERNMENT, OR THE UNITED
+STATES DEPARTMENT OF ENERGY, NOR ANY OF THEIR EMPLOYEES, BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
+OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,29 @@
+# ComStock
+ComStock is an NREL model of the U.S. commercial building stock. The model takes some building characteristics from the
+U.S. Department of Energy's (DOE's) Commercial Prototype Building Models and Commercial Reference Building. However,
+unlike many other building stock models, ComStock also combines these with a variety of additional public- and
+private-sector data sets. Collectively, this information provides high-fidelity building stock representation with a
+realistic diversity of building characteristics.
+
+This repository contains the source code used to build and execute ComStock models, including upgrade scenarios. In
+addition, the sampling of buildings characteristics used for the initial ComStock (V1.0) release is provided.  At present
+the ComStock model is under active calibration and development, and as such this repository is not yet supported.
+
+Execution of this repo is managed through the buildstockbatch repository, a shared asset of ResStock and ComStock,
+specifically developed to scale to execution of tens of millions of simulations through multiple infrastructure
+providers.
+
+The results of the initial ComStock (V1.0) release can be found at the accompanying
+[VizStock website](https://comstock.nrel.gov) and additional information about ComStock found on the
+[NREL Buildings Website](https://www.nrel.gov/buildings/comstock.html). For more details about ongoing model development
+please consult the [End Use Load Profiles](https://www.nrel.gov/buildings/end-use-load-profiles.html) website.
+
+ComStock is a direct result of the NREL residential stock modeling tool
+[ResStock](https://www.nrel.gov/buildings/resstock.html) (recipient of a 
+[R&D100 award](https://www.rdworldonline.com/rd100/resstock-a-21st-century-tool-for-energy-efficiency-modeling-with-unparalleled-granularity/))
+and was inspired by the high-fidelity solar & storage adoption model [dGen](https://www.nrel.gov/analysis/dgen/).
+Additionally, this tool would not be possible without the decades of work undertaken by the
+[OpenStudio](https://www.openstudio.net/) and [EnergyPlus](https://energyplus.net/) visionaries and contributors,
+significant funding, feedback and support from the [Los Angeles Department of Water and Power](https://www.ladwp.com/),
+and the [Department of Energy's Building Technology Office](https://www.energy.gov/eere/buildings/building-technologies-office)
+ongoing support of and investment in building energy modeling software. 

build/Dockerfile

@@ -0,0 +1,131 @@
+# ComStock™, Copyright (c) 2020 Alliance for Sustainable Energy, LLC. All rights reserved.
+# See top level LICENSE.txt file for license terms.
+
+FROM ubuntu:16.04 AS base
+
+MAINTAINER Henry (Ry) Horsey henry.horsey@nrel.gov
+
+# If installing a CI build version of OpenStudio, then pass in the CI path into the build command. For example:
+#   docker build --build-arg DOWNLOAD_PREFIX="_CI/OpenStudio"
+ARG DOWNLOAD_PREFIX=""
+
+# Set the version of OpenStudio when building the container. For example `docker build --build-arg
+# OPENSTUDIO_VERSION=2.6.0 --build-arg OPENSTUDIO_SHA=e3cb91f98a .` in the .travis.yml. Set with the ENV keyword to
+# inherit the variables into child containers
+ARG OPENSTUDIO_VERSION
+ARG OPENSTUDIO_SHA
+ARG OS_BUNDLER_VERSION=1.17.1
+ENV OPENSTUDIO_VERSION=$OPENSTUDIO_VERSION
+ENV OPENSTUDIO_SHA=$OPENSTUDIO_SHA
+ENV OS_BUNDLER_VERSION=$OS_BUNDLER_VERSION
+
+
+# Modify the OPENSTUDIO_VERSION and OPENSTUDIO_SHA for new versions
+ENV RUBY_VERSION=2.2.4 \
+    RUBY_SHA=b6eff568b48e0fda76e5a36333175df049b204e91217aa32a65153cc0cdcb761
+
+# Don't combine with above since ENV vars are not initialized until after the above call
+ENV OPENSTUDIO_DOWNLOAD_FILENAME=OpenStudio-$OPENSTUDIO_VERSION.$OPENSTUDIO_SHA-Linux.deb
+
+# Install gdebi, then download and install OpenStudio, then clean up.
+# gdebi handles the installation of OpenStudio's dependencies including Qt5,
+# Boost, and Ruby 2.2.4.
+# OpenStudio 2.4.3 requires libwxgtk3.0-0 -- install manually for now
+
+# install locales and set to en_US.UTF-8. This is needed for running the CLI on some machines
+# such as singularity.
+RUN apt-get update && apt-get install -y autoconf \
+        build-essential \
+        ca-certificates \
+        curl \
+        gdebi-core \
+        git \
+        libfreetype6 \
+        libjpeg8 \
+        libdbus-glib-1-2 \
+        libfontconfig1 \
+        libglu1 \
+        libreadline-dev \
+        libsm6 \
+        libssl-dev \
+        libtool \
+        libwxgtk3.0-0v5 \
+        libxi6 \
+        libxml2-dev \
+        locales \
+        sudo \
+        zlib1g-dev \
+    && curl -sL https://raw.githubusercontent.com/NREL/OpenStudio-server/develop/docker/deployment/scripts/install_ruby.sh -o /usr/local/bin/install_ruby.sh \
+    && chmod +x /usr/local/bin/install_ruby.sh \
+    && /usr/local/bin/install_ruby.sh $RUBY_VERSION $RUBY_SHA \
+    && if [ -z "${DOWNLOAD_PREFIX}" ]; then \
+            export OPENSTUDIO_DOWNLOAD_URL=https://openstudio-builds.s3.amazonaws.com/$OPENSTUDIO_VERSION/OpenStudio-$OPENSTUDIO_VERSION.$OPENSTUDIO_SHA-Linux.deb; \
+       else \
+            export OPENSTUDIO_DOWNLOAD_URL=https://openstudio-builds.s3.amazonaws.com/$DOWNLOAD_PREFIX/OpenStudio-$OPENSTUDIO_VERSION.$OPENSTUDIO_SHA-Linux.deb; \
+       fi \
+    && echo "OpenStudio Package Download URL is ${OPENSTUDIO_DOWNLOAD_URL}" \
+    && curl -SLO $OPENSTUDIO_DOWNLOAD_URL \
+    # Verify that the download was successful (not access denied XML from s3)
+    && grep -v -q "<Code>AccessDenied</Code>" ${OPENSTUDIO_DOWNLOAD_FILENAME} \
+    && gdebi -n $OPENSTUDIO_DOWNLOAD_FILENAME \
+    # Cleanup
+    && rm -f /usr/local/bin/install_ruby.sh \
+    && rm -f $OPENSTUDIO_DOWNLOAD_FILENAME \
+    && rm -rf /var/lib/apt/lists/* \
+    && if dpkg --compare-versions "${OPENSTUDIO_VERSION}" "gt" "2.5.1"; then \
+            rm -rf /usr/local/openstudio-${OPENSTUDIO_VERSION}/SketchUpPlugin; \
+       else \
+            rm -rf /usr/SketchUpPlugin; \
+       fi \
+    && locale-gen en_US en_US.UTF-8 \
+    && dpkg-reconfigure locales
+
+
+## Add RUBYLIB link for openstudio.rb. Support new location and old location.
+ENV RUBYLIB=/usr/local/openstudio-${OPENSTUDIO_VERSION}/Ruby:/usr/Ruby
+ENV ENERGYPLUS_EXE_PATH=/usr/local/openstudio-${OPENSTUDIO_VERSION}/EnergyPlus/energyplus
+
+# The OpenStudio Gemfile contains a fixed bundler version, so you have to install and run specific to that version
+RUN gem install bundler -v $OS_BUNDLER_VERSION && \
+    mkdir /var/oscli && \
+    cp /usr/local/openstudio-${OPENSTUDIO_VERSION}/Ruby/Gemfile /var/oscli && \
+    cp /usr/local/openstudio-${OPENSTUDIO_VERSION}/Ruby/openstudio-gems.gemspec /var/oscli && \
+    sed -i '/openstudio-standards/d' /var/oscli/openstudio-gems.gemspec
+WORKDIR /var/oscli
+RUN OLDSTD="gem 'openstudio-standards'" && \
+    NEWSTD="gem 'openstudio-standards', github: 'NREL/openstudio-standards', branch: 'la100_merge'" && \
+    sed -i -e "s|$OLDSTD.*|$NEWSTD|g" /var/oscli/Gemfile
+RUN bundle _${OS_BUNDLER_VERSION}_ install --path=gems --jobs=4 --retry=3
+RUN rm -rf /var/oscli/gems/ruby/2.2.0/bundler/gems/openstudio-standards-05f92903f9bb/.git \
+    /var/oscli/gems/ruby/2.2.0/bundler/gems/openstudio-standards-05f92903f9bb/test \
+    /var/oscli/gems/ruby/2.2.0/cache/bundler
+
+# Configure the bootdir & confirm that openstudio is able to load the bundled gem set in /var/gemdata
+VOLUME /var/simdata/openstudio
+WORKDIR /var/simdata/openstudio
+RUN openstudio --verbose --bundle /var/oscli/Gemfile --bundle_path /var/oscli/gems openstudio_version
+
+CMD [ "/bin/bash" ]
+
+FROM ubuntu:16.04 AS cli
+
+ARG OPENSTUDIO_VERSION
+
+# copy executable and energyplus from install
+COPY --from=base /usr/local/openstudio-${OPENSTUDIO_VERSION}/bin/openstudio /usr/local/openstudio-${OPENSTUDIO_VERSION}/bin/
+COPY --from=base /usr/local/openstudio-${OPENSTUDIO_VERSION}/EnergyPlus /usr/local/openstudio-${OPENSTUDIO_VERSION}/EnergyPlus
+COPY --from=base /var/oscli/ /var/oscli/
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+            libdbus-glib-1-2 \
+            libglu1 \
+            libssl-dev \
+            libgdbm-dev \
+            libpng-dev \
+     && rm -rf /var/lib/apt/lists/*
+
+# link executable from /usr/local/bin
+RUN ln -s /usr/local/openstudio-${OPENSTUDIO_VERSION}/bin/openstudio /usr/local/bin/openstudio
+
+VOLUME /var/simdata/openstudio
+WORKDIR /var/simdata/openstudio

build/README.md

@@ -0,0 +1,53 @@
+# Creating Singularity Image
+
+* Tested with docker version 18.09.2
+
+* Remove all Docker processes, images, and volumes
+
+    ```bash
+    docker rm $(docker ps -aq)
+    docker image rm $(docker image ls -aq)
+    docker volume rm $(docker volume ls -q)
+    ```
+
+* Build Singularity/Docker Image
+
+    ```bash
+    docker build -t singularity -f singularity/Dockerfile .
+    ```
+
+* Build OpenStudio Container (Locally)
+
+    ```bash
+    # Set the version of OpenStudio to install
+    export OPENSTUDIO_VERSION=2.8.1
+    export OPENSTUDIO_SHA=6914d4f590
+
+    docker build -t docker-openstudio --build-arg OPENSTUDIO_VERSION=$OPENSTUDIO_VERSION --build-arg OPENSTUDIO_SHA=$OPENSTUDIO_SHA .
+    ```
+
+* Launch the Container (in privileged mode with docker.sock mounted in the container)
+
+    ```bash
+    docker run -it --rm --privileged -v $(pwd):/root/build -v /var/run/docker.sock:/var/run/docker.sock singularity bash
+    ```
+
+* Inside singularity build the docker container
+
+    ```bash
+
+    # start an instance of the container for export
+    docker run --name for_export docker-openstudio /bin/true
+    if [ -f docker-openstudio.simg ]; then rm -f docker-openstudio.simg; else echo "File does not exist"; fi
+    singularity image.create -s 3000 docker-openstudio.simg
+    docker export for_export | singularity image.import docker-openstudio.simg
+
+    # test singularity
+    singularity shell -B $(pwd):/singtest docker-openstudio.simg
+    bash
+    openstudio --bundle /var/oscli/Gemfile --bundle_path /var/oscli/gems gem_list
+    # then compare to SHA in Dockerfile
+
+    ```
+
+* Exit out of the container and singularity image will be in the build directory

build/singularity/Dockerfile

@@ -0,0 +1,40 @@
+# ComStock™, Copyright (c) 2020 Alliance for Sustainable Energy, LLC. All rights reserved.
+# See top level LICENSE.txt file for license terms.
+
+FROM ubuntu:18.04
+
+ENV SINGULARITY_VERSION=2.5.1
+# singularity, then other dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
+        python \
+        python-pip \
+        python-setuptools \
+        libarchive-dev \
+        squashfs-tools \
+        curl \
+        apt-transport-https \
+        ca-certificates \
+        software-properties-common \
+        gnupg \
+        vim \
+    && curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - \
+    && add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends docker-ce \
+    && rm -rf /var/lib/apt/lists/*
+
+# Build and install singularity
+WORKDIR /root/singularity-build
+RUN curl -SLO https://github.com/singularityware/singularity/releases/download/$SINGULARITY_VERSION/singularity-$SINGULARITY_VERSION.tar.gz \
+    && tar xvf singularity-$SINGULARITY_VERSION.tar.gz \
+    && cd singularity-$SINGULARITY_VERSION \
+    && ./configure --prefix=/usr/local \
+    && make \
+    && make install \
+    && singularity --version
+
+COPY singularity/requirements.txt requirements.txt
+RUN pip install -r requirements.txt
+
+WORKDIR /root/build
+CMD ['/bin/bash']

build/singularity/requirements.txt

@@ -0,0 +1,2 @@
+boto3
+requests