Issue591 add semantic tags

.travis.yml

@@ -74,22 +74,22 @@ jobs:
     script: cd testing && make build_jm_image && make test_twozone_apartment_hydronic
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air_travis ARG="-s test_peak_heat_day,test_typical_heat_day"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air ARG="-s test_peak_heat_day,test_typical_heat_day"
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air_travis ARG="-s test_peak_cool_day,test_typical_cool_day,test_mix_day"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air ARG="-s test_peak_cool_day,test_typical_cool_day,test_mix_day"
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air_travis ARG="-s API"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_office_simple_air ARG="-s API"
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic_travis ARG="-s test_peak_heat_day,test_shoulder"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic ARG="-s test_peak_heat_day,test_shoulder"
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic_travis ARG="-s test_typical_heat_day,test_summer"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic ARG="-s test_typical_heat_day,test_summer"
   - python: 3.9
     install: pip install --upgrade pip && pip install pandas==1.2.5 numpy==1.20.2 matplotlib==3.3.4 requests==2.25.1
-    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic_travis ARG="-s API"
+    script: cd testing && make build_jm_image && travis_wait 90 make test_multizone_residential_hydronic ARG="-s API"
   - python: 2.7
     install: pip install --upgrade pip && pip install pandas==0.24.2 numpy==1.16.6 matplotlib==2.1.1 requests==2.18.4
     script: cd testing && make test_python2

Dockerfile

@@ -1,4 +1,4 @@
-FROM ubuntu:20.04
+FROM --platform=linux/x86_64 ubuntu:20.04
 
 # Install required packages
 RUN 	apt-get update && \

README.md

@@ -68,7 +68,7 @@ Example RESTful interaction:
 
 | Interaction                                                           | Request                                                   |
 |-----------------------------------------------------------------------|-----------------------------------------------------------|
-| Advance simulation with control input and receive measurements.        |  POST ``advance`` with optional json data "{<input_name>:<value>}" |
+| Advance simulation with control input and receive measurements.        |  POST ``advance`` with optional arguments ``<input_name_u>:<value>``, and corresponding ``<input_name_activate>:<0 or 1>``, where 1 enables value overwrite and 0 disables (0 is default)  |
 | Initialize simulation to a start time using a warmup period in seconds.  Also resets point data history and KPI calculations.     |  PUT ``initialize`` with required arguments ``start_time=<value>``, ``warmup_period=<value>``|
 | Receive communication step in seconds.                                 |  GET ``step``                                             |
 | Set communication step in seconds.                                     |  PUT ``step`` with required argument ``step=<value>``              |

contributors.md

@@ -19,6 +19,7 @@ Thank you to all who have provided guidance on the development of this software.
 - Robert Lutes, Pacific Northwest National Laboratory
 - Kefei Mo, Pacific Northwest National Laboratory
 - Erik Paulson, Independent
+- Matt Robinson, University of Colorado - Boulder
 - Jermy Thomas, National Renewable Energy Laboratory
 - Christian Veje, University of Southern Denmark
 - Draguna Vrabie, Pacific Northwest National Laboratory

data/data_manager.py

@@ -333,20 +333,36 @@ def get_data(self, horizon=24*3600, interval=None, index=None,
             # the closest possible point under stop will be the end
             # point in order to keep interval unchanged among index.
             index = np.arange(start,stop+0.1,interval).astype(int)
+        else:
+            if not isinstance(index, np.ndarray):
+                index = np.asarray(index)
 
         # Reindex to the desired index
-        data_slice_reindexed = data_slice.reindex(index)
-
-        for key in data_slice_reindexed.keys():
-            # Use linear interpolation for continuous variables
-            if key in self.categories['weather']:
-                f = interpolate.interp1d(self.case.data.index,
-                    self.case.data[key], kind='linear')
-            # Use forward fill for discrete variables
-            else:
-                f = interpolate.interp1d(self.case.data.index,
-                    self.case.data[key], kind='zero')
-            data_slice_reindexed.loc[:,key] = f(index)
+        start = index[0]
+        # 1 year in (s)
+        year = 31536000
+        # Starting year
+        year_start = int(np.floor(start/year))*year
+        # Normalizing index with respect to starting year
+        index_norm = index - year_start
+        stop_norm = index_norm[-1]
+        # If stop happens across the year divide df and interpolate separately
+        if stop_norm > data_slice.index[-1]:
+            idx_year = (np.abs(index_norm - year)).argmin() + 1
+            # Take previous index value if index at idx_year > year
+            if index_norm[idx_year - 1] - year > np.finfo(float).eps:
+                idx_year = idx_year -1
+            df_slice1 = data_slice.reindex(index_norm[:idx_year])
+            df_slice1 = self.interpolate_data(df_slice1,index_norm[:idx_year])
+            df_slice2 = data_slice.reindex(index_norm[idx_year:] - year)
+            df_slice2 = self.interpolate_data(df_slice2,index_norm[idx_year:] - year)
+            df_slice2.index = df_slice2.index + year
+            data_slice_reindexed = pd.concat([df_slice1,df_slice2])
+        else:
+            data_slice_reindexed = data_slice.reindex(index_norm)
+            data_slice_reindexed = self.interpolate_data(data_slice_reindexed,index_norm)
+        # Add starting year back to index desired by user
+        data_slice_reindexed.index = data_slice_reindexed.index + year_start
 
         if plot:
             if category is None:
@@ -504,6 +520,33 @@ def get_data_metadata(self):
 
         return data_metadata
 
+    def interpolate_data(self,df,index):
+        '''Interpolate testcase data.
+
+        Parameters
+        ----------
+        df: pandas.DataFrame
+            Dataframe that needs to be interpolated
+        index: np.array()
+            Index to use to get interpolated data
+
+        Returns
+        -------
+        df: pandas.DataFrame
+            Interpolated dataframe
+
+        '''
+        for key in df.keys():
+            # Use linear interpolation for continuous variables
+            if key in self.categories['weather']:
+                f = interpolate.interp1d(self.case.data.index,
+                    self.case.data[key], kind='linear')
+            # Use forward fill for discrete variables
+            else:
+                f = interpolate.interp1d(self.case.data.index,
+                    self.case.data[key], kind='zero')
+            df.loc[:,key] = f(index)
+        return df
 
 if __name__ == "__main__":
     import sys

data/get_html_IO.py

@@ -7,10 +7,9 @@
 2. Run BOPTEST test case on localhost:5000
 3. Run this script
 
-Outputs:
-"inputs.txt": html code documenting the inputs
-"measurements.txt": html code documenting the outputs
-
+Output:
+"inputs_measurements_forecasts.html" html code documenting inputs, outputs and 
+forecasts together
 """
 
 # GENERAL PACKAGE IMPORT
@@ -40,24 +39,40 @@ def run():
 
     # GET TEST INFORMATION
     # --------------------
+    # Create single I/O file
     # Inputs available
     inputs = requests.get('{0}/inputs'.format(url)).json()['payload']
-    with open('inputs.txt', 'w') as f:
+    with open('inputs_measurements_forecasts.html', 'w') as f:
+        f.write('<h3>Model IO\'s</h3>\n')
+        f.write('<h4>Inputs</h4>\n')
+        f.write('The model inputs are:\n')
+        f.write('<ul>\n')
         for i in sorted(inputs.keys()):
             if 'activate' not in i:
                 f.write('<li>\n<code>{0}</code> [{1}] [min={2}, max={3}]: {4}\n</li>\n'.format(i,inputs[i]['Unit'],inputs[i]['Minimum'], inputs[i]['Maximum'], inputs[i]['Description']))
+            else:
+                f.write('<li>\n<code>{0}</code> [1] [min=0, max=1]: Activation signal to overwrite input {1} where 1 activates, 0 deactivates (default value)\n</li>\n'.format(i,i.replace('activate','')+'u'))
+        f.write('</ul>\n')
     # Measurements available
     measurements = requests.get('{0}/measurements'.format(url)).json()['payload']
-    with open('measurements.txt', 'w') as f:
+    with open('inputs_measurements_forecasts.html', 'a') as f:
+        f.write('<h4>Outputs</h4>\n')
+        f.write('The model outputs are:\n')
+        f.write('<ul>\n')
         for i in sorted(measurements.keys()):
             if 'activate' not in i:
                 f.write('<li>\n<code>{0}</code> [{1}] [min={2}, max={3}]: {4}\n</li>\n'.format(i,measurements[i]['Unit'],measurements[i]['Minimum'], measurements[i]['Maximum'], measurements[i]['Description']))
+        f.write('</ul>\n')
     # Forecasts available
     forecast_points = requests.get('{0}/forecast_points'.format(url)).json()['payload']
-    with open('forecast_points.txt', 'w') as f:
+    with open('inputs_measurements_forecasts.html', 'a') as f:
+        f.write('<h4>Forecasts</h4>\n')
+        f.write('The model forecasts are:\n')
+        f.write('<ul>\n')
         for i in sorted(forecast_points.keys()):
             if 'activate' not in i:
                 f.write('<li>\n<code>{0}</code> [{1}]: {2}\n</li>\n'.format(i,forecast_points[i]['Unit'],forecast_points[i]['Description']))
+        f.write('</ul>\n')
     # --------------------
 
 if __name__ == "__main__":

docs/design/source/forecasts.rst

@@ -3,10 +3,13 @@
 Forecast Generation
 ===================
 
+Forecaster Module
+-----------------
+
 A forecaster module is developed to retrieve forecast from a BOPTEST
 testcase, which is needed for predictive controllers. This module uses the
-data_manager object of a test case to read the deterministic forecast from
-the testcase wrapped.fmu. In future developments it will be possible to
+:code:`data_manager` object of a test case to read the deterministic forecast from
+the testcase :code:`wrapped.fmu`. In future developments it will be possible to
 request stochastic forecast with a predefined distribution over the
 deterministic forecast for research purposes. This distribution will be
 added on the top of the deterministic forecast mentioned before.
@@ -15,3 +18,30 @@ The controller developer can choose the prediction horizon and interval of
 the forecast from the actual simulation time. The controller developer may
 also filter the forecast for a specific data category or request all data
 variables and filter it afterwards.
+
+Getting Weather Forecasts Across Year-End
+-----------------------------------------
+
+The data in TMY weather files used in BOPTEST test cases are discontinuous
+at the end of the year relative to the start of the year.
+Therefore, so is the weather data in the .csv files supplied for weather
+forecast generation.  As an example, see the relative humidity in the
+figure below (orange line). If weather forecasts are asked for that cross
+year-end, the :code:`data_manager` object used by the forcaster module splits the
+data at year-end into one portion that is inclusive of the last data point
+at the end of the year (midnight), and one portion after the end of the
+year that is not inclusive of the first data point at the start of the
+year (midnight). The implementation is done this way so that the forecast
+is more consistent for any interval through the full first year if a user
+only intends to simulate one year. The relative humidity plot below shows
+the interpolation behavior of the implementation graphically for forecast
+intervals of 1800s and 123s (intervals used in the unit tests), compared
+to the reference weather boundary condition data at 1800s intervals
+in the .csv file.
+
+.. figure:: images/relative_humidity_over_year_sim.png
+    :scale: 50 %
+
+    Forecast retrieved across one year for intervals of 1800s (blue) and
+    123s (yellow) compared to the reference data in the boundary condition
+    .csv file (orange), shown for relative humidity.

docs/design/source/testcasedev.rst

@@ -54,7 +54,8 @@ following structure:
 	|  |  |  |--prices.csv 		// Energy pricing schedules
 	|  |  |  |--emissions.csv 	// Energy emission factor schedules
 	|  |  |  |--setpoints.csv 	// Thermal  and IAQ comfort region schedules
-	|  |--bacnet.ttl        // BACnet object definition file
+	|  |--bacnet.ttl        	// BACnet object definition file
+	|  |--semantic_model.ttl
 	|--doc				// Documentation directory
 	|  |--doc.html 			// Copy of .mo file documentation
 	|  |--images 			// Image directory
@@ -146,7 +147,7 @@ The second function is to export a wrapper FMU that utilizes the signal exchange
 
 4. Add one output for every Read block found named :code:`<block_instance_path>_y`.  Assign :code:`<block_instance_path>_y` the unit, descriptions, min/max, and other signal attribute data specified by the Read block.
 
-5. For Overwrite blocks, connect :code:`<block_instance_path>_u` to :code:`<block.instance.path>.u`, :code:`<block_instance_path>_activate` to :code:`<block.instance.path>.activate`, and :code:`<block_instance_path>_y` to :code:`<block.instance.path>.y`.
+5. For Overwrite blocks, connect :code:`<block_instance_path>_u` to :code:`<block.instance.path>.uExt.y`, :code:`<block_instance_path>_activate` to :code:`<block.instance.path>.activate.y`, and :code:`<block_instance_path>_y` to :code:`<block.instance.path>.y`.
 
 6. For Read blocks, connect :code:`<block_insance_path>_y` to :code:`<block.instance.path>.y`.
 
@@ -307,6 +308,18 @@ The file can be created automatically using the script
 :code: `bacnet/create_ttl.py` located in the IBPSA github repository.
 
 
+Semantic model .ttl generation
+--------------------------------------------
+In order to enable a semantic application to read and write to BOPTEST points, a
+:code:`semantic_model.ttl` file is available in the testcase :code:`models` directory. This
+model can be generated by parsing the semantic annotations present in the testcase model
+with :code:`modelica-json` (https://github.com/lbl-srg/modelica-json) and finilize the .ttl file
+with the python script present in the testcase folder.
+The semantic tagging in the model is done on the :code:`read` and :code:`overwrite` blocks
+where the points are assigned using the Brick schema https://brickschema.org/ and the model
+equipement is automatically inferred using modelica-json and the custom python script.
+
+
 Data Generation and Collection Module
 -------------------------------------
 

examples/javascript/Dockerfile

@@ -4,18 +4,21 @@ ARG testcase
 
 RUN apt-get update && \
     apt-get install -y \
+	python3.7 \
 	python3-pip \
 	firefox \
 	wget
 
-RUN pip3 install splinter urllib3 pandas selenium
+RUN python3.7 -m pip install --upgrade pip
+
+RUN python3.7 -m pip install splinter urllib3 pandas selenium
 
 ENV PATH $PATH:/home
 
 WORKDIR /home
 
 RUN cd /home && \
-    wget https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz && \
+    wget https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz && \
 	tar -xvzf geckodriver*  && \
 	rm geckodriver-*
 

examples/javascript/makefile

@@ -16,7 +16,7 @@ remove-image:
 
 run:
 	$(COMMAND_RUN) && \
-	docker exec -it ${IMG_NAME} /bin/bash -c "python3 wrapper.py ${Script}" && \
+	docker exec -it ${IMG_NAME} /bin/bash -c "python3.7 wrapper.py ${Script}" && \
 	make copy_from_container ARGS=. && \
 	docker stop ${IMG_NAME}
 
@@ -25,3 +25,4 @@ copy_from_container:
 
 stop:
 	docker stop ${Script}
+

releasenotes.md

@@ -7,6 +7,17 @@ Released on xx/xx/xxxx.
 **The following changes are backwards-compatible and do not significantly change benchmark results:**
 
 - Add materials for RLEM23 workshop at ``docs/workshops/RlemWorkshop_20231112``. This is for [#585](https://github.com/ibpsa/project1-boptest/issues/585).
+- Change JModelica docker container address  in ``testing/Dockerfile``. This is for [#590](https://github.com/ibpsa/project1-boptest/issues/590).
+- Specify the Python version (3.7) used for building the wrapper to execute the example JavaScript controllers in the unit test. This is for [#594](https://github.com/ibpsa/project1-boptest/issues/594).
+- Allow simulations and forecast to work across the end of the year to the next year. This is for [#239](https://github.com/ibpsa/project1-boptest/issues/239).
+- Pin base Docker image to ``linux/x86_64`` platform. This is for [#608](https://github.com/ibpsa/project1-boptest/issues/608).
+- Correct typo in design documentation about connecting inputs to overwrite blocks in wrapper model. This is for [#601](https://github.com/ibpsa/project1-boptest/issues/601).
+- Add Git LFS configuration in the ``testing/Dockerfile`` image used in tests and compilation. This is for [#613](https://github.com/ibpsa/project1-boptest/issues/613).
+- Correct typo in documentation for ``multizone_office_simple_air``, cooling setback temperature changed from 12 to 30. This is for [#605](https://github.com/ibpsa/project1-boptest/issues/605).
+- Modify unit tests for test case scenarios to only simulate two days after warmup instead of the whole two-week scenario. This is for [#576](https://github.com/ibpsa/project1-boptest/issues/576).
+- Fix unit tests for possible false passes in certain test cases. This is for [#620](https://github.com/ibpsa/project1-boptest/issues/620).
+- Add ``activate`` control inputs to all test case documentation and update ``get_html_IO.py`` to print one file with all inputs, outputs, and forecasts. This is for [#555](https://github.com/ibpsa/project1-boptest/issues/555).
+- Add storing of scenario result trajectories, kpis, and test information to simulation directory within test case docker container. This is for [#626](https://github.com/ibpsa/project1-boptest/issues/626).
 
 
 ## BOPTEST v0.5.0