Merge pull request #22 from BDonnot/v0.7.1

ready for v0.7.1

.circleci/config.yml

@@ -3,7 +3,7 @@ version: 2.1
 executors:
   grid2op-executor:
     docker:
-      - image: bdonnot/grid2op:latest
+      - image: bdonnot/grid2op:test
     working_directory: /Grid2Op
 
 jobs:

.gitignore

@@ -170,7 +170,7 @@ getting_started/study_agent_getting_started/
 **Untitled.ipynb
 grid2op/tests/test_agent/
 grid2op/tests/start_datetime.info
-grid2op/data/*/chronics/
+# grid2op/data/*/chronics/
 
 l2rpn_2019/data/
 l2rpn_2019/expe_saved/
@@ -190,4 +190,6 @@ getting_started/awesome_agent_logs/
 getting_started/path_agents/
 **Test.ipynb
 baselines/
+test_saving_chron_proper_format/
+test_split_and_save.py
 

CHANGELOG.rst

@@ -8,10 +8,9 @@ Change Log
 - [???] test and doc for opponent
 - [???] better logging
 - [???] rationalize the public and private part of the API. Some members now are public but should be private.
-
+- [???] refacto the backend and the actions, backend you get set only, env takes care of the rest
+- [???] refacto the dataset handling
 - [???] better explanation of the notebook 3 with action silently
-- [???] have something remembering the topology in the environment, and when an object is
-  reconnected, and no buses are specified, then it connects it to last buses.
 - [???] see if "simulate" performances can be improved, and performances in general.
 - [???] simulate in MultiEnv
 - [???] in MultiEnv, when some converter of the observations are used, have each child process to compute
@@ -22,6 +21,26 @@ Change Log
 - [???] modeled dumps in grid2op (stuff that have a given energy max, and cannot produce more than the available energy)
 - [???] fix notebook 5 texts
 
+
+[0.7.1] - 2020-04-xx
+----------------------
+- [FIXED] a bug in the chronics making it not start at the appropriate time step
+- [FIXED] a bug in "OneChangeThenNothing" agent that prevent it to be restarted properly.
+- [FIXED] a bug with the generated docker file that does not update to the last version of the package.
+- [FIXED] numpy, by default does not use the same datatype depending on the platform. We ensure that
+  floating value are always `np.float32` and integers are always `np.int32`
+- [ADDED] a method to extract only some part of a chronic.
+- [ADDED] a method to "fast forward" the chronics
+- [ADDED] class `grid2op.Reward.CombinedScaledReward`: A reward combiner with linear interpolation to stay within a given range.
+- [ADDED] `grid2op.Reward.BaseReward.set_range`: All rewards have a default setter for their `reward_min` and `reward_max` attributes.
+- [ADDED] `grid2op.PlotGrid`: Revamped plotting capabilities while keeping the interface we know from `grid2op.Plot`
+- [ADDED] `grid2op.replay` binary: This binary is installed with grid2op and allows to replay a runner log with visualization and gif export
+- [ADDED] a `LicensesInformation` file that put a link for all dependencies of the project.
+- [ADDED] make multiple dockers, one for testing, one for distribution with all extra, and one "light"
+- [UPDATED] test data and datasets are no longer included in the package distribution
+- [UPDATED] a new function `make_new` that will make obsolete the "grid2op.download" script in future versions
+- [UPDATED] the python "requests" package is now a dependency
+
 [0.7.0] - 2020-04-15
 --------------------
 - [BREAKING] class `grid2op.Environment.BasicEnv` has been renamed `BaseEnv` for consistency. As this class

Dockerfile

@@ -24,19 +24,23 @@ RUN apt-get update && \
     ca-certificates
 
 # Retrieve Grid2Op
-RUN git clone https://github.com/rte-france/Grid2Op
+RUN git clone https://github.com/bdonnot/Grid2Op
 
 # Install Grid2Op
 WORKDIR /Grid2Op
 # Use the latest release
 RUN git pull
-RUN git fetch --all --tags
-RUN git checkout "tags/v0.7.0" -b "v0.7.0-branch"
+RUN git remote update
+RUN git fetch --all --tags --force
+RUN git pull origin 071_dev
 # Install Dependencies
-RUN pip3 install -U .
-RUN pip3 install -U .[optional]
 RUN pip3 install -U .[test]
-RUN pip3 install -U .[challenge]
+# force the "install" of test data in proper repo
+WORKDIR /
+RUN python3 -c "import grid2op; import os; print(os.path.split(grid2op.__file__)[0])" > cptest_data
+WORKDIR /Grid2Op
+RUN tail -n 1 /cptest_data | xargs cp -R grid2op/data_test
+RUN rm /cptest_data
 WORKDIR /
 
 # Make port 80 available to the world outside this container

LicensesInformation.md

@@ -0,0 +1,82 @@
+Copyright (c) 2019-2020, RTE (https://www.rte-france.com)
+
+See [AUTHORS.txt](AUTHORS.txt)
+
+This Source Code Form is subject to the terms of the Mozilla Public License, version 2.0.
+
+If a copy of the Mozilla Public License, version 2.0 was not distributed with this file,
+
+you can obtain one at http://mozilla.org/MPL/2.0/.
+
+SPDX-License-Identifier: MPL-2.0
+
+This file is part of Grid2Op, Grid2Op a testbed platform to model sequential decision making in power systems.
+
+# Project dependencies
+
+In this file, we try to recap the licenses of the project on which grid2op depends on. This file is NOT a substitute
+for the [LICENSE](LICENSE.md) files that presents the terms of the MPLv2.0 license, license under which this
+project is released.
+
+## Base package (required)
+
+|    Package |      License     |
+|------------|:----------------:|
+| numpy      |    [BSD](https://numpy.org/license.html)           |  
+| scipy      | [BSD-3-Clause "New"](https://www.scipy.org/scipylib/license.html)  |
+| pandas     | [BSD-3-Clause "New"](https://github.com/pandas-dev/pandas/blob/master/LICENSE) |
+| pandapower | [BSD 3-Clause "New"](https://pandapower.readthedocs.io/en/v1.4.1/about/license.html)|
+| tqdm       |    [MPL 2.0](https://github.com/tqdm/tqdm/blob/master/LICENCE)       | 
+| pathlib    |      [MIT](https://en.wikipedia.org/wiki/MIT_License)         |
+| networkx   | [BSD 3-Clause "New"](https://networkx.github.io/documentation/networkx-1.10/reference/legal.html)  |
+| requests   |[Apache License 2.0](https://github.com/psf/requests/blob/master/LICENSE)|
+| imageio    |   [BSD-2-Clause "Simplified"](https://github.com/imageio/imageio/blob/master/LICENSE)   |
+| matplotlib |      [PSF](https://matplotlib.org/3.2.1/users/license.html)         |            
+
+## Optional packages
+
+### Extra package (optional)
+
+|    Package    |      License     |
+|---------------|:----------------:|
+| nbformat      |    [BSD](https://fr.wikipedia.org/wiki/Licence_BSD) |  
+| jupyter-client|  [BSD 3-Clause "New"](https://github.com/jupyter/jupyter/blob/master/LICENSE) |
+| jyquickhelper |  [MIT](https://github.com/sdpython/jyquickhelper/blob/master/LICENSE.txt) |
+
+### Optional (optional)
+
+|    Package |      License     |
+|------------|:----------------:|
+| numba   |     [BSD 2-Clause "Simplified"](https://github.com/numba/numba/blob/master/LICENSE)          |  
+| plotly      | [MIT](https://github.com/plotly/plotly.js/blob/master/LICENSE)  |
+| seaborn     | [BSD 3-Clause "new"](https://github.com/mwaskom/seaborn/blob/master/LICENSE) |
+| pygifsicle  |  [MIT](https://en.wikipedia.org/wiki/MIT_License) | 
+| psutil      |  [BSD 3-Clause "new"](https://github.com/giampaolo/psutil/blob/master/LICENSE)|
+
+### deprecated (optional)
+
+|    Package |      License     |
+|------------|:----------------:|
+| pygame   |       [GNU LGPL v2.1](https://www.gnu.org/copyleft/lesser.html)      |  
+| imageio-ffmpeg |  [BSD 2-Clause "Simplified"](https://github.com/imageio/imageio-ffmpeg/blob/master/LICENSE)|
+
+### doc (optional)
+
+|    Package |      License     |
+|------------|:----------------:|
+| numpydoc | [BSD](https://github.com/numpy/numpydoc/blob/master/LICENSE.txt) |
+| sphinx  | [BSD](https://github.com/sphinx-doc/sphinx/blob/3.x/LICENSE)|
+| sphinx-rtd-theme | [MIT](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/LICENSE) |
+| sphinxcontrib-trio | [MIT or Apache 2](https://github.com/python-trio/sphinxcontrib-trio/blob/master/LICENSE) |
+| autodocsumm | [GPL v2](https://github.com/Chilipp/autodocsumm/blob/master/LICENSE)|
+
+### challenge (optional)
+
+|    Package |      License     |
+|------------|:----------------:|
+| tensorflow | [Apache License 2.0](https://github.com/tensorflow/tensorflow/blob/master/LICENSE) |
+| Keras | [MIT](https://github.com/keras-team/keras/blob/master/LICENSE) |
+| torch | [BSD 3-Clause "New"](https://github.com/intel/torch/blob/master/LICENSE.md)|
+| statsmodels | [BSD](https://github.com/statsmodels/statsmodels/blob/master/LICENSE.txt)|
+| scikit-learn | [BSD 3-clauses](https://fr.wikipedia.org/wiki/Licence_BSD) |
+| gym | [MIT](https://github.com/openai/gym/blob/master/LICENSE.md) |

docs/makeenv.rst

@@ -1,9 +1,63 @@
+.. currentmodule:: grid2op.MakeEnv
+
 Using pre defined Environments
 ===================================
 
-.. include:: global.rst
+Objectives
+-----------
+The function define in this module is the easiest and most convenient ways to create a valid
+:class:`grid2op.Environment.Environment`.
+
+To get started with such an environment, you can simply do:
+
+..code-block:: python
+
+    import grid2op
+    env = grid2op.make()
+
+
+You can consult the different notebooks in the `getting_stared` directory of this package for more information on
+how to use it.
+
+Created Environment should behave exactly like a gym environment. If you notice any unwanted behavior, please address
+an issue in the official grid2op repository: `Grid2Op <https://github.com/rte-france/Grid2Op>`_
+
+The environment created with this method should be fully compatible with the gym framework: if you are developing
+a new algorithm of "Reinforcement Learning" and you used the openai gym framework to do so, you can port your code
+in a few minutes (basically this consists in adapting the input and output dimension of your BaseAgent) and make it work
+with a Grid2Op environment. An example of such modifications is exposed in the getting_started/ notebooks.
+
+Important notes
+---------------
+As of version 0.7.1 a new function called ":func:`make_new`" has been developed in grid2op. This function, which will
+replace the current implementation of :func:`make` in future versions, merges the behaviour of "grid2op.download"
+script and "make" function.
+
+It has the following behavior:
+
+1) if you specify a full path to a local environment (containing the chronics and the default parameters),
+   it will be used
+2) if you specify the name of an environmnet that you have already downloaded, it will use this environment (NB
+   currently no checks are implemented if the environment has been updated remotely, which can happen if
+   we realize there were some issues with it.)
+3) if the flag `local` is set to ``False`` (default behaviour) and none of the above conditions are met, the
+   :func:`make_new` will download the data of this environment locally the first time it is called. If you don't want
+   do download anything then you can pass the flag ``local=True``
+4) if ``local=True`` (NON default behaviour) nothing will be loaded, and the :func:`make_new` will attempt to use a
+   pre defined environment provided with the python package. We want to emphasize that because the environments provided
+   with this package contains only little data, they are not suitable for leaning a consistent agent / controler. That
+   is why a warning is sent in this case. Also, keep in mind that if you don't pass ``local=True`` then you will not
+   have the possibility to search for these environments provided in the package.
+5) if nothing is found, :func:`make_new` throws a EnvError.
+
+TODO: explain behaviour of ".grid2opconfig.json"
+
+
+Detailed Documentation by class
+--------------------------------
 
 .. automodule:: grid2op.MakeEnv
-   :members:
+    :members:
+    :autosummary:
 
 .. include:: final.rst