Skip to content

Commit

Permalink
Merge pull request #139 from BDonnot/master
Browse files Browse the repository at this point in the history 
Upgrade to version 1.2.0
  • Loading branch information
@BDonnot
BDonnot authored Aug 3, 2020
2 parents 1e56b36 + e057b90 commit 4f086db
Show file tree
Hide file tree
Showing 38 changed files with 2,379 additions and 164 deletions.
29 changes: 29 additions & 0 deletions CHANGELOG.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,35 @@ Change Log
- [???] model batteries / pumped storage in grid2op (generator but that can be charged / discharged)
- [???] model dumps (as in dump storage) in grid2op (stuff that have a given energy max, and cannot produce more than the available energy)

[1.2.0] - 2020-08-03
---------------------
- [ADDED] `ActionSpace.sample` method is now implemented
- [ADDED] DeltaRedispatchRandomAgent: that takes redispatching actions of a configurable [-delta;+delta] in MW on random generators.
- [FIXED] `Issue #129<https://github.com/rte-france/Grid2Op/issues/129>`_: game over count for env_actions
- [FIXED] `Issue #127 <https://github.com/rte-france/Grid2Op/issues/127>`_: Removed no longer existing attribute docstring `indisponibility`
- [FIXED] `Issue #133 <https://github.com/rte-france/Grid2Op/issues/133>`_: Missing positional argument `space_prng` in `Action.SerializableActionSpace`
- [FIXED] `Issue #131 <https://github.com/rte-france/Grid2Op/issues/131>`_: Forecast values are accessible without needing to call `obs.simulate` beforehand.
- [FIXED] `Issue #134 <https://github.com/rte-france/Grid2Op/issues/134>`_: Backend iadd actions with lines extremities disconnections (set -1)
- [FIXED] issue `Issue #125 <https://github.com/rte-france/Grid2Op/issues/125>`_
- [FIXED] issue `Issue #126 <https://github.com/rte-france/Grid2Op/issues/126>`_ Loading runner logs no longer checks environment actions ambiguity
- [IMPROVED] issue `Issue #16 <https://github.com/rte-france/Grid2Op/issues/16>`_ improving openai gym integration.
- [IMPROVED] `Issue #134 <https://github.com/rte-france/Grid2Op/issues/134>`_ lead us to review and rationalize the
behavior of grid2op concerning the powerline status. Now it behave more rationally and has now the following
behavior: if a powerline origin / extremity bus is "set" to -1 at one end and not modified at the other, it will disconnect this
powerline, if a powerline origin / extremity bus is "set" to 1 or 2 at one end and not modified at the other, it will
reconnect the powerline. If a powerline bus is "set" to -1 at one end and set to 1 or 2 at its other
end the action is ambiguous.
- [IMPROVED] way to count what is affect by an action (affect the cooldown of substation and powerline
and the legality of some action). And action disconnect a powerline (using the "set_bus") will be
considered to affect only
this powerline (and not on its substations) if and only if the powerline was connected (otherwise it
affects also on the substation). An action that connects a powerline (using the "set_bus") will affect
only this powerline (and not its substations) if and only if this powerline was disconnected (
otherwise it affects the substations but not the powerline). Changing the bus of an extremity of
a powerline if this powerline is connected has no impact on its status and therefor it considers
it only affects the corresponding substation.
- [IMPROVED] added documentation and usage example for `CombineReward` and `CombineScaledReward`

[1.1.1] - 2020-07-07
---------------------
- [FIXED] the EpisodeData now properly propagates the end of the episode
Expand Down
74 changes: 74 additions & 0 deletions docs/action.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,80 @@ An action can be incorrect because of two main factors:

Ambiguous or Illegal, the action will be replaced by a "do nothing" without any other incidents on the game.

Note on powerline status
------------------------
As of grid2op version 1.2.0, we attempted to clean and rationalize the API concerning the change of
powerline status (see explanatory notebook `getting_started/3_Action_GridManipulation` for more detailed
explanation.

The powerline status (connected / disconnected) can now be affected in two different ways:

- by `setting` / `changing` its status directly (using the "set_line_status" or "change_line_status" keyword).
- [NEW] by modifying the bus on any of the end (origin or extremity) of a powerline

In that later case, the behavior is:

- if the bus of a powerline end (origin or extremity) is "set" to -1 and not modified at the other and if the powerline
was connected, it will disconnect this powerline
- if the bus of a powerline end (origin or extremity) is "set" to 1 or 2 at one end and not modified at the other and
if the powerline was connected, it will reconnect the powerline
- if the bus of a powerline end (origin or extremity) is "set" to -1 at one end and set to 1 or 2 at its other end the
action is **ambiguous**.

The way to compute the impact of the action has also been adjusted to reflect these changes.

In the table below we try to summarize all the possible actions and their impact on the powerline.
This table is made considering that "`LINE_ID`" is an id of a powerline and "`SUB_OR`" is the id of the origin of the
substation. If a status is 0 it means the powerlines is disconnected, if the status is 1 it means it is connected.

============================================= ================ ============ ==================== ====================
action original status final status substations affected line status affected
============================================= ================ ============ ==================== ====================
{"set_line_status": [(LINE_ID, -1)]} 1 0 None LINE_ID
{"set_line_status": [(LINE_ID, +1)]} 1 1 None LINE_ID
{"set_line_status": [(LINE_ID, -1)]} 0 0 None LINE_ID
{"set_line_status": [(LINE_ID, +1)]} 0 1 None LINE_ID
{"change_line_status": [LINE_ID]} 1 0 None LINE_ID
{"change_line_status": [LINE_ID]} 0 1 None LINE_ID
{"set_bus": {"lines_or_id": [(LINE_ID, -1)]}} 1 0 None INE_ID
{"set_bus": {"lines_or_id": [(LINE_ID, -1)]}} 0 0 SUB_OR None
{"set_bus": {"lines_or_id": [(LINE_ID, 2)]}} 1 1 SUB_OR None
{"set_bus": {"lines_or_id": [(LINE_ID, 2)]}} 0 1 None LINE_ID
{"change_bus": {"lines_or_id": [LINE_ID]}} 1 1 SUB_OR None
{"change_bus": {"lines_or_id": [LINE_ID]}} 0 0 SUB_OR None
============================================= ================ ============ ==================== ====================

This has other impacts. In grid2op there is a convention that if an object is disconnected,
then it is assigned to bus "-1". For a powerline this entails that a status changed affects the bus of

As we explained in the previous paragraph, some action on one end of a powerline can reconnect a
powerline or disconnect it. This means they modify the bus of **both** the extremity of the powerline.

Here is a table summarizing how the buses are impacted. We denoted by "`PREVIOUS_OR`" the last bus at which
the origin end of the powerline was connected and "`PREVIOUS_EX`" the last bus at which the extremity end of the
powerline was connected. Note that for clarity when something is not modified by the action we decided to write on
the table "not modified" (this entails that after this action, if the powerline is connected then "new origin bus" is
"`PREVIOUS_OR`" and "new extremity bus" is "`PREVIOUS_EX`"). We remind the reader that "-1" encode for a
disconnected object.

============================================= ================ ============ ============== ========================
action original status final status new origin bus new extremity bus
============================================= ================ ============ ============== ========================
{"set_line_status": [(LINE_ID, -1)]} 1 0 -1 -1
{"set_line_status": [(LINE_ID, +1)]} 1 1 Not modified Not modified
{"set_line_status": [(LINE_ID, -1)]} 0 0 Not modified Not modified
{"set_line_status": [(LINE_ID, +1)]} 0 1 PREVIOUS_OR PREVIOUS_EX
{"change_line_status": [LINE_ID]} 1 0 -1 -1
{"change_line_status": [LINE_ID]} 0 1 PREVIOUS_OR PREVIOUS_EX
{"set_bus": {"lines_or_id": [(LINE_ID, -1)]}} 1 0 -1 -1
{"set_bus": {"lines_or_id": [(LINE_ID, -1)]}} 0 0 Not modified Not modified
{"set_bus": {"lines_or_id": [(LINE_ID, 2)]}} 1 1 2 Not modified
{"set_bus": {"lines_or_id": [(LINE_ID, 2)]}} 0 1 2 PREVIOUS_EX
{"change_bus": {"lines_or_id": [LINE_ID]}} 1 1 \* Not modified
{"change_bus": {"lines_or_id": [LINE_ID]}} 0 0 Not modified Not modified
============================================= ================ ============ ============== ========================

\* means that this bus is affected: if it was on bus 1 it moves on bus 2 and vice versa.

Easier actions manipulation
----------------------------
Expand Down
61 changes: 60 additions & 1 deletion docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ for example.

- abstract the computation of the cascading failures
- ability to have the same code running with multiple powerflow calculator
- parallel execution of one ageent on multiple independent scenarios (multiprocessing)
- parallel execution of one agent on multiple independent scenarios (multiprocessing)
- fully customisable: this software has been built to be fully customizable to serve different
purposes and not only reinforcement learning, or the L2RPN competition.

Expand All @@ -43,6 +43,65 @@ Plotting capabilities

plot

Compatibility with openAI gym
-----------------------------
The gym framework in reinforcement learning is widely used. Starting from version 1.2.0 we improved the
compatibility with this framework.

Before grid2op 1.2.0 only some classes fully implemented the open AI gym interface:

- the :class:`grid2op.Environment` (with methods such as `env.reset`, `env.step` etc.)
- the :class:`grid2op.Agent` (with the `agent.act` etc.)
- the creation of pre defined environments (with `grid2op.make`)


Starting from 1.2.0 we implemented some automatic converters that are able to automatically map
grid2op representation for the action space and the observation space into open AI gym "spaces". More precisely these
are represented as gym.spaces.Dict.

If you are interested by this feature, we recommend you to proceed like this:

.. code-block:: python
import grid2op
from grid2op.Converters import GymActionSpace, GymObservationSpace
from grid2op.Agent import BaseAgent
class MyAgent(BaseAgent):
def __init__(self, action_space, observation_space):
BaseAgent.__init__(self, action_space)
self.gym_obs_space = GymObservationSpace(observation_space)
self.gym_action_space = GymActionSpace(observation_space)
def act(self, obs, reward, done=False):
# convert the observation to gym like one:
gym_obs = self.gym_obs_space.to_gym(obs)
# do whatever you want, as long as you retrieve a gym-like action
gym_action = ...
grid2op_action = self.gym_action_space.from_gym(gym_action)
# NB advanced usage: if action_space is a grid2op.converter (for example coming from IdToAct)
# then what's called "grid2op_action" is in fact an action that can be understood by the converter.
# to convert it back to grid2op action you need to convert it. See the documentation of GymActionSpace
# for such purpose.
return grid2op_action
env = grid2op.make(...)
my_agent = MyAgent(env.action_space, env.observation_space)
# and now do anything you like
# for example
done = False
reward = env.reward_range[0]
obs = env.reset()
while not done:
action = my_agent.act(obs, reward, done)
obs, reward, done, info = env.step(action)
We also implemented some "converter" that allow the conversion of some action space into more convenient
`gym.spaces` (this is only available if gym is installed of course). Please check
:class:`grid2op.Converter.GymActionSpace` for more information and examples.

Technical Documentation
----------------------------

Expand Down
Loading

0 comments on commit 4f086db

Please sign in to comment.