Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScoreL2RPN2020 used reco_powerline for score reference of 80 and not do nothing #228

Closed
marota opened this issue Jun 15, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@marota
Copy link
Contributor

marota commented Jun 15, 2021

Environment

  • Grid2op version: 1.5.2
  • System: osx ...
  • Additional system information

Bug description

In 2020 L2RPN competitions, the score rescaled used the reco_powerline agent to set base score of 80 and 100 (see for instance https://github.com/marota/Grid2Op_EnvironmentDesign/blob/master/competition_codalab/score_config.py).
rather than the do-nothing which is currently called here https://github.com/rte-france/Grid2Op/blob/097b756d5ac6a96f2eb17446b5ce749c6555452a/grid2op/utils/l2rpn_2020_scores.py#L93

How to reproduce

When running ScoreL2RPN2020 with reco_powerline with NO_OVERFLOW_DISCONNECTION": True we should have a score of 80 on a given episode.

Command line

# command line used if any 

Code snippet

import grid2op
from grid2op.Reward import RedispReward
from grid2op.utils import ScoreL2RPN2020

from grid2op.Parameters import Parameters
from grid2op.Agent import RecoPowerlineAgent

# create the backend
try:
    from lightsim2grid.LightSimBackend import LightSimBackend
    backend = LightSimBackend()
except:
    from grid2op.Backend import PandaPowerBackend
    backend = PandaPowerBackend()
    
reward = RedispReward

#############
input_dir_neurips="YourPath/input_data_val"
####

param = Parameters()
param.init_from_dict({"NO_OVERFLOW_DISCONNECTION": True})
real_env = grid2op.make(input_dir_neurips,
                        reward_class=reward,backend=backend,param=param)

###computing L2RPNScore
nb_episode=1
my_score = ScoreL2RPN2020(real_env,
                          nb_scenario=nb_episode,
                          env_seeds=[0],
                          agent_seeds=[0]
                          )

my_agent = RecoPowerlineAgent(real_env.action_space)
final_score_reco=my_score.get(my_agent)
assert(final_score_reco[0][0]==80)

Current output

The output of the code snippet above

Expected output

The expected output and/or expected behavior description
@marota marota added the bug Something isn't working label Jun 15, 2021
BDonnot referenced this issue in BDonnot/Grid2Op Jun 15, 2021
@BDonnot
Copy link
Collaborator

BDonnot commented Jun 22, 2021

Fixed and merged in version 1.6.0, now available on pypi

@BDonnot BDonnot closed this as completed Jun 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants