Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/20230828 fix evaluation script #107

Merged
merged 3 commits into from
Sep 12, 2023
Merged

Feature/20230828 fix evaluation script #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4adaa11
Fix properly apply save-dir option
ishihara-y Aug 28, 2023
0028b64
Add batch-size option
ishihara-y Aug 28, 2023
0a1eaf6
Fix copyright
ishihara-y Aug 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions bin/evaluate_algorithm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright 2021,2022 Sony Group Corporation.
# Copyright 2021,2022,2023 Sony Group Corporation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -132,6 +132,8 @@ do
SAVE_DIR="$1";;
"--num_seeds" )
NUM_SEEDS="$1";;
"--batch_size" )
BATCH_SIZE="$1";;
"-h" )
echo ""
echo "[Usage] : ./evaluate_algo
Expand All @@ -142,6 +144,7 @@ do
--env_end_index <env_end_index>
--envs <env1> <env2> <env3>
--save_dir <save_dir>
--batch_size <batch_size>

e.g. ./evaluate_algo --algo_name dqn --base_env_name atari --env_start_index 0 --env_end_index 1 --save_dir sample
or if you want directly specify the training envs
Expand All @@ -156,7 +159,11 @@ do
done

REPRODUCTION_CODE_DIR="${ROOT_DIR}/reproductions/algorithms/${BASE_ENV_NAME}/${ALGO_NAME}"
SAVE_DIR=$REPRODUCTION_CODE_DIR
if [ -n "$SAVE_DIR" ]; then
SAVE_DIR=$REPRODUCTION_CODE_DIR/$SAVE_DIR
else
SAVE_DIR=$REPRODUCTION_CODE_DIR
fi

for ENV_NAME in ${ENVS}
do
Expand All @@ -179,5 +186,9 @@ do
ENV_NAME=${DELAYED_MUJOCO_ENV_LIST[$INDEX]}
fi
echo "Start running training for: " ${ENV_NAME}
${ROOT_DIR}/bin/train_with_seeds "${REPRODUCTION_CODE_DIR}/${ALGO_NAME}_reproduction.py" $GPU_ID $ENV_NAME $SAVE_DIR $NUM_SEEDS &
if [ -n "$BATCH_SIZE" ]; then
${ROOT_DIR}/bin/train_with_seeds "${REPRODUCTION_CODE_DIR}/${ALGO_NAME}_reproduction.py" $GPU_ID $ENV_NAME $SAVE_DIR $NUM_SEEDS $BATCH_SIZE &
else
${ROOT_DIR}/bin/train_with_seeds "${REPRODUCTION_CODE_DIR}/${ALGO_NAME}_reproduction.py" $GPU_ID $ENV_NAME $SAVE_DIR $NUM_SEEDS &
fi
done
18 changes: 13 additions & 5 deletions bin/train_with_seeds
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright 2021,2022 Sony Group Corporation.
# Copyright 2021,2022,2023 Sony Group Corporation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -13,8 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

if [ $# -ne 5 ]; then
echo "usage: $0 <script_file_name> <gpu_id> <env> <save_dir> <num_seeds>"
if [ $# -lt 5 ] && [ $# -gt 6 ]; then
echo "usage: $0 <script_file_name> <gpu_id> <env> <save_dir> <num_seeds> <batch_size>"
exit 1
fi
RESULTDIR="$4/$3_results"
Expand All @@ -23,11 +23,19 @@ NUM_SEEDS=$5
if [ $NUM_SEEDS -eq 3 ]; then
for seed in 1 10 100
do
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4
if [ $# -eq 5 ]; then
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4
else
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4 --batch-size $6
fi
done
else
for seed in $(seq 1 $NUM_SEEDS);
do
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4
if [ $# -eq 5 ]; then
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4
else
python $1 --gpu $2 --seed $seed --env $3 --save-dir $4 --batch-size $6
fi
done
fi