Application/traffic_light_control #665

wangzelong0663 · 2021-06-23T04:51:04Z

reproduce the traffic_light_control methods of presslight and FRAP based on paddle.

zenghsh3 · 2021-07-05T07:48:00Z

examples/Application/Traffic-Light-Control/README.md

+
+Note that for the method `sotl`, different `t_min`, `min_green_vehicle` and `max_red_vehicle` configs may cause huge different results, which may not fair for sotl to compare its result with others, so we don't list the result of the `sotl` above.
+
+And results of the last two rows of the table ,`presslight*` and `FRAP*`, they are the results of the code [tlc-baselines](https://github.com/gjzheng93/tlc-baselines) provided from the paper authors' team. We run the [code](https://github.com/gjzheng93/tlc-baselines) just changing the yellow time and the action intervals to keep them same as our config as the papers without changing any other parameters. `--` in the table means the origins code doesn't perform well in the last four `anon_4X4` datas, the average travel time results of it will be more than 1000, maybe it will perform better than the `max_pressure`if you modify the other hyperparameters of the DQN agents, such as the buffer size, update_model_freq, the gamma or others.


yellow time -> yellow signal time

zenghsh3 · 2021-07-05T07:48:44Z

examples/Application/Traffic-Light-Control/world.py

+
+        # define yellow phases, currently the default yellow phase is 0, so make sure the first phase of the roadnet is yellow phase
+        self.yellow_phase_id = [0]
+        # the default time of the yellow time is 5 seconds, you can change it to the real case.


yellow time -> yellow signal

zenghsh3 · 2021-07-05T07:50:07Z

examples/Application/Traffic-Light-Control/README.md

+ cityflow==0.1
+
+### Training 
+First, download the data from [here](https://traffic-signal-control.github.io/) or [MPLight data](https://github.com/Chacha-Chen/MPLight/tree/master/data) and put them in the `data` directory. And the run the training script. The `train_presslight.py `for the presslight, each intersection has its own model as default(you can also choose to train with that all the intersections share one model in the script, just as what the paper MPLight used, it is suggested when the number of the intersections is large, just setting the `--is_share_model` to `True`).


And the run the training script -> And run the training script

zenghsh3 · 2021-07-05T07:52:53Z

examples/Application/Traffic-Light-Control/README.md

+We don't use the distributed traing or the parallel actors for collect the datas from the cityflow env, if you want to use the parallel actors with the cluster, you can refer to [here](https://github.com/PaddlePaddle/PARL/tree/develop/examples/A2C) or our [documentation](https://parl.readthedocs.io/en/latest/parallel_training/setup.html) for details. 
+
+### Some Suggestions and Conclusions
+ The classic method `max_pressure`, `solt` or `greedy`(just set green lights to the roads with the most vehicles) can get the not bad baselines, when you use the RL method, you can compare to those baselines to make sure there is no mistakes in the RL code and the training process.


there is no mistakes -> there are no mistakes

zenghsh3 · 2021-07-05T08:30:55Z

examples/Application/Traffic-Light-Control/ddqn.py

@@ -0,0 +1,95 @@
+#   Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.


Can we use the parl.algorithms.DDQN directly?

There are some tricks used in the ddqn.py, such as grad clip, epsilon decay ,lr_decay, which don't use in the parl.algorithms.DDQN . If using the parl.algorithms.DDQN directly, maybe all the experiments should be run again to make sure that parl.algorithms.DDQN performs well.

zenghsh3 · 2021-07-05T08:31:58Z

examples/Application/Traffic-Light-Control/replay_buffer.py

@@ -0,0 +1,100 @@
+#   Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.


Can we use the parl.utils.ReplayMemory directly?

Yes, fixed.

TomorrowIsAnOtherDay · 2021-07-12T11:38:27Z

examples/Application/Traffic-Light-Control/README.md

@@ -0,0 +1,87 @@
+## Reproduce Some Baselines of Traffic Light Control


Baseline Algorithms For Traffic Light Control

TomorrowIsAnOtherDay · 2021-07-12T11:47:46Z

examples/Application/Traffic-Light-Control/README.md

+And we use the cityflow simuator in the experiments, as for how to install the cityflow, please refer [here](https://cityflow.readthedocs.io/en/latest/index.html) for more informations.
+
+### Benchmark Result
+Note that we set the yellow signal time to 5 seconds to clear the intersection, and the action intervals is set to 10 seconds as the papers, you can refer the `config.py` for details, you also can change the time as what you want. The different values of the times above may cause different results of the experiments.


for details -> for more details.
And remove the sentences after that. People may suspect that your implementations are not robust.

TomorrowIsAnOtherDay · 2021-07-12T12:00:10Z

examples/Application/Traffic-Light-Control/README.md

+| FRAP* | 130.53| 159.54| 750.68| 713.48|--| -- |-- | -- |
+
+
+Note that for the method `sotl`, different `t_min`, `min_green_vehicle` and `max_red_vehicle` configs may cause huge different results, which may not fair for sotl to compare its result with others, so we don't list the result of the `sotl` above.


We also provide the implementation for that SOTL algorithm, but its performance heavily relies on the environment variables such as t_min and min_green_vehicle. We do not list its result here.

TomorrowIsAnOtherDay · 2021-07-12T12:06:04Z

examples/Application/Traffic-Light-Control/README.md

+    + Different algorithms have different obs and rewards generators.
+
+
+### Something about the Distributed Training


Please remove the section if we do not provide parallel training algorithms.

TomorrowIsAnOtherDay · 2021-07-12T12:08:13Z

examples/Application/Traffic-Light-Control/README.md

+
+We don't use the distributed traing or the parallel actors for collect the datas from the cityflow env, if you want to use the parallel actors with the cluster, you can refer to [here](https://github.com/PaddlePaddle/PARL/tree/develop/examples/A2C) or our [documentation](https://parl.readthedocs.io/en/latest/parallel_training/setup.html) for details. 
+
+### Some Suggestions and Conclusions


Please remove the section. PARL will not provide suggestions for choosing the algorithm.

TomorrowIsAnOtherDay · 2021-07-12T12:09:26Z

examples/Application/Traffic-Light-Control/agent/agent.py

+
+    def sample(self, obs):
+        # The epsilon-greedy action selector.
+        def sample_random(act_dim):


Please remove the simple function. Just call np.random.randint(0, act_dim).

TomorrowIsAnOtherDay · 2021-07-12T12:12:37Z

examples/Application/Traffic-Light-Control/examples/config_hz_1.json

@@ -0,0 +1,12 @@
+{


can we rename the folder examples as scenarios ?

yes, fixed.

TomorrowIsAnOtherDay · 2021-07-12T12:14:07Z

examples/Application/Traffic-Light-Control/test.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+CUDA_VISIBLE_DEVICES=0 python test.py  --config_path_name './examples/config_hz_1.json' --result_name 'hz_1' --is_test_frap False --save_dir 'save_model/presslight'&
+wait


remove the $ at the last line and wait.

TomorrowIsAnOtherDay · 2021-07-12T12:14:38Z

examples/Application/Traffic-Light-Control/train_presslight.sh

+CUDA_VISIBLE_DEVICES=0 python train_presslight.py  --config_path_name './examples/config_hz_1.json' --save_dir 'save_model/hz_1' --is_share_model False&
+# CUDA_VISIBLE_DEVICES=1 python train_presslight.py  --config_path_name './examples/config_hz_2.json' --save_dir 'save_model/hz_2' --is_share_model False&
+
+wait


CLAassistant · 2024-09-25T03:47:55Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

wangzelong0663 added 4 commits June 23, 2021 11:22

add application with the traffic light control

acaf8a8

rm some useless code and .DS_store file

fc8d6e0

fix readme

f7031de

rm json data and fix readme

b9b7e63

zenghsh3 suggested changes Jul 5, 2021

View reviewed changes

fix readme and use the parl.utils.ReplayMemory directly

33d6903

TomorrowIsAnOtherDay reviewed Jul 12, 2021

View reviewed changes

wangzelong0663 added 3 commits July 15, 2021 10:13

fix readme bug and change the folder name

cd376f1

add new line to the sh file

94664b7

modify test.py and yapf mode

990c39c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Application/traffic_light_control #665

Application/traffic_light_control #665

wangzelong0663 commented Jun 23, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

zenghsh3 Jul 5, 2021

wangzelong0663 Jul 12, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021 •

edited

Loading

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

TomorrowIsAnOtherDay Jul 12, 2021

wangzelong0663 Jul 15, 2021

CLAassistant commented Sep 25, 2024


		Note that for the method `sotl`, different `t_min`, `min_green_vehicle` and `max_red_vehicle` configs may cause huge different results, which may not fair for sotl to compare its result with others, so we don't list the result of the `sotl` above.

		And results of the last two rows of the table ,`presslight` and `FRAP`, they are the results of the code [tlc-baselines](https://github.com/gjzheng93/tlc-baselines) provided from the paper authors' team. We run the [code](https://github.com/gjzheng93/tlc-baselines) just changing the yellow time and the action intervals to keep them same as our config as the papers without changing any other parameters. `--` in the table means the origins code doesn't perform well in the last four `anon_4X4` datas, the average travel time results of it will be more than 1000, maybe it will perform better than the `max_pressure`if you modify the other hyperparameters of the DQN agents, such as the buffer size, update_model_freq, the gamma or others.

		@@ -0,0 +1,95 @@
		# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.

		@@ -0,0 +1,100 @@
		# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.

		@@ -0,0 +1,87 @@
		## Reproduce Some Baselines of Traffic Light Control

		\| FRAP* \| 130.53\| 159.54\| 750.68\| 713.48\|--\| -- \|-- \| -- \|


		Note that for the method `sotl`, different `t_min`, `min_green_vehicle` and `max_red_vehicle` configs may cause huge different results, which may not fair for sotl to compare its result with others, so we don't list the result of the `sotl` above.

		+ Different algorithms have different obs and rewards generators.


		### Something about the Distributed Training


		We don't use the distributed traing or the parallel actors for collect the datas from the cityflow env, if you want to use the parallel actors with the cluster, you can refer to [here](https://github.com/PaddlePaddle/PARL/tree/develop/examples/A2C) or our [documentation](https://parl.readthedocs.io/en/latest/parallel_training/setup.html) for details.

		### Some Suggestions and Conclusions

Application/traffic_light_control #665

Are you sure you want to change the base?

Application/traffic_light_control #665

Conversation

wangzelong0663 commented Jun 23, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomorrowIsAnOtherDay Jul 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CLAassistant commented Sep 25, 2024

TomorrowIsAnOtherDay Jul 12, 2021 •

edited

Loading