Multiobjective Fitness: NSGA-II

README.md

@@ -1,13 +1,18 @@
 [![Build Status](https://travis-ci.org/CodeReclaimers/neat-python.svg)](https://travis-ci.org/CodeReclaimers/neat-python)
 [![Coverage Status](https://coveralls.io/repos/CodeReclaimers/neat-python/badge.svg?branch=master&service=github)](https://coveralls.io/github/CodeReclaimers/neat-python?branch=master)
 
-## STATUS NOTE ##
+## FORK STATUS ##
 
-This project is currently in maintenance-only mode. I will make bug fixes, do cleanup, and possibly improve sample code
-as I have time, but I will not be adding any new features.  The forks by
-[@drallensmith](https://github.com/drallensmith/neat-python) and [@bennr01](https://github.com/bennr01/neat-python) have
-been extended beyond this implementation a great deal, so those might be better starting points if you need more
-features than what you see here.
+This is a fork from [CodeReclaimers](https://github.com/CodeReclaimers/neat-python).
+The main focus is Non-dominated Sorting for Multiobjective Fitness. That means having more than one fitness value that should be optimized.
+This is done throught the implementation of [NSGA-II](https://ieeexplore.ieee.org/document/996017) as a Reproduction method. More details on the `neat/nsga2/` readme.
+
+The current repository also presents a hoverboard game/simulation to be used as a problem for testing the NSGA-II feature, as well as examples for training it with and without NSGA-II.
+Check the readme on `examples/nsga2` for more details.
+
+![hoverboard-reference](https://i.imgur.com/CfrdHmr.gif)
+
+I've tried keeping the minimal amount of change to the core library, so merging to the main fork should be easy. All these changes are backwards-compatible.
 
 ## About ##
 
@@ -31,12 +36,12 @@ The documentation, is available on [Read The Docs](http://neat-python.readthedoc
 ## Citing ##
 
 Here is a Bibtex entry you can use to cite this project in a publication. The listed authors are the maintainers of
-all iterations of the project up to this point. 
+all iterations of the project up to this point.
 
 ```
 @misc{neat-python,
     Title = {neat-python},
     Author = {Alan McIntyre and Matt Kallada and Cesar G. Miguel and Carolina Feher da Silva},
     howpublished = {\url{https://github.com/CodeReclaimers/neat-python}}   
   }
-```
+```

examples/hoverboard/README.md

@@ -0,0 +1,92 @@
+## NSGA-II examples ##
+
+The scripts in this directory show examples of using NEAT to control a hoverboard on a game.
+It uses Recurrent Networks to control the intensity of both left and right thrusters of the hoverboard, based on it's velocity, angular velocity and normal vector. All those informations could be retreived from real world sensors.
+
+There are two examples:
+- __time__: Evolves network with single-fitness _DefaultReproduction_ method, optimizing flight time.
+- __timedist__: Evolves network with _NSGA2Reproduction_ method, optimizing two fitness values: flight time and mean squared distance from center.
+
+![hoverboard-reference](https://i.imgur.com/CfrdHmr.gif)
+
+## hoverboard.py
+
+This file implements the game using [pygame](http://pygame.org/).
+
+You can manually play it! However, it's designed to be near impossible without AI (or some USB flight controllers, I guess).
+
+```python
+pip install pygame
+python hoverboard.py
+```
+
+- Q/A : +/- left thruster
+- P/L : +/- right thruster
+
+## evolve-time.py
+
+A reference example using a Recurrent Network with the _DefaultReproduction_ method, optimizing a single value of fitness: flightime.
+The input values are: velocity (X/Y), angular velocity and normal vector (X/Y).
+
+![hoverboard-reference](https://i.imgur.com/UpJ2HA7.gif)
+
+The evolution converges fast on simple behaviours such as overcoming gravity by boosting both thrusters simultaneously, however a more refined fitness method should include the total variation of velocities and normal vector to help it converge faster to a stable controller.
+
+```
+> python evolve-time.py <START_ANGLE>
+
+> python evolve-time.py --help
+```
+
+## evolve-timedist.py
+
+A working example using a Recurrent Network with _NSGA2Reproduction_ method, optimizing two fitness values: flight time and mean squared distance from center.
+The input values are: velocity (X/Y), angular velocity, normal vector (X/Y) and distance to center (X/Y).
+
+For each genome, instead of a single cycle this method runs 10 game cycles, starting from 5 preset points (including center) with the starting angle A and -A. The fitness results are accumulated and then divided by 10.
+
+![hoverboard-reference](https://i.imgur.com/CfrdHmr.gif)
+
+This method converges a lot faster to results way beyond the convergence point of the default method. More about this at the _Results_ section of this document.
+
+```
+> python evolve-timedist.py <START_ANGLE>
+
+> python evolve-timedist.py --help
+```
+
+## visualize.py
+
+This is a small tool for viewing the generation data stored at checkpoints.
+It allows you to watch the best genome of each generation, as well as plotting fitness and species data over generations.
+The plots on the _Results_ section of this document were made with this tool.
+
+```
+> python visualize.py <START_ANGLE> <EXPERIMENT>
+
+> python visualize.py --help
+```
+
+## gui.py
+
+This is an utilitary lib for drawing neat-python networks using pygame.
+
+# Results
+
+Here's a quick comparison of results found for this particular hoverboard game with and without the use of NSGA-II. These experiments must be improved in order to better outline the benefits and downsides of this approach. Please feel free to develop them further.
+
+The fitness value plotted is Flight Time on both cases. As described above, the NSGA-II example takes the average of 10 runs starting from preset points, to avoid developing behaviours biased on starting at the center.
+The observed increase in mean convergence does not seem to rely on these 10 runs evaluation, it is actually harder to evolve in those conditions.
+
+![results_fitness](https://s1.imghub.io/05eik.png)
+
+The distribution of species over generations is heavily affected by NSGA-II. More research is due to evaluate it's cost and benefits. Overall, the species tend to stabilize, having more time to evolve it's features.
+A plot of species on the solution space is due to evaluate their distribution, that should be grouped and moving towards the pareto-front.
+
+![results_species](https://s1.imghub.io/05H6H.png)
+
+This plot is messy and needs to be improved. It's a scatter plot of every genome on every generation, color coded.
+In order to visualize the overall movement of the population in the solution space, each generation set of points is filled with a Delaunay Triangulation. You can see the generation shapes moving towards the pareto front.
+A black line represents the best solution of each generation, so you can see the optimization path and convergence.
+
+![results_pareto](https://s1.imghub.io/05GrJ.png)

examples/hoverboard/config-default

@@ -0,0 +1,81 @@
+#--- parameters for the XOR-2 experiment ---#
+
+[NEAT]
+fitness_criterion     = max
+fitness_threshold     = 1000
+pop_size              = 50
+reset_on_extinction   = False
+
+[DefaultGenome]
+# node activation options
+activation_default      = random
+activation_mutate_rate  = 0.5
+activation_options      = relu gauss
+
+# node aggregation options
+aggregation_default     = random
+aggregation_mutate_rate = 0.3
+aggregation_options     = max sum
+
+# node bias options
+bias_init_mean          = 0.0
+bias_init_stdev         = 1.0
+bias_max_value          = 2.0
+bias_min_value          = -2.0
+bias_mutate_power       = 0.5
+bias_mutate_rate        = 0.7
+bias_replace_rate       = 0.1
+
+# genome compatibility options
+compatibility_disjoint_coefficient = 1.0
+compatibility_weight_coefficient   = 0.5
+
+# connection add/remove rates
+conn_add_prob           = 0.5
+conn_delete_prob        = 0.5
+
+# connection enable options
+enabled_default         = True
+enabled_mutate_rate     = 0.01
+
+feed_forward            = False
+initial_connection      = partial_nodirect 0.6
+
+# node add/remove rates
+node_add_prob           = 0.4
+node_delete_prob        = 0.4
+
+# network parameters
+num_hidden              = 0
+num_inputs              = 5
+num_outputs             = 2
+
+# node response options
+response_init_mean      = 1.0
+response_init_stdev     = 0.0
+response_max_value      = 30.0
+response_min_value      = -30.0
+response_mutate_power   = 0.0
+response_mutate_rate    = 0.0
+response_replace_rate   = 0.0
+
+# connection weight options
+weight_init_mean        = 0.0
+weight_init_stdev       = 60.0
+weight_max_value        = 100
+weight_min_value        = -100
+weight_mutate_power     = 3
+weight_mutate_rate      = 0.8
+weight_replace_rate     = 0.3
+
+[DefaultSpeciesSet]
+compatibility_threshold = 20.0
+
+[DefaultStagnation]
+species_fitness_func = max
+max_stagnation       = 20
+species_elitism      = 2
+
+[DefaultReproduction]
+elitism            = 1
+survival_threshold = 0.2

examples/hoverboard/config-nsga2

@@ -0,0 +1,80 @@
+#--- parameters for the XOR-2 experiment ---#
+
+[NEAT]
+fitness_criterion     = max
+fitness_threshold     = 1000
+pop_size              = 50
+reset_on_extinction   = False
+
+[DefaultGenome]
+# node activation options
+activation_default      = random
+activation_mutate_rate  = 0.5
+activation_options      = relu gauss
+
+# node aggregation options
+aggregation_default     = random
+aggregation_mutate_rate = 0.3
+aggregation_options     = max sum
+
+# node bias options
+bias_init_mean          = 0.0
+bias_init_stdev         = 1.0
+bias_max_value          = 2.0
+bias_min_value          = -2.0
+bias_mutate_power       = 0.5
+bias_mutate_rate        = 0.7
+bias_replace_rate       = 0.1
+
+# genome compatibility options
+compatibility_disjoint_coefficient = 1.0
+compatibility_weight_coefficient   = 0.5
+
+# connection add/remove rates
+conn_add_prob           = 0.5
+conn_delete_prob        = 0.5
+
+# connection enable options
+enabled_default         = True
+enabled_mutate_rate     = 0.01
+
+feed_forward            = False
+initial_connection      = partial_nodirect 0.6
+
+# node add/remove rates
+node_add_prob           = 0.4
+node_delete_prob        = 0.4
+
+# network parameters
+num_hidden              = 0
+num_inputs              = 7
+num_outputs             = 2
+
+# node response options
+response_init_mean      = 1.0
+response_init_stdev     = 0.0
+response_max_value      = 30.0
+response_min_value      = -30.0
+response_mutate_power   = 0.0
+response_mutate_rate    = 0.0
+response_replace_rate   = 0.0
+
+# connection weight options
+weight_init_mean        = 0.0
+weight_init_stdev       = 60.0
+weight_max_value        = 100
+weight_min_value        = -100
+weight_mutate_power     = 3
+weight_mutate_rate      = 0.8
+weight_replace_rate     = 0.3
+
+[DefaultSpeciesSet]
+compatibility_threshold = 20.0
+
+[DefaultStagnation]
+species_fitness_func = max
+max_stagnation       = 20
+species_elitism      = 2
+
+[NSGA2Reproduction]
+survival_threshold = 0.2