Our previous algorithm was based on simulated annealing and random search. Because of the enormous randomness in the process, the generator could not backtrack. With this milestone, we aim to implement a new algorithm based on policy search that can generate higher quality and more consistent crossword puzzles.