Skip to content
/ scan Public

SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

License

Notifications You must be signed in to change notification settings

miyosuda/scan

Repository files navigation

SCAN

About

Replicating SCAN algorithm described in Google DeepMind's paper "SCAN: Learning Abstract Hierarchical Compositional Visual Concepts"

Image datasets are created with Rodent envrironment.

Requirements

  • Tensorflow 1.2 or later
  • Python2 or 3

How to train

First extract dataset, and then run main.py

$ tar xvf data.tar.gz
$ python main.py

Result

Sym2img

Symbol to image conversion result.

  1. Generated images when wall_color=white is specified.

  1. Generated images when wall_color=white, floor_color=white are specified.

  1. Generated images when wall_color=white, floor_color=white, obj_color=white are specified.

  1. Generated images when wall_color=white, floor_color=white, obj_color=white, obj_id=ice_lolly are specified.

Img2sym

Input Output
obj_color=white, wall_color=white, floor_color=white, obj_id=ice_lolly

(All of the outputs are correct.)

Input Output
obj_color=purple, wall_color=dark_yellow, obj_id=hat

(Correct obj_color was red, but confused as purple. floor_color was not specifiled in the output.)

Concept recombination

Concept reombination result as image outputs.

  1. Recombination result of (obj_color=white) AND (wall_color=white). (Correct result should have white object and white wall.)

  1. Recombination result of (obj_color=white, obj_id=ice_lolly) IN_COMMON (obj_color=white, wall_color=white. (Correct result should have white object.)

  1. Recombination result of (obj_color=white, wall_color=white) IGNORE (obj_color=white). (Correct result should have white wall.)

βVAE disentanglement

Disentanglement result for latent variables for object parameters. (Wall color, Object color, Floor Color, Object Type, Object position).

Wall color

Obj color

Floor color

Obj Type

Obj pos (and Obj Type)

About

SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published