-
Notifications
You must be signed in to change notification settings - Fork 270
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs/book: add building networks chapter
- Loading branch information
MichaelHirn
committed
Mar 31, 2016
1 parent
4e5c5f6
commit f648454
Showing
6 changed files
with
240 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,77 @@ | ||
| # Network from Layers | ||
| # Create a Network | ||
|
|
||
| In the previous chapters, we learned that everything is a layer. Even the network | ||
| itself is a layer and therefore behaves like any other layer which means, | ||
| that it could be used to create even bigger networks. This is possible, because | ||
| a `Layer` can implement any behavior as long as it takes an input and produces | ||
| an output. In [2.1 Layer Lifecycle](./layer-lifecycle.html) | ||
| we have seen, that only one `LayerConfig` can be used to turn it via | ||
| `Layer::from_config` into an actual `Layer`. But as Deep Learning relies on | ||
| chaining multiple layers together, we need a `Layer`, who implements this | ||
| behavior for us. Enter the container layers. | ||
|
|
||
| ### Networks via the `Sequential` layer | ||
|
|
||
| A `Sequential` is a layer of the container layer category. The config of a | ||
| container layer, e.g. `SequentialConfig` has a special method called, | ||
| `.add_layer` which takes one `LayerConfig` and adds it to an ordered list in the | ||
| `SequentialConfig`. | ||
|
|
||
| When turning a `SequentialConfig` into a `Layer` by passing the config to | ||
| `Layer::from_config`, the behavior of the Sequential is to initialize all the | ||
| layers which were added via `.add_layer` and connect the layers with each other. | ||
| This means, the output of one layer becomes the input of the next layer in the | ||
| list. The input of a `Layer` with a sequential worker, becomes the input of the | ||
| first layer in the sequential worker, the sequential worker then takes care | ||
| of passing the input through all the layers and the output of the last layer | ||
| then becomes the output of the `Layer` with the sequential worker. Therefore | ||
| a sequential `Layer` fulfills the requirements of a `Layer`. | ||
|
|
||
| ```rust | ||
| // short form for: &LayerConfig::new("net", LayerType::Sequential(cfg)) | ||
| let mut net_cfg = SequentialConfig::default(); | ||
|
|
||
| net_cfg.add_input("data", &vec![batch_size, 28, 28]); | ||
| net_cfg.add_layer(LayerConfig::new("reshape", ReshapeConfig::of_shape(&vec![batch_size, 1, 28, 28]))); | ||
| net_cfg.add_layer(LayerConfig::new("conv", ConvolutionConfig { num_output: 20, filter_shape: vec![5], stride: vec![1], padding: vec![0] })); | ||
| net_cfg.add_layer(LayerConfig::new("pooling", PoolingConfig { mode: PoolingMode::Max, filter_shape: vec![2], stride: vec![2], padding: vec![0] })); | ||
| net_cfg.add_layer(LayerConfig::new("linear1", LinearConfig { output_size: 500 })); | ||
| net_cfg.add_layer(LayerConfig::new("sigmoid", LayerType::Sigmoid)); | ||
| net_cfg.add_layer(LayerConfig::new("linear2", LinearConfig { output_size: 10 })); | ||
| net_cfg.add_layer(LayerConfig::new("log_softmax", LayerType::LogSoftmax)); | ||
|
|
||
| // set up the sequential layer aka. a deep, convolutional network | ||
| let mut net = Layer::from_config(backend.clone(), &net_cfg); | ||
| ``` | ||
|
|
||
| As a sequential layer is like any other layer, we can use sequential layers as | ||
| building blocks for larger networks. Important building blocks of a network can | ||
| be grouped into a sequential layer and published as a crate for others to use. | ||
|
|
||
| ```rust | ||
| // short form for: &LayerConfig::new("net", LayerType::Sequential(cfg)) | ||
| let mut conv_net = SequentialConfig::default(); | ||
|
|
||
| conv_net.add_input("data", &vec![batch_size, 28, 28]); | ||
| conv_net.add_layer(LayerConfig::new("reshape", ReshapeConfig::of_shape(&vec![batch_size, 1, 28, 28]))); | ||
| conv_net.add_layer(LayerConfig::new("conv", ConvolutionConfig { num_output: 20, filter_shape: vec![5], stride: vec![1], padding: vec![0] })); | ||
| conv_net.add_layer(LayerConfig::new("pooling", PoolingConfig { mode: PoolingMode::Max, filter_shape: vec![2], stride: vec![2], padding: vec![0] })); | ||
| conv_net.add_layer(LayerConfig::new("linear1", LinearConfig { output_size: 500 })); | ||
| conv_net.add_layer(LayerConfig::new("sigmoid", LayerType::Sigmoid)); | ||
| conv_net.add_layer(LayerConfig::new("linear2", LinearConfig { output_size: 10 })); | ||
|
|
||
| let mut net_cfg = SequentialConfig::default(); | ||
|
|
||
| net_cfg.add_layer(conv_net); | ||
| net_cfg.add_layer(LayerConfig::new("linear", LinearConfig { output_size: 500 })); | ||
| net_cfg.add_layer(LayerConfig::new("log_softmax", LayerType::LogSoftmax)); | ||
|
|
||
| // set up the 'big' network | ||
| let mut net = Layer::from_config(backend.clone(), &net_cfg); | ||
| ``` | ||
|
|
||
| ### Networks via other container layers | ||
|
|
||
| So far, there is only the sequential layer, but other container layers, with | ||
| slightly different behaviors are conceivable. For example a parallel or | ||
| concat layer in addition to the sequential layer. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters