realization of two networks #773
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
qingqing01
reviewed
Apr 16, 2018
There was a problem hiding this comment.
@denglelaibh 非常感谢你的贡献！
我看到这个PR有3个模型：
- drn
- teach_read
- text_cnn_classify
为了review方便，请分别提PR。
另外，drn主要是增加了identity_resnet吧？ 这个加到https://github.com/PaddlePaddle/models/blob/develop/image_classification 是否合适？
|
|
||
| （2）对于任意深的单元 **L**，它的特征 <img src="https://latex.codecogs.com/gif.latex?$x_L&space;=&space;x_0&space;+&space;\sum_{i=0}^{L−1}F(x_i,W_i)$" title="$x_L = x_0 + \sum_{i=0}^{L−1}F(x_i,W_i)$" />，即为之前所有残差函数输出的总和(加上<img src="https://latex.codecogs.com/gif.latex?$x_0$" title="$x_0$" />)。而正好相反的是，“plain network”中的特征<img src="https://latex.codecogs.com/gif.latex?$x_L$" title="$x_L$" />是一系列矩阵向量的乘积，也就是<img src="https://latex.codecogs.com/gif.latex?$\prod_{i=0}^{L−1}W_i&space;x_0$" title="$\prod_{i=0}^{L−1}W_i x_0$" />，而求和的计算量远远小于求积的计算量。 | ||
|
|
||
| 实验发现，<img src="https://latex.codecogs.com/gif.latex?$h(x_l)&space;=&space;x_l$" title="$h(x_l) = x_l$" />的误差衰减最快，误差也最低(下图a子图所示)： |
There was a problem hiding this comment.
公式不能使用图片格式，需要latext格式。 上面所有公式都需要修改。
There was a problem hiding this comment.
@denglelaibh 浏览器添加公式渲染的插件即可，比如采用chrome插件https://chrome.google.com/webstore/detail/github-with-mathjax/ioemnmodlmafdkllaclgeombjnmnbima
| @@ -0,0 +1,79 @@ | |||
| import paddle.v2 as paddle | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
那drn这个：
drn主要是增加了identity_resnet吧？ 这个加到https://github.com/PaddlePaddle/models/blob/develop/image_classification 是否合适？
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here I realized two networks, a lstm based teach_read network, and resNet 2016, would you please check it and judge if it's ok to add in the model repo.