survey on computation graph #4398
Conversation
|
We have put forward similar solution now in #4052， close this survey. |
doc/graph_survey.md
Outdated
| 用户配置网络的核心概念是`Symbol`，Mxnet在C++端实现了`Symbol`，并通过CAPI暴露到Python端。在这里可以参考Mxnet中对`Symbol`的注释： | ||
|
|
||
| `Symbol` is help class used to represent the operator node in Graph. | ||
| `Symbol` acts as an interface for building graphs from different components like Variable, Functor and Group. `Symbol` is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value. |
There was a problem hiding this comment.
Curious what is Group used for?
There was a problem hiding this comment.
The Group is like a Concat Operator, just put some symbol together, and use a symbol to represent them.
doc/graph_survey.md
Outdated
| ```python | ||
| def get_symbol(num_classes=10, **kwargs): | ||
| data = mx.symbol.Variable('data') | ||
| data = mx.sym.Flatten(data=data) |
There was a problem hiding this comment.
data = mx.sym.Flatten(data=data) ==>
data = mx.symbol.Flatten(data=data)
doc/graph_survey.md
Outdated
|
|
||
| ### TensorFlow | ||
|
|
||
| 用户配置网络的核心概念是`Tensor`，在Python端定义了`Tensor`，在这里可以直接参考TensorFlow对Tensor的注释： |
There was a problem hiding this comment.
Yes, Symbol/Tensor/Expression are user-end, and they contain all information(mainly data dependency) and can be parsed to Graph-like(Node and Edge) structure.
There was a problem hiding this comment.
Symbol/Tensor/Expression ----> Graph(Nodes and Edges)
doc/design/graph_survey.md
Outdated
| @@ -0,0 +1,232 @@ | |||
| ## Survey on Graph | |||
|
|
|||
| Neural network framework often provides Symbolic api for users to write network topology conveniently. This doc manily focus on Symbolic api in most popular neural network frameworks, and try to find out how to parse Symbolic configuration to a portable file, such as protobuf or json. | |||
There was a problem hiding this comment.
Maybe change all "Symbolic" to "symbolic", and all "api" to "API".
Please check all English text with www.grammarly.com
doc/design/graph_survey.md
Outdated
|
|
||
| The core concept of Symbolic api is `Tensor`. Tensorflow defines `Tensor` in Python. Please refer to the comments in TensorFlow: | ||
|
|
||
| A `Tensor` is a symbolic handle to one of the outputs of an `Operation`. It does not hold the values of that operation's output, but instead provides a means of computing those values in a TensorFlow @{tf.Session}. |
There was a problem hiding this comment.
Don't quite understand what the "@{tf.Session}" syntax means, maybe use [session](link-to-session-doc).
|
@QiJune @helinwang @jacquesqiao this PR lacks activity. Either close or merge it. |
Our Block(NetOp) is now derived from OperatorBase, and a NetOp can nest NetOp arbitrarily. However, when I dive into Mxnet/TensorFlow/Dynet, I find that there will be only one Graph in a topology. Users write symbolic code to configure their topology. The symbolic concepts in Mxnet/TensorFlow/Dynet are Symbol/Tensor/Expression.
In if-else operator, we will have nesting Block, but in most cases, we should not allow nesting Block. fc_op should not be an independent block, and both forward and backward part operators should be in one Block, not two Block.