-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add TensorArray #4459
add TensorArray #4459
Conversation
paddle/framework/tensor_array.cc
Outdated
|
||
namespace detail { | ||
|
||
void ta_check_index(size_t index, size_t MAX_SIZE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
may be just check_index
? ta
is a litter confusing
paddle/framework/tensor_array.cc
Outdated
|
||
namespace detail { | ||
|
||
void ta_check_index(size_t index, size_t MAX_SIZE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个大写的 MAX_SIZE 和定义的那个const看起来有点容易混淆
paddle/framework/tensor_array.cc
Outdated
return detail::PackDynamicBatch(values_, meta, lod, level); | ||
} | ||
|
||
std::vector<DySeqMeta> TensorArray::Unpack(const LoDTensor& source, int level, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个函数里面,似乎没有用到 length_desend 这个参数?
paddle/framework/tensor_array.h
Outdated
* Split LoDTensor in some `level` and write the generated batches to | ||
* `values`, if set `desend`, will sort by length in descending order. | ||
*/ | ||
std::vector<DySeqMeta> Unpack(const LoDTensor &source, int level, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what if length_desend is set to false
void Unstack(const LoDTensor &source, bool data_shared) const; | ||
|
||
private: | ||
mutable std::vector<LoDTensor> values_; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this mutable necessary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it will be changed in Read(), expand dynamically.
@@ -0,0 +1,114 @@ | |||
#include "paddle/framework/tensor_array.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
copyright info
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
|
||
TensorArray ta; | ||
const int batch_size{16}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here ~
LoDTensor packed = ta.Pack(0, meta, lod); | ||
} | ||
|
||
TEST_F(TensorArrayTester, size) { ASSERT_EQ(ta.size(), (size_t)batch_size); } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use cpp style cast instead of c style cast
https://google.github.io/styleguide/cppguide.html#Casting
using value_type = float; | ||
|
||
// max number of values allowed to store. | ||
const size_t MAX_SIZE{100000}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! except for some const value name style
No description provided.