add mine_hard_examples operator #7679

wanghaox · 2018-01-19T03:33:56Z

resolve #7639

qingqing01 · 2018-01-19T11:26:30Z

paddle/operators/mine_hard_examples_op.cc

+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput(
+        "ClsLoss",
+        "(Tensor, default Tensor<float>), The classification loss wit shape "


qingqing01 · 2018-01-22T03:51:15Z

paddle/operators/mine_hard_examples_op.cc

+             "wit shape [N, Np], N is the batch size and Np is the number of "
+             "prior box.")
+        .AsDispensable();
+    AddInput("MatchIndics",


Typo error: MatchIndics -> MatchIndices , fix it in all files.

qingqing01 · 2018-01-31T04:43:44Z

paddle/operators/mine_hard_examples_op.cc

+             "(Tensor, Tensor<int>), Matched indices with shape [N, Np], N is "
+             "the batch size and Np is the number of prior box. "
+             "MatchIndics[i][j] equal -1 means box[j] does not match any "
+             "entity, otherwise means Box[j] is matched to row.");


There is no box[j] Box[j] in context,

MatchIndics[i][j] equal -1 means box[j] does not match any

If MatchIndics[i][j] is -1, it means the j-th prior box in i-th instance does not match any ground-truth box.

otherwise means Box[j] is matched to row."

please also modify this sentence.

qingqing01 · 2018-01-31T04:44:06Z

paddle/operators/mine_hard_examples_op.cc

+             "the batch size and Np is the number of prior box. "
+             "MatchIndics[i][j] equal -1 means box[j] does not match any "
+             "entity, otherwise means Box[j] is matched to row.");
+    AddInput("MatchDis",


MatchDis -> MatchDist ?

qingqing01 · 2018-01-31T04:44:54Z

paddle/operators/mine_hard_examples_op.cc

+                   "(float) The ratio of the negative box to the positive "
+                   "box. Use only when mining_type is equal to max_negative.")
+        .SetDefault(1.0);
+    AddAttr<float>("neg_dis_threshold",


dis -> dist?

qingqing01 · 2018-01-31T05:13:55Z

paddle/operators/mine_hard_examples_op.h

+        if (IsEligibleMining(mining_type, match_indices(n, m), match_dis(n, m),
+                             neg_dis_threshold)) {
+          T loss = cls_loss(n, m);
+          if (mining_type == MiningType::kHardExample) {


if (mining_type == MiningType::kHardExample && in_loc_loss) {

qingqing01 · 2018-01-31T05:20:47Z

paddle/operators/mine_hard_examples_op.h

+      std::vector<int> neg_indices;
+      for (int n = 0; n < neg_sel; ++n) {
+        sel_indices.insert(loss_idx[n].second);
+      }


std::transform also can be used for this for loop.

http://zh.cppreference.com/w/cpp/algorithm/transform

qingqing01 · 2018-01-31T05:24:21Z

paddle/operators/mine_hard_examples_op.h

+      }
+      all_neg_indices.push_back(neg_indices);
+      all_neg_num += neg_indices.size();
+    }


The out_neg_indics_lod[0] can be calculated in for (int n = 0; n < batch_size; ++n) {}, then set LoD like： https://github.com/PaddlePaddle/Paddle/pull/7953/files#diff-7c37cd7e1af079fdcb9b9fef46650007R289

qingqing01 · 2018-01-31T05:26:09Z

paddle/operators/mine_hard_examples_op.h

+    for (auto neg_indices : all_neg_indices) {
+      for (auto neg_idx : neg_indices) {
+        neg_data[neg_offset++] = neg_idx;
+      }


use std::copy instead of for loop.

qingqing01 · 2018-01-31T05:27:10Z

paddle/operators/mine_hard_examples_op.cc

+              "[[1], [0]].");
+
+    AddOutput("UpdatedMatchIndics",
+              "(Tensor) The output of updated MatchIndics, a tensor with "


Tensor<int>

… hard_example

qingqing01 · 2018-02-02T03:46:07Z

paddle/operators/mine_hard_examples_op.cc

+  }
+}
+
+MiningType GetMiningType(std::string str) {


inline MiningType GetMiningType(const std::string& str) { }

qingqing01 · 2018-02-02T03:57:39Z

paddle/operators/mine_hard_examples_op.cc

+        neg_sel = std::min(sample_size, neg_sel);
+      }
+
+      std::sort(loss_idx.begin(), loss_idx.end(), SortScoreDescend<int>);


loss_idx is std::vector<std::pair<T, size_t>>, should sorting by SortScoreDescend<T> not SortScoreDescend<int>. But I see the origin code in Caffe is also SortScoreDescend<int>. A little strange.

The T in SortScoreDescend is corresponding the size_t in std::pair<T, size_t>. so SortScoreDescend<size_t> is more reasonable.

Oh, I see. Thanks!

qingqing01 · 2018-02-02T04:04:09Z

paddle/operators/mine_hard_examples_op.cc

+            neg_indices.push_back(m);
+          }
+        }
+      }


从代码效率来讲, line 124 - line 135写成：

if (mining_type == MiningType::kHardExample) { // ... } else { // 直接拷贝 sel_indices中的it.second到neg_indices中即可。这样单测需要注意，neg_indices里的idx顺序有所变化。 }

qingqing01 · 2018-02-02T04:44:14Z

paddle/operators/mine_hard_examples_op.cc

+        "[N, Np], N is the batch size and Np is the number of prior box.");
+    AddInput("LocLoss",
+             "(Tensor, optional, default Tensor<float>), The localization loss "
+             "wit shape [N, Np], N is the batch size and Np is the number of "


wit -> with

qingqing01 · 2018-02-02T05:32:39Z

paddle/operators/mine_hard_examples_op.cc

+                   "box. Use only when mining_type is equal to max_negative.")
+        .SetDefault(1.0);
+    AddAttr<float>("neg_dist_threshold",
+                   "(float) The negative box dis value threshold. "


The negative overlap upper bound for the unmatched predictions.

qingqing01 · 2018-02-02T05:36:31Z

paddle/operators/mine_hard_examples_op.cc

+Mine hard examples Operator.
+This operator implements hard example mining to select a subset of negative box indices.
+For each image, selects the box with highest losses. subject to the condition that the box cannot have
+an Matcht > neg_dist_threshold when mining_type is equals max_negative. The selected number is 


subject to the condition that the box cannot have an Matcht > neg_dist_threshold when mining_type is equals max_negative

这句话不是很懂，需要改进下。

when mining_type is equals max_negative -> when mining_type is max_negative .

qingqing01 · 2018-02-02T05:38:36Z

paddle/operators/mine_hard_examples_op.cc

+an Matcht > neg_dist_threshold when mining_type is equals max_negative. The selected number is 
+min(sample_size, max_negative_box_number) when mining_type is equals hard_example,
+or min(neg_pos_ratio * positive_box_number, max_negative_box_number) when mining_type is 
+equals max_negative, where the max_negative_box_number is the count of MatchIndices elements with value -1.


when mining_type is equals max_negative -> when mining_type is max_negative

… hard_example

qingqing01 · 2018-02-02T08:44:50Z

paddle/operators/mine_hard_examples_op.cc

+            neg_indices.push_back(m);
+          }
+        }
+      }


Line 137 to line 142:

neg_indices.reserve(sel_indices.size()); std::transform( sel_indices.begin(), sel_indices.end(), neg_indices.begin(), [](int d) { return d;});

qingqing01 · 2018-02-02T08:50:36Z

paddle/operators/mine_hard_examples_op.cc

+      std::vector<int> neg_indices;
+      std::transform(loss_idx.begin(), loss_idx.begin() + neg_sel,
+                     std::inserter(sel_indices, sel_indices.begin()),
+                     [](std::pair<T, size_t> l) -> int {


std::pair<T, size_t> l -> std::pair<T, size_t>& l

qingqing01

Approved this PR. But the unit test may need to enhance in the future.

wanghaox requested review from qingqing01 and pkuyym January 19, 2018 03:34

add mine_hard_examples operator

c5a14ed

qingqing01 mentioned this pull request Jan 19, 2018

The TODO lists for MobileNet-SSD model. #7488

Closed

25 tasks

qingqing01 reviewed Jan 31, 2018

View reviewed changes

wanghaox added 3 commits January 31, 2018 19:29

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8190552

… hard_example

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

00280ba

… hard_example

update mine_hard_examples_op

ff5570c

qingqing01 reviewed Feb 2, 2018

View reviewed changes

wanghaox added 2 commits February 2, 2018 15:24

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

62dc593

… hard_example

update mine_hard_examples op

4284b85

qingqing01 reviewed Feb 2, 2018

View reviewed changes

update mine_hard_examples_op

8137dd9

qingqing01 approved these changes Feb 2, 2018

View reviewed changes

wanghaox merged commit a43594f into PaddlePaddle:develop Feb 2, 2018

wanghaox deleted the hard_example branch February 2, 2018 09:38

add mine_hard_examples operator #7679

add mine_hard_examples operator #7679

Conversation

wanghaox commented Jan 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Feb 2, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 left a comment

Choose a reason for hiding this comment

qingqing01 Feb 2, 2018 •

edited

Loading