Skip to content

Commit

Permalink
use flat_hash_map and small_vector in kernel factory
Browse files Browse the repository at this point in the history
  • Loading branch information
@chenwhql
chenwhql committed Oct 15, 2021
1 parent 3f5f789 commit 2309149
Showing 1 changed file with 21 additions and 15 deletions.
36 changes: 21 additions & 15 deletions paddle/tcmpt/core/kernel_factory.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@

#include <ostream>
#include <string>
#include <unordered_map>
#include <utility>

#include "paddle/tcmpt/core/backend.h"
#include "paddle/tcmpt/core/dtype.h"
#include "paddle/tcmpt/core/kernel_def.h"
#include "paddle/tcmpt/core/layout.h"
#include "paddle/utils/flat_hash_map.h"
#include "paddle/utils/small_vector.h"

// See Note [ Why still include the fluid headers? ]
#include "paddle/fluid/platform/enforce.h"
Expand Down Expand Up @@ -209,25 +210,30 @@ class KernelArgsDef {
attribute_defs_.emplace_back(AttributeArgDef(type_index));
}

const std::vector<TensorArgDef>& input_defs() const { return input_defs_; }
const paddle::SmallVector<TensorArgDef>& input_defs() const {
return input_defs_;
}

const std::vector<TensorArgDef>& output_defs() const { return output_defs_; }
const paddle::SmallVector<TensorArgDef>& output_defs() const {
return output_defs_;
}

const std::vector<AttributeArgDef>& attribute_defs() const {
const paddle::SmallVector<AttributeArgDef>& attribute_defs() const {
return attribute_defs_;
}

std::vector<TensorArgDef>& input_defs() { return input_defs_; }
paddle::SmallVector<TensorArgDef>& input_defs() { return input_defs_; }

std::vector<TensorArgDef>& output_defs() { return output_defs_; }
paddle::SmallVector<TensorArgDef>& output_defs() { return output_defs_; }

std::vector<AttributeArgDef>& attribute_defs() { return attribute_defs_; }
paddle::SmallVector<AttributeArgDef>& attribute_defs() {
return attribute_defs_;
}

private:
// TODO(chenweihang): replaced by paddle::small_vector
std::vector<TensorArgDef> input_defs_{{}};
std::vector<TensorArgDef> output_defs_{{}};
std::vector<AttributeArgDef> attribute_defs_{{}};
paddle::SmallVector<TensorArgDef> input_defs_{{}};
paddle::SmallVector<TensorArgDef> output_defs_{{}};
paddle::SmallVector<AttributeArgDef> attribute_defs_{{}};
};

class Kernel {
Expand Down Expand Up @@ -263,10 +269,10 @@ class Kernel {
class KernelFactory {
public:
// replaced by paddle::flat_hash_map later
using KernelMap =
std::unordered_map<KernelName,
std::unordered_map<KernelKey, Kernel, KernelKey::Hash>,
KernelName::Hash>;
using KernelMap = paddle::flat_hash_map<
KernelName,
paddle::flat_hash_map<KernelKey, Kernel, KernelKey::Hash>,
KernelName::Hash>;

static KernelFactory& Instance();

Expand Down

1 comment on commit 2309149

@paddle-bot-old
Copy link

@paddle-bot-old paddle-bot-old bot commented on 2309149 Oct 15, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕵️ CI failures summary

🔍 PR: #34425 Commit ID: 2309149 contains failed CI.

🔹 Failed: PR-CI-APPROVAL

approve_failed
2021-10-15 11:52:47 正在保存至: “bk.txt”

2021-10-15 11:52:47      0K                                                       100% 3.21M=0s

2021-10-15 11:52:47 2021-10-15 11:52:47 (3.21 MB/s) - 已保存 “bk.txt” [5/5])

2021-10-15 11:52:55 ****************

2021-10-15 11:52:55 0. You must have one RD (lanxianghit (Recommend), phlrain or luotao1) approval for changing the FLAGS, which manages the environment variables.

2021-10-15 11:52:55 1. You must have Dianhai approval for change 20+ files or add than 1000+ lines of content.

2021-10-15 11:52:55 2. You must have one RD (XiaoguangHu01,chenwhql,zhiqiu,Xreki,luotao1) approval for paddle/fluid/framework/operator.h, which manages the underlying code for fluid.

2021-10-15 11:52:55 3. You must have one RD (zhiqiu (Recommend) , phlrain) approval for the changes of paddle/fluid/pybind/op_function_generator.cc, which manages the logic of automatic generating op functions for dygraph.

2021-10-15 11:52:55 4. You must have one RD (XiaoguangHu01,chenwhql,zhiqiu,Xreki,luotao1) approval for the usage of const_cast.

2021-10-15 11:52:55 5. You must have one RD (Avin0323(Recommend) or zhouwei25 or wanghuancoder or luotao1) approval for modifying unity_build_rule.cmake which the rules of Unity Build.

2021-10-15 11:52:55 There are 6 approved errors.

2021-10-15 11:52:55 ****************

2021-10-15 11:52:55 + EXCODE=6

2021-10-15 11:52:55 + echo 'EXCODE: 6'

2021-10-15 11:52:55 EXCODE: 6

2021-10-15 11:52:55 + echo 'ipipe_log_param_EXCODE: 6'

2021-10-15 11:52:55 ipipe_log_param_EXCODE: 6

2021-10-15 11:52:55 + exit 6

🔹 Failed: PR-CI-Windows

build_failed
2021-10-15 11:58:33 注锟�: 锟�:     ..\paddle/utils/small_vector.h

2021-10-15 11:58:33 注锟�: 锟�:      C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\cassert

2021-10-15 11:58:33 注锟�: 锟�:       C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\assert.h

2021-10-15 11:58:33 注锟�: 锟�:    ..\paddle/tcmpt/core/scalar.h

2021-10-15 11:58:33 注锟�: 锟�:  ..\paddle/fluid/string/pretty_log.h

2021-10-15 11:58:48 [43/1319] Building CUDA object paddle\fluid\framework\CMakeFiles\custom_tensor.dir__\extension\src.ext_tensor.cu.obj

2021-10-15 11:58:48 .ext_tensor.cu

2021-10-15 11:58:48 Unknown compiler version - please run the configure tests and report the results

2021-10-15 11:58:48 ninja: build stopped: subcommand failed.

2021-10-15 11:58:48 7

2021-10-15 11:58:48 Build Paddle failed, will exit

2021-10-15 11:58:49 EXCODE: 7

🔹 Failed: PR-CI-Windows-OPENBLAS

build_failed
2021-10-15 12:03:27        "C:\home\workspace\Paddle\build\paddle\fluid\framework\ir\memory_optimize_pass\buffer_shared_inplace_op_pass.vcxproj" (default target) (93) ->

2021-10-15 12:03:27        "C:\home\workspace\Paddle\build\paddle\fluid\framework\executor_gc_helper.vcxproj" (default target) (124) ->

2021-10-15 12:03:27        "C:\home\workspace\Paddle\build\paddle\fluid\framework\executor.vcxproj" (default target) (157) ->

2021-10-15 12:03:27          C:\home\workspace\Paddle\paddle/utils/small_vector.h(93): error C2589: '(': illegal token on right side of '::' (compiling source file C:\home\workspace\Paddle\paddle\fluid\framework\hogwild_worker.cc) [C:\home\workspace\Paddle\build\paddle\fluid\framework\executor.vcxproj]

2021-10-15 12:03:27          C:\home\workspace\Paddle\paddle/utils/small_vector.h(93): error C2062: type 'unknown-type' unexpected (compiling source file C:\home\workspace\Paddle\paddle\fluid\framework\hogwild_worker.cc) [C:\home\workspace\Paddle\build\paddle\fluid\framework\executor.vcxproj]

2021-10-15 12:03:27          C:\home\workspace\Paddle\paddle/utils/small_vector.h(93): error C2059: syntax error: ')' (compiling source file C:\home\workspace\Paddle\paddle\fluid\framework\hogwild_worker.cc) [C:\home\workspace\Paddle\build\paddle\fluid\framework\executor.vcxproj]

2021-10-15 12:03:27     546 Warning(s)

2021-10-15 12:03:27     9 Error(s)

2021-10-15 12:03:27 Time Elapsed 00:11:40.79

2021-10-15 12:03:27 7

2021-10-15 12:03:27 Build Paddle failed, will exit

2021-10-15 12:03:27 EXCODE: 7

🔹 Failed: PR-CI-OP-benchmark

Unknown Failed
2021-10-15 13:32:06 + echo '[tools/test_ci_op_benchmark.sh:271] [ERROR] Missing test script of "mean"(paddle/fluid/operators/mean_op.cu) in benchmark.'

2021-10-15 13:32:06 [tools/test_ci_op_benchmark.sh:271] [ERROR] Missing test script of "mean"(paddle/fluid/operators/mean_op.cu) in benchmark.

2021-10-15 13:32:06 + for op_name in '${!CHANGE_OP_MAP[@]}'

2021-10-15 13:32:06 + '[' -z '' ']'

2021-10-15 13:32:06 + exit_code=8

2021-10-15 13:32:06 + LOG '[ERROR] Missing test script of "fill_any_like"(paddle/fluid/operators/fill_any_like_op.cu) in benchmark.'

2021-10-15 13:32:06 + echo '[tools/test_ci_op_benchmark.sh:271] [ERROR] Missing test script of "fill_any_like"(paddle/fluid/operators/fill_any_like_op.cu) in benchmark.'

2021-10-15 13:32:06 [tools/test_ci_op_benchmark.sh:271] [ERROR] Missing test script of "fill_any_like"(paddle/fluid/operators/fill_any_like_op.cu) in benchmark.

2021-10-15 13:32:06 + for op_name in '${!CHANGE_OP_MAP[@]}'

2021-10-15 13:32:06 + '[' -z matmul,matmul,matmul.json,True ']'

2021-10-15 13:32:06 + '[' 8 -ne 0 ']'

2021-10-15 13:32:06 + LOG '[INFO] See https://github.com/PaddlePaddle/Paddle/wiki/PR-CI-OP-benchmark-Manual for details.'

2021-10-15 13:32:06 + echo '[tools/test_ci_op_benchmark.sh:275] [INFO] See https://github.com/PaddlePaddle/Paddle/wiki/PR-CI-OP-benchmark-Manual for details.'

2021-10-15 13:32:06 [tools/test_ci_op_benchmark.sh:275] [INFO] See https://github.com/PaddlePaddle/Paddle/wiki/PR-CI-OP-benchmark-Manual for details.

2021-10-15 13:32:06 + LOG '[INFO] Or you can apply for one RD (Avin0323(Recommend), Xreki, luotao1) approval to pass this PR.'

2021-10-15 13:32:06 + echo '[tools/test_ci_op_benchmark.sh:276] [INFO] Or you can apply for one RD (Avin0323(Recommend), Xreki, luotao1) approval to pass this PR.'

2021-10-15 13:32:06 [tools/test_ci_op_benchmark.sh:276] [INFO] Or you can apply for one RD (Avin0323(Recommend), Xreki, luotao1) approval to pass this PR.

2021-10-15 13:32:06 + exit 8

2021-10-15 13:32:06 {build code state=8}

🔹 Failed: PR-CE-Framework

Unknown Failed
Unknown Failed

Please sign in to comment.