* get_api_md5 should prefer use the real name rather than the alias names

* case for ArgSpec style. update the unittests

test=document_fix

* fix sublayer error with include_sublayers=False

* add ut

* refactor include_sublayers related api

* fix ut

* fix ut of transformer

* fix ut of transformer

* remove useless code

* change sublayer api

* polish code

* add test for include_self=True

* support dp & mp

…to develop (#32294)

* [NPU] support GarbageCollector for npu (#31874)

* support GarbageCollector for npu

* fix typo

* fix gather_grad

* disable NPUDefaultStreamGarbageCollector on NPU

* [NPU] support npu for memcpy op (#31808)

* support npu for memcpy op

* add ut

* fix ut

* fix typo

* 【NPU】fix bug of using temp vector (#31963)

* fix bug when beta1_pow on cpu (#31995)

* [NPU] support npu profiler (#31684)

* support npu profiler

* add python api

* fix bugs

* add wrapper for incomplete type

* update profile proto

* record npu wait

* add xpu placeholder

* fix adam (#32016)

* [NPU] enable async copy and  add wait before sync operation (#31956)

* enable async copy and  add wait before sync operation

* remove unneccessary wait

* add FillNpuTensorWithConstant

* refine

* fix fill_constant

* make TensorFromVector/TensorToVector sync

* [NPU] Support dataloader on npu place. (#31867)

* [NPU] Wait on NPUPlace (#32086)

* [NPU] fix cast op (#32121)

* fix npu kernel of cast op to handle casting to same dtype

* add comments

* [NPU] support cann 20.3 (#32044)

* fix compile problem on cann 20.3

* fix ut

* fix test_mul

* fix check_finite_and_scale

* fix lookup_table_v2_grad

* fix cmake

* support print op

* [NPU] Support npu save load (#31893)

* support save load for NPU

* add save load npu unittest

* support np.array transform in NPU

* fix errors

* delete dygraph in unittest

* add Wait

* fix unittest

* fix review comment

* fix unittest problem

* fix little problem

* change aclrtSynchronizeDevice to aclrtSynchronizeStream for better performance (#32196)

* change aclrtSynchronizeDevice to aclrtSynchronizeStream for better performace

* refine code

* fix NPUDeviceContext in all c++ unittest (#32198)

* fix NPUDeviceContext in all c++ unittest

* refine log

Co-authored-by: pangyoki <pangyoki@126.com>

* [NPU] Remove TensorFromVector and avoid sync copy in npu op kernel for better performance (#31994)

* enable async copy and  add wait before sync operation

* remove unneccessary wait

* add FillNpuTensorWithConstant

* refine

* fix fill_constant

* change TensorFromVector to FillNpuTensorWithConstant

* fix ignored api

* delete extra unittest

* fix little error

* fix update_loss_scaling_op_npu and check_finite_and_unscale_op_npu

* change TensorCopySync to TensorCopy

* delete useless Wait and add StreamWait

* fix npu_stream error

* fix check_finite_and_unscale_op_npu TensorCopy

* only save stream wait

* fix NPUDeviceContext in all c++ unittest

* delete wait

Co-authored-by: zhiqiu <chenqiuliang@baidu.com>

* delete useless unittest file (#32206)

* Fix op test (#32231)

* fix conditional block (#32243)

* fix adam bug again (#32246)

* fix compile

* fix ut

* fix ut

Co-authored-by: liym27 <33742067+liym27@users.noreply.github.com>
Co-authored-by: pangyoki <pangyoki@126.com>

add npu check nan and inf (#32340)

* test for mac task,notest,test=mac_py3

* fix the bug that the error message is not displayed

* build task cost

* return pool

* sharding: update config DOC

* update pipeline config

* sharding update doc

* add paddle.nn.unfold
* update Parameters of Unfold

Change-Id: Ie35a09772e46f7d90cb68ca82c1d18b9201d1abe

* large scale kv store optimize

Change-Id: I582cc661afdaa20749ec7493eae1b88c32b967f7

* replace std::unorded_map with roundrobin map

Change-Id: I48ee0efef38853876c92d982cdfcac6603c52c88

* remove license

* fix cpp lint

Change-Id: Ia21fafa65adc09bb9094f7dbc987e31d5af2686e

* remove fluid for auto_checkpoint.

* fix bug.

* add retry on gcda_clean.py

* add exit code for paddle_coverage.sh

* fix format error

* fix format error

* flush denormal in the tracer op, test=develop

* add cmake dependencies, test=develop

* add a macro, test=develop

* fix the windows case, test=develop

…n multi XPU cards CI (#32302)

* remove thrust includes, test=develop

* fix compilation error, test=develop

* fix compilation of truncated_gaussian_random_op, test=develop

* [NPU] register finalize on exit

* fix

* graph engine demo

* upload unsaved changes

* fix dependency error

* fix shard_num problem

* py client

* remove lock and graph-type

* add load direct graph

* add load direct graph

* add load direct graph

* batch random_sample

* batch_sample_k

* fix num_nodes size

* batch brpc

* batch brpc

* add test

* add test

* add load_nodes; change add_node function

* change sample return type to pair

* resolve conflict

* resolved conflict

* resolved conflict

* separate server and client

* merge pair type

* fix

* resolved conflict

* fixed segment fault; high-level VLOG for load edges and load nodes

* random_sample return 0

* rm useless loop

* test:load edge

* fix ret -1

* test: rm sample

* rm sample

* random_sample return future

* random_sample return int

* test fake node

* fixed here

* memory leak

* remove test code

* fix return problem

* add common_graph_table

* random sample node &test & change data-structure from linkedList to vector

* add common_graph_table

* sample with srand

* add node_types

* optimize nodes sample

* recover test

* random sample

* destruct weighted sampler

* GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* pybind sample nodes api

* pull nodes with step

* fixed pull_graph_list bug; add test for pull_graph_list by step

* add graph table;name

* add graph table;name

* add pybind

* add pybind

* add FeatureNode

* add FeatureNode

* add FeatureNode Serialize

* add FeatureNode Serialize

* get_feat_node

* avoid local rpc

* fix get_node_feat

* fix get_node_feat

* remove log

* get_node_feat return  py:bytes

* merge develop with graph_engine

* fix threadpool.h head

* fix

* fix typo

* resolve conflict

* fix conflict

* recover lost content

* fix pybind of FeatureNode

* recover cmake

* recover tools

* resolve conflict

* resolve linking problem

* code style

* change test_server port

* fix code problems

* remove shard_num config

* remove redundent threads

* optimize start server

* remove logs

* fix code problems by reviewers' suggestions

* move graph files into a folder

* code style change

* remove graph operations from base table

* optimize get_feat function of graph engine

Co-authored-by: Huang Zhengjie <270018958@qq.com>
Co-authored-by: Weiyue Su <weiyue.su@gmail.com>
Co-authored-by: suweiyue <suweiyue@baidu.com>
Co-authored-by: luobin06 <luobin06@baidu.com>
Co-authored-by: liweibin02 <liweibin02@baidu.com>
Co-authored-by: tangwei12 <tangwei12@baidu.com>

)

* add allreduce and broadcast without test (#31024)

add allreduce and broadcast without test

* Refactor HCCLCommContext to be compatible with Paddle (#31359)

Refactor HCCLCommContext to be compatible with Paddle (#31359)

* [NPU] add npu kernel for communication op (#31437)

* add allreduce and broadcast without test

* add c_broadcast_test case

* build c_comm_init and c_create_group operators

* make the whole thing compile

* add broadcast and init op test case but run failed

* make unit test compile

* fix broadcast test bug and change into hcom for ccl

* change c_comm_init and c_create_group ops accordingly

* make tests compile

* transfer code to 27

* compiled successfully in 28, but run failed

* test broadcast in 28, but failed

* make hcom primitives work

* change hccl data type for base.h

* fix broadcast bug

* make attributes work

* fix group name bug

* add allreduce but test failed

* allreduce bug for qiuliang

* allreduce finished

* add allgather and reducescatter

* merge all op code

* add allgather test

* finish run all ccl op test exclude send/recv

* all all op and test exclude send/recv

* send_v2_npu.cc recv_v2_npiu.cc compiled

* fix ccl core dump bug and test allgather, reducescatter, broadcast op

* fix allreduce bug just for test

* hcom send&recv test pass, without hcom_destroy

* for qiuliang test

* Ascend Send&Recv Test Pass

* all op (ex send/recv) ok

* fix bug

* merge all ccl op

* style merge to PaddlePaddle

* merge style

* new merge style

* merge style 2

* insert an empty at the end

* disable ctest for hcom to pass ci

Co-authored-by: void-main <voidmain1313113@gmail.com>
Co-authored-by: f2hkop <f2huestc@outlook.com>

* Add auto-increasing tag id for Hcom OPs (#31702)

* add c_reduce_sum op (#31793)

add c_reduce_sum op

* update Ascendrc hccl to 20.3 (#32126)

update Ascendrc hccl to 20.3 (#32126)

* fix merge code

* change cmake.txt1

* [NPU] Support npu kernel for c sync stream op (#31386)

* sync stream npu op

* add with_ascend_acl

* update c++ unittest

* compile all failed

* try to pre commit

* after pre commit

* merge&compile&test hccl successfully!

* fix code style

* fix code style

* fix bugs about hccl

* fix some bugs

* fix code style

* fix style

* fix style

* fix

* fixed

* merge develop

Co-authored-by: lw921014 <liuwei921014@yeah.net>
Co-authored-by: Void Main <voidmain1313113@gmail.com>
Co-authored-by: f2hkop <f2huestc@outlook.com>
Co-authored-by: xiayanming <41795079@qq.com>

* add support for optimizer with varbase input

* refine cond

* fix failed unittest

* add test for coverage

* support save/load binary format tensor

* Fix error when create cudaplace

* Fix error when create cudaplace

* Fix error when create cudaplace

* get devive context from pool.

* move define of 'SerializeToStream' and 'DeserializeFromStream' to 'lod_tensor.cc' and 'selected_rows.cc'.

* improve coverage.

* improve coverage.

* polish API

* deal with conflict

* disable save/load large file in unnittest

* split unnittest.

* graph engine demo

* upload unsaved changes

* fix dependency error

* fix shard_num problem

* py client

* remove lock and graph-type

* add load direct graph

* add load direct graph

* add load direct graph

* batch random_sample

* batch_sample_k

* fix num_nodes size

* batch brpc

* batch brpc

* add test

* add test

* add load_nodes; change add_node function

* change sample return type to pair

* resolve conflict

* resolved conflict

* resolved conflict

* separate server and client

* merge pair type

* fix

* resolved conflict

* fixed segment fault; high-level VLOG for load edges and load nodes

* random_sample return 0

* rm useless loop

* test:load edge

* fix ret -1

* test: rm sample

* rm sample

* random_sample return future

* random_sample return int

* test fake node

* fixed here

* memory leak

* remove test code

* fix return problem

* add common_graph_table

* random sample node &test & change data-structure from linkedList to vector

* add common_graph_table

* sample with srand

* add node_types

* optimize nodes sample

* recover test

* random sample

* destruct weighted sampler

* GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* pybind sample nodes api

* pull nodes with step

* fixed pull_graph_list bug; add test for pull_graph_list by step

* add graph table;name

* add graph table;name

* add pybind

* add pybind

* add FeatureNode

* add FeatureNode

* add FeatureNode Serialize

* add FeatureNode Serialize

* get_feat_node

* avoid local rpc

* fix get_node_feat

* fix get_node_feat

* remove log

* get_node_feat return  py:bytes

* merge develop with graph_engine

* fix threadpool.h head

* fix

* fix typo

* resolve conflict

* fix conflict

* recover lost content

* fix pybind of FeatureNode

* recover cmake

* recover tools

* resolve conflict

* resolve linking problem

* code style

* change test_server port

* fix code problems

* remove shard_num config

* remove redundent threads

* optimize start server

* remove logs

* fix code problems by reviewers' suggestions

* move graph files into a folder

* code style change

* remove graph operations from base table

* optimize get_feat function of graph engine

* fix long long count problem

Co-authored-by: Huang Zhengjie <270018958@qq.com>
Co-authored-by: Weiyue Su <weiyue.su@gmail.com>
Co-authored-by: suweiyue <suweiyue@baidu.com>
Co-authored-by: luobin06 <luobin06@baidu.com>
Co-authored-by: liweibin02 <liweibin02@baidu.com>
Co-authored-by: tangwei12 <tangwei12@baidu.com>

add glu in nn.functional

…in Dygraph (#32354)

* add clip/check

* add amp & clip grad in dygraph

* add logging

* modify conv2d_transpose docs

* import sequence_* API to new namespace

* fix typos, remove alias marking

* update sample code

* fix sample code

* fix docstring for sequence_mask

support int32 and int64 kernel for clip operator

* fix two error message

* fix two error message

* fix error

* fix error

* fix error

* fix error

* fix some error message

* fix some error

* fix error

* fix some error

* fix some error

* fix some error

* fix one error

* fix some error

* fix seven error message

* fix error

* fix error

* fix error

* fix error

* add c_concat op

solve hccl communicate conflict (#32447)

… CI (#32378)

* fix Windows CI MP compile and environment install script

* clear Windows CI environment

* clear Windows CI environment

* clear Windows CI environment

* move semantic checks to op_teller

* more ops

* more ops

* revert block related change

* part1

* revert activation

* remove if

* remove const_cast

* reslove conflict

* remove const_cast

* delete useless var

* replace vlog(1) with vlog(3), replace assert with PADDLE_ENFORCE

* down to 19 files

* refactor_check_finite_and_scale_npu_kernel

* fix compile

* add alloc_float_status op

* add alloc_float_status op

* add FloatStatus for check_finite_and_unscale

* refine code

* remove unneccessary logic

* refine for fleet

* Refine Constructor logic of ParallelExecutor

* refine function name

* refine code comment

* remove ut from parallel_ut_rule caused by timeout

* remove timeout ut from parallel_ut_rule file

* move convert_model2dot_ernie to TWO_PARALLEL_JOB list

* add c_identity op, test=develop

* remove useless ext headers

* fix boost header compile failed

Reduce max iter size to fix windows openblas test_yolov3 random failure.
Decrease batch size to fix pe related unittest random failure.

* print the real name for Functions instead of the ArgSpec

class function method

* some API's name is not __module__ + __name__

so, we disard them temporarily.

* update the logging format for console

* ommit the top level of paddle package.

* these APIs have been removed.

test=document_fix

* Another Error occerd

* print_signatures.py 's stdout is redirect to spec file, so should not print any other info.

so sad.

* print the error msg to stderr

* disable the __init__ magic method

* update unittest for sampcd_processor.py

update unittest for sampcd_processor.py

* PR-CI-APPROVAL 's python interpreter name is not 'python3'.

it's a python3.9;
it does not have paddle installed yet.

此句在CI流水线竟然不可以用。报python3找不到
此句在CI流水线竟然不可以用。因为环境没有安装paddle

* testing only extract api from __all__

paddle module(the top module) does not have __add__
test=document_fix

* should import paddle here

* update the mechanism of extractiong and executing for the sample-codes test.

更新抽取代码和执行代码的逻辑
优化输出打印

* good code style

* Add dlnne engine runtime

* Fix log

* Remove <const_cast> and remove unrelated modify with dlnne, +clang-format

* Fix CMakeList format error

* Add copyright message

* Fix dlnne CMakeList.txt

* Add some paddlepaddle_pass to support more networks

* Fix some format bug

* use ZerosLike instead of NPUMemsetAsync

* fix compile

…32428)

* let paddle.utils.install_check support CPU package with GPU device

* use use_cuda in dygraph checking

* add unittest for install_check

* fix tc trt shape

* fix fc dynamic shape

* add fc shape assert

* update

…2470)

* fix bug: when x.dim < y.dim, the result of compare_op is inverse to expected result

* support the cuda for fix the compare broadcast bug

* update

* add pipeline layer

* add Hub Module for easy to use pre-trained models.
*   support list, load, help fucntions.
*   support load models by github, gitee, local 

Co-authored-by: LielinJiang <jianglielin@baidu.com>

* support save/load binary format tensor

* Fix error when create cudaplace

* Fix error when create cudaplace

* Fix error when create cudaplace

* get devive context from pool.

* move define of 'SerializeToStream' and 'DeserializeFromStream' to 'lod_tensor.cc' and 'selected_rows.cc'.

* support complex object

* improve coverage.

* improve coverage

* improve coverage.

* fix a bug.

* polish API

* save/load program

* paddle.save/load: layer

* deal with conflict

* if PY2, block test_paddle_save_load.TestSaveLoadLayer

* polish code.

* polish code

* edit unnittest

* The condition for object to be identified as state_dict becomes strict

* use 'core._cuda_synchronize'

* add trt runtime version check

* use different wrap, and change to major version check

* support python39 and delete python35

* support python39 in Dockerfile.centos

* fix ubuntu18 bug

* update Dockerfile.ubuntu setuptools

* fix centos py39 errors

* fix centos py39 error2