【Inplace api】Add copy for inplace #54683

GGBond8488 · 2023-06-15T07:01:29Z

PR types

Others

PR changes

Others

Description

Pcard-72375
add Input copy for inpalce api backward calculate

Normal api change

Before:

paddle::Tensor pow_ad_func(const paddle::Tensor& x, paddle::experimental::Scalar y) {
...
 // Get Input AutoGradMeta
 egr::AutogradMeta* x_autograd_meta = egr::EagerUtils::nullable_autograd_meta(x);
...
 // Forward API Call
 auto api_result = paddle::experimental::pow(x, y);
...
...
 // Get Outputs
 auto& out = api_result;

 // Get Output AutoGradMeta
 egr::AutogradMeta* out_autograd_meta = egr::EagerUtils::autograd_meta(&out);
 bool trace_backward = egr::Controller::Instance().HasGrad();
 bool require_any_grad = egr::EagerUtils::ComputeRequireGrad(trace_backward,x_autograd_meta);

 // Check Inplace if needed

 // Node Creation
 if(require_any_grad) {
...
   // Node Construction
   auto grad_node = std::shared_ptr<PowGradNode>(new PowGradNode(1, 1));
   // Set for forward trace
 if (FLAGS_check_nan_inf) {
   grad_node->SetForwardTrace(egr::Controller::Instance().GetPythonStack());
 }
   // SetAttributes if needed
   grad_node->SetAttributefactor(factor);
   // Set TensorWrappers for Forward Inputs if needed
   grad_node->SetTensorWrapperx(x);
   // SetGradOutMeta & SetEdges
   grad_node->SetGradOutMeta(x, 0);
   // SetOutRank & SetHistory & SetGradInMeta
   if (out_autograd_meta) {
     egr::EagerUtils::SetOutRankWithSlot(out_autograd_meta, 0);
   }
   if (out_autograd_meta) {
     egr::EagerUtils::SetHistory(out_autograd_meta, grad_node);
   }
   grad_node->SetGradInMeta(out, 0);
   // Set TensorWrappers for Forward Outputs if needed

 }

After this pr merge:

paddle::Tensor pow_ad_func(const paddle::Tensor& x, paddle::experimental::Scalar y) {
...
  // Get Input AutoGradMeta
  egr::AutogradMeta* x_autograd_meta = egr::EagerUtils::nullable_autograd_meta(x);

  bool trace_backward = egr::Controller::Instance().HasGrad();
  bool require_any_grad = egr::EagerUtils::ComputeRequireGrad(trace_backward,x_autograd_meta);

  // Node Declaration
  std::shared_ptr<PowGradNode> grad_node;

  // Set grad_node before API Call
  if(require_any_grad) {
    paddle::platform::RecordEvent node_creation_record_event("pow node_creation", paddle::platform::TracerEventType::OperatorInner, 1);
    // Node Construction
    grad_node = std::shared_ptr<PowGradNode>(new PowGradNode(1, 1));
    // Set for forward trace
  if (FLAGS_check_nan_inf) {
    grad_node->SetForwardTrace(egr::Controller::Instance().GetPythonStack());
  }
    // SetAttributes if needed
    grad_node->SetAttributey(y);
    // Set TensorWrappers for Forward Inputs if needed
    grad_node->SetTensorWrapperx(x);
  }

 // Forward API Call
  auto api_result = paddle::experimental::pow(x, y);
...
  // Get Outputs
  auto& out = api_result;

  // Get Output AutoGradMeta
  egr::AutogradMeta* out_autograd_meta = egr::EagerUtils::autograd_meta(&out);
...
  // Set grad_node after API call
  if(require_any_grad) {
    egr::EagerUtils::PassStopGradient(false,out_autograd_meta);
    // SetGradOutMeta & SetEdges
    grad_node->SetGradOutMeta(x, 0);
    // SetOutRank & SetHistory & SetGradInMeta
    if (out_autograd_meta) {
      egr::EagerUtils::SetOutRankWithSlot(out_autograd_meta, 0);
    }
    if (out_autograd_meta) {
      egr::EagerUtils::SetHistory(out_autograd_meta, grad_node);
    }
    grad_node->SetGradInMeta(out, 0);
    // Set TensorWrappers for Forward Outputs if needed
  }

  VLOG(4) << "Finish AD API: pow";`

Inplace api change

will add Input copy in inplace pow as follow

paddle::Tensor pow__ad_func(const paddle::Tensor& x, paddle::experimental::Scalar y) {
...
  // Set grad_node before API Call
  if(require_any_grad) {
    paddle::platform::RecordEvent node_creation_record_event("pow node_creation", paddle::platform::TracerEventType::OperatorInner, 1);
    // Node Construction
    grad_node = std::shared_ptr<PowGradNode>(new PowGradNode(1, 1));
    // Set for forward trace
  if (FLAGS_check_nan_inf) {
    grad_node->SetForwardTrace(egr::Controller::Instance().GetPythonStack());
  }
    // SetAttributes if needed
    grad_node->SetAttributey(y);
    // Set TensorWrappers for Forward Inputs if needed
    auto x_clone = paddle::experimental::assign(x);
    grad_node->SetTensorWrapperx(x_clone);
  }

 // Forward API Call
  auto api_result = paddle::experimental::pow(x, y);
  // Check NaN and Inf if needed

  // Get Outputs
  auto& out = api_result;
...

paddle-bot · 2023-06-15T07:01:34Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

chenwhql · 2023-06-27T02:49:12Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

+  bool trace_backward = egr::Controller::Instance().HasGrad();
+  bool require_any_grad = egr::EagerUtils::ComputeRequireGrad({});
+
+ // Node Declaration


生成代码的缩进也需要对齐

chenwhql · 2023-06-27T02:49:25Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

+ // Node Declaration
+  std::shared_ptr<{}> grad_node;
+
+  //Set grad_node before API Call


注释后需要一个空格

chenwhql · 2023-06-27T02:49:30Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

  // Check Inplace if needed
 {}{}
-  // Node Creation
+  //Set grad_node after API call


jeff41404 · 2023-06-27T04:32:47Z

test/legacy_test/test_pow.py

+            'y must be scalar or tensor type, but received: %s ' % (type([2])),
+        ):
+            paddle.pow_(var, [2])
+


shall we check gradients of inplace paddle.pow_?

TestInplacePowerScalar(Tensor) inherit from TestDygraphInplace, which contains the backward test

jeff41404 · 2023-06-27T04:35:36Z

in description, before code of pow_ad_func, auto api_result = paddle::experimental::sparse::pow(x, factor); shall be auto api_result = paddle::experimental::pow(x, y); ?

GGBond8488 · 2023-06-27T06:15:14Z

in description, before code of pow_ad_func, auto api_result = paddle::experimental::sparse::pow(x, factor); shall be auto api_result = paddle::experimental::pow(x, y); ?

It has been changed to auto api_result = paddle::experimental::pow(x, y);

jeff41404

LGTM

sunzhongkai588 · 2023-06-28T07:45:24Z

python/paddle/tensor/math.py

+def pow_(x, y, name=None):
+    """
+    Inplace version of ``pow`` API, the output Tensor will be inplaced with input ``x``.
+    Please refer to :ref:`api_tensor_pow`.


好像这个标签不正确，导致没有正确引用。试试改成api_paddle_pow，或者在api_label 中加入该标签

add clone for inpalce

03aeb98

GGBond8488 added 9 commits June 19, 2023 08:23

fix name

83155b5

add inplace pow

4cf782a

fix typro

915f711

add note

63e7c08

fix typro

b3e23c0

fix typro

3510e07

fix bug

46eb460

fix test error

7378b79

add type error test

6ac3f04

chenwhql reviewed Jun 27, 2023

View reviewed changes

jeff41404 reviewed Jun 27, 2023

View reviewed changes

adjust indentation

ed31049

chenwhql approved these changes Jun 28, 2023

View reviewed changes

jeff41404 approved these changes Jun 28, 2023

View reviewed changes

This comment was marked as abuse.

Sign in to view

sunzhongkai588 reviewed Jun 28, 2023

View reviewed changes

jeff41404 merged commit 98debaa into PaddlePaddle:develop Jun 28, 2023

GGBond8488 changed the title ~~【Inplace】Add copy for inplace~~ 【Inplace api】Add copy for inplace Jul 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【Inplace api】Add copy for inplace #54683

【Inplace api】Add copy for inplace #54683

GGBond8488 commented Jun 15, 2023 •

edited

Loading

paddle-bot bot commented Jun 15, 2023

chenwhql Jun 27, 2023

chenwhql Jun 27, 2023

chenwhql Jun 27, 2023

jeff41404 Jun 27, 2023

GGBond8488 Jun 27, 2023

jeff41404 commented Jun 27, 2023

GGBond8488 commented Jun 27, 2023

jeff41404 left a comment

This comment was marked as abuse.

sunzhongkai588 Jun 28, 2023

【Inplace api】Add copy for inplace #54683

【Inplace api】Add copy for inplace #54683

Conversation

GGBond8488 commented Jun 15, 2023 • edited Loading

PR types

PR changes

Description

Normal api change

Inplace api change

paddle-bot bot commented Jun 15, 2023

chenwhql Jun 27, 2023

Choose a reason for hiding this comment

chenwhql Jun 27, 2023

Choose a reason for hiding this comment

chenwhql Jun 27, 2023

Choose a reason for hiding this comment

jeff41404 Jun 27, 2023

Choose a reason for hiding this comment

GGBond8488 Jun 27, 2023

Choose a reason for hiding this comment

jeff41404 commented Jun 27, 2023

GGBond8488 commented Jun 27, 2023

jeff41404 left a comment

Choose a reason for hiding this comment

This comment was marked as abuse.

sunzhongkai588 Jun 28, 2023

Choose a reason for hiding this comment

GGBond8488 commented Jun 15, 2023 •

edited

Loading