Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PRIM][IR] Migrate vjp rules to new ir in non primitive mode #55647

Merged
merged 45 commits into from
Aug 8, 2023

Conversation

Charles-hit
Copy link
Contributor

@Charles-hit Charles-hit commented Jul 24, 2023

PR types

New features

PR changes

Others

Description

Pcard-66975

VJP规则与新IR打通(非组合模式)

1. 本PR主要完成的工作

  • 组合算子基础数据结构与新IR适配(DescTensor)
  • 设计组合与非组合模式统一的vjp接口,基于Tanh算子和Mean算子打通从python到C++ 反向规则调用流程(非组合模式)

2. 接下来需要完善功能

  • 基于mean前向拆解将反向组合模式vjp与新IR打通
  • 推理单测mac问题修复
  • 控制流适配
  • pd_op_vjp_manual.cc合入pd_op.cc进行代码自动生成, vjp模块中vjp.hvjp.cc自动代码生成,backend模块中动静态图自动代码生成。

3. 整体架构设计
image

4. 使用实例

import unittest

import paddle
from paddle import ir
from paddle.fluid.core import call_vjp, has_vjp

paddle.enable_static()
main_program, start_program = (
      paddle.static.Program(),
      paddle.static.Program(),
  )
  with paddle.static.program_guard(main_program, start_program):
      x = paddle.static.data('x', [4, 4], 'float32')
      x.stop_gradient = False
      paddle.mean(x, axis=[0, 1])
      paddle.tensor.fill_constant(shape=[1], dtype='float32', value=2.0)
  newir_program = ir.translate_to_new_ir(main_program.desc)
  fill_constant_op = newir_program.block().get_ops()[-1]
  mean_op = newir_program.block().get_ops()[-2]
  out_grads = [[fill_constant_op.result(0)]]
  stop_gradients = [[0]]
  with paddle.ir.core.program_guard(newir_program):
      grad_outs = call_vjp(mean_op, out_grads, stop_gradients)

@cyber-pioneer
Copy link
Contributor

Nice work!!

j,
vjp_res.size()));
return res;
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

该逻辑不要放到IR中

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

std::vector<std::vector<paddle::Tensor>> mean_vjp(
const Tensor& x,
const Tensor& out_grad,
std::vector<int64_t> axis,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const IntArray&

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这儿会在下个pr修改成跟phi一致,后续需要做一层转换转到新ir下参数。

YuanRisheng
YuanRisheng previously approved these changes Aug 7, 2023
zhangbo9674
zhangbo9674 previously approved these changes Aug 7, 2023
cxxly
cxxly previously approved these changes Aug 7, 2023
Copy link
Contributor

@cxxly cxxly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with TODO:fix comment problems in next pr.

Copy link
Contributor

@xiaoguoguo626807 xiaoguoguo626807 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

前向API 和 反向 API是不是分开两个文件合理些

Aurelius84
Aurelius84 previously approved these changes Aug 7, 2023
zhangbo9674
zhangbo9674 previously approved these changes Aug 7, 2023
YuanRisheng
YuanRisheng previously approved these changes Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants