[Typing][C-21,C-22,C-23,C-24] Add type annotations for `python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py}` #66864

Lans1ot · 2024-07-31T07:54:20Z

PR Category

User Experience

PR Types

Improvements

Description

#65008
尝试添加函数类型信息

paddle-bot · 2024-07-31T07:54:25Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

megemini · 2024-07-31T10:50:18Z

python/paddle/distributed/communication/reduce_scatter.py

-):
+    tensor: Tensor,
+    tensor_list: list[Tensor],
+    op: ReduceOp = ReduceOp.SUM,


这里类型不是 ReduceOp ，参考 https://github.com/PaddlePaddle/Paddle/pull/66803/files

合一下代码，从那里导入 _ReduceOp 吧～

megemini · 2024-07-31T12:23:16Z

python/paddle/distributed/communication/reduce_scatter.py

-):
+    output: Tensor,
+    input: Tensor,
+    op: ReduceOp = ReduceOp.SUM,


megemini · 2024-07-31T12:24:10Z

python/paddle/distributed/communication/scatter.py

-def scatter(tensor, tensor_list=None, src=0, group=None, sync_op=True):
+def scatter(
+    tensor: Tensor,
+    tensor_list: list[Tensor] | tuple[Tensor] | None = None,


这里 tuple 缺少 ... ～用 Sequence 吧～

python/paddle/distributed/communication/scatter.py

Co-authored-by: megemini <megemini@outlook.com>

… into develop+type_hint_2

Lans1ot · 2024-08-02T06:10:43Z

@megemini 请问一下，static-check里提示error: Argument 2 to "scatter_object_list" has incompatible type "List[Dict[str, List[int]]]"; expected "Optional[List[object]]" [arg-type]，但是我看paddle的api文档是list[Any]，我修改的代码写的是list[object]，请问为什么会出现这个情况呢

python/paddle/distributed/communication/scatter.py

Co-authored-by: megemini <megemini@outlook.com>

megemini · 2024-08-03T03:12:50Z

python/paddle/distributed/communication/scatter.py

+    out_object_list: list[Any],
+    in_object_list: list[Any] | None = None,


没有导入 Any

SigureMo · 2024-08-05T02:11:11Z

python/paddle/distributed/communication/stream/all_gather.py

+    group: Group | None = None,
+    sync_op: bool = True,
+    use_calc_stream: bool = False,
+) -> task:


Suggested change

) -> task:

) -> task | None:

SigureMo · 2024-08-05T02:11:33Z

python/paddle/distributed/communication/stream/all_gather.py

-def _all_gather_in_static_mode(tensor_list, tensor, group, sync_op):
+def _all_gather_in_static_mode(
+    tensor_list: list[Tensor], tensor: Tensor, group: Group, sync_op: bool
+) -> task:


Suggested change

) -> task:

) -> None:

SigureMo · 2024-08-05T02:12:27Z

python/paddle/distributed/communication/send.py

+    dst: int = 0,
+    group: Group | None = None,
+    sync_op: bool = True,
+) -> task:


Suggested change

) -> task:

) -> task | None:

SigureMo · 2024-08-05T02:13:00Z

python/paddle/distributed/communication/send.py

@@ -51,7 +65,7 @@ def send(tensor, dst=0, group=None, sync_op=True):
    )


-def isend(tensor, dst, group=None):
+def isend(tensor: Tensor, dst: int, group: Group | None = None) -> task:


Suggested change

def isend(tensor: Tensor, dst: int, group: Group | None = None) -> task:

def isend(tensor: Tensor, dst: int, group: Group | None = None) -> task | None:

SigureMo · 2024-08-05T02:14:05Z

python/paddle/distributed/communication/scatter.py

+    src: int = 0,
+    group: Group | None = None,
+    sync_op: bool = True,
+) -> None:


Suggested change

) -> None:

) -> task | None:

task 记得 import

SigureMo · 2024-08-05T02:15:22Z

python/paddle/distributed/communication/reduce_scatter.py

+    if op not in [
+        ReduceOp.AVG,
+        ReduceOp.MAX,
+        ReduceOp.MIN,
+        ReduceOp.PROD,
+        ReduceOp.SUM,
+    ]:
+        raise RuntimeError(
+            "Invalid ``op`` function. Expected ``op`` to be of type ``ReduceOp.SUM``, ``ReduceOp.Max``, ``ReduceOp.MIN``, ``ReduceOp.PROD`` or ``ReduceOp.AVG``."
+        )


这里还有下面的检查代码是为了解决什么呢？如果没什么特别需要解决的问题最好还是不要加，因为本任务只涉及类型提示修改，不涉及运行时逻辑修改

这是因为我之前做其他任务的时候，有类似的情况。
就是参数只能在特定值中间进行选择，不然就会抛出一个错误
所以我这里就根据python/paddle/distributed/communication/batch_isend_irecv.py仿写了一段

SigureMo

LGTMeow

Lans1ot added 3 commits July 31, 2024 14:12

补全参数类型信息

13e76d7

修改参数类型信息

feb3ba9

追加遗漏的类型信息

f7dac7a

megemini reviewed Jul 31, 2024

View reviewed changes

megemini mentioned this pull request Jul 31, 2024

[Type Hints] 为公开 API 标注类型提示信息 #65008

Closed

luotao1 added contributor External developers HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务 labels Jul 31, 2024

luotao1 assigned luotao1 and SigureMo Jul 31, 2024

Lans1ot and others added 4 commits August 1, 2024 10:51

Merge branch 'develop' into develop+type_hint_2

b4d048e

Update python/paddle/distributed/communication/scatter.py

0e4d12c

Co-authored-by: megemini <megemini@outlook.com>

按要求进行修改

8c7e446

Merge branch 'develop+type_hint_2' of https://github.com/Lans1ot/Paddle…

bb40649

… into develop+type_hint_2

megemini reviewed Aug 2, 2024

View reviewed changes

python/paddle/distributed/communication/scatter.py Outdated Show resolved Hide resolved

Update python/paddle/distributed/communication/scatter.py

4c0c198

Co-authored-by: megemini <megemini@outlook.com>

megemini reviewed Aug 3, 2024

View reviewed changes

修改导入错误

f488c7d

SigureMo reviewed Aug 5, 2024

View reviewed changes

Lans1ot and others added 3 commits August 5, 2024 14:19

根据建议进行修改

3d4abf3

追加类型信息

9f1f05f

ignore [union-attr]

3832021

SigureMo approved these changes Aug 5, 2024

View reviewed changes

SigureMo merged commit 58f7899 into PaddlePaddle:develop Aug 6, 2024
30 of 31 checks passed

Lans1ot mentioned this pull request Aug 9, 2024

[WeeklyReports] 2024.07.27~2024.08.09 周报收集 PFCCLab/Starter#300

Open

38 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Typing][C-21,C-22,C-23,C-24] Add type annotations for `python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py}` #66864

[Typing][C-21,C-22,C-23,C-24] Add type annotations for `python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py}` #66864

Lans1ot commented Jul 31, 2024 •

edited by SigureMo

Loading

paddle-bot bot commented Jul 31, 2024

megemini Jul 31, 2024

megemini Jul 31, 2024

megemini Jul 31, 2024

Lans1ot commented Aug 2, 2024

megemini Aug 3, 2024

SigureMo Aug 5, 2024

SigureMo Aug 5, 2024

SigureMo Aug 5, 2024

SigureMo Aug 5, 2024

SigureMo Aug 5, 2024

SigureMo Aug 5, 2024

Lans1ot Aug 5, 2024

SigureMo left a comment

		out_object_list: list[Any],
		in_object_list: list[Any] \| None = None,

	def isend(tensor: Tensor, dst: int, group: Group \| None = None) -> task:
	def isend(tensor: Tensor, dst: int, group: Group \| None = None) -> task \| None:

[Typing][C-21,C-22,C-23,C-24] Add type annotations for python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py} #66864

[Typing][C-21,C-22,C-23,C-24] Add type annotations for python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py} #66864

Conversation

Lans1ot commented Jul 31, 2024 • edited by SigureMo Loading

PR Category

PR Types

Description

paddle-bot bot commented Jul 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lans1ot commented Aug 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SigureMo left a comment

Choose a reason for hiding this comment

[Typing][C-21,C-22,C-23,C-24] Add type annotations for `python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py}` #66864

[Typing][C-21,C-22,C-23,C-24] Add type annotations for `python/paddle/distributed/communication/{reduce_scatter.py, scatter.py, send.py, all_gather.py}` #66864

Lans1ot commented Jul 31, 2024 •

edited by SigureMo

Loading