Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove paddle.where in cross_tropy_loss #38456

Merged
merged 20 commits into from
Jan 10, 2022
Merged

Remove paddle.where in cross_tropy_loss #38456

Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
0a7abb7
Remove the labels range check under the dynamic graph
HydrogenSulfate Dec 26, 2021
c86e268
Delete the labels range check in the dynamic graph
HydrogenSulfate Dec 26, 2021
bb7569b
Remove the labels range check under the dynamic graph
HydrogenSulfate Dec 26, 2021
ba490d6
Remove the labels range check under the dynamic graph
HydrogenSulfate Dec 26, 2021
5e18873
restore test for min,max labels
HydrogenSulfate Dec 26, 2021
389d937
change error to IndexError
HydrogenSulfate Dec 26, 2021
b29af0d
change error to IndexError
HydrogenSulfate Dec 26, 2021
2d2909d
Merge branch 'newest' into remove_check_cross_entropy
HydrogenSulfate Dec 26, 2021
7095918
change to ValueError
HydrogenSulfate Dec 26, 2021
e080992
change to IndexError
HydrogenSulfate Dec 27, 2021
c3db6e3
change to IndexError
HydrogenSulfate Dec 27, 2021
301dae1
remove hard labels check
HydrogenSulfate Dec 27, 2021
83e90cd
Update test_cross_entropy_loss.py
HydrogenSulfate Dec 28, 2021
d816760
replace .where to '=='
HydrogenSulfate Dec 28, 2021
103eb7a
Merge branch 'newest' into remove_check_cross_entropy
HydrogenSulfate Dec 28, 2021
082dc2a
Merge branch 'newestt' into remove_check_cross_entropy
HydrogenSulfate Dec 28, 2021
3b598e2
add static label check
HydrogenSulfate Dec 28, 2021
056394c
update code
HydrogenSulfate Dec 28, 2021
e99f74e
Merge branch 'develop' into remove_check_cross_entropy
HydrogenSulfate Jan 9, 2022
7c3d0fe
replace where with min and max
HydrogenSulfate Jan 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 13 additions & 21 deletions python/paddle/nn/functional/loss.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1666,25 +1666,16 @@ def cross_entropy(input,
label = paddle.unsqueeze(label, axis=axis)
if in_dygraph_mode():
if soft_label == False:
valid_label = paddle.where(label == ignore_index,
paddle.zeros_like(label), label)
# TODO: Temporarily use paddle.nonzero instead of paddle.max
# to detect and find out possible illegal label values
if len(paddle.nonzero(valid_label < 0)) > 0:
invalid_label = paddle.gather_nd(
valid_label, paddle.nonzero(valid_label < 0))
raise ValueError(
"Target({}) is out of class_dimension's lower bound({})".
format(invalid_label[0], 0))
# TODO: Temporarily use paddle.nonzero instead of paddle.max
# to detect and find out possible illegal label values
if len(paddle.nonzero(valid_label >= input.shape[axis])) > 0:
invalid_label = paddle.gather_nd(
valid_label,
paddle.nonzero(valid_label >= input.shape[axis]))
raise ValueError(
"Target({}) is out of class_dimension's upper bound({})".
format(invalid_label[0], input.shape[axis] - 1))
valid_label = paddle.cast(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

之前讨论过在weight is not None的条件下，才需要判断label的合法性。需要把检查移动到weight is not None中吗？

Copy link
Contributor Author

@HydrogenSulfate HydrogenSulfate Jan 10, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

由于softmax_with_cross_entropy中没有对labels进行充分的合法性检查，所以这里在hard label条件下就判断label的合法性，无论是否用了weight

label != ignore_index, dtype=label.dtype) * label
label_min = paddle.min(valid_label)
label_max = paddle.max(valid_label)
if label_min < 0:
raise ValueError("label should not out of bound, but got{}".
format(label_min))
if label_max >= input.shape[axis]:
raise ValueError("label should not out of bound, but got{}".
format(label_max))
if core.is_compiled_with_npu():
_, _, out = _C_ops.softmax_with_cross_entropy(
input, label, 'soft_label', soft_label, 'ignore_index',
Expand Down Expand Up @@ -1838,8 +1829,9 @@ def cross_entropy(input,
"when weight is provided" \
.format(input.shape[axis], weight.shape[-1]))

valid_label = paddle.where(label == ignore_index,
paddle.zeros_like(label), label)
valid_label = paddle.multiply(
paddle.cast(
label != ignore_index, dtype=label.dtype), label)
ignore_weight_mask = paddle.cast((label != ignore_index),
input.dtype)
if ignore_weight_mask.ndim > 1 and ignore_weight_mask.shape[
Expand Down