Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification: new mean option is synonym with sum_over_batch_size in loss function base class #20352

Merged
merged 5 commits into from
Oct 15, 2024

Conversation

jm-willy
Copy link
Contributor

@jm-willy jm-willy commented Oct 14, 2024

Since sum_over_batch_size actually is mean, mean option was added as equivalent of sum_over_batch_size to explicitly and clearly state what sum_over_batch_size does, under the expected name mean, while keeping sum_over_batch_size for backwards compatibility. See also #18818, where the problem with the current name is noted.

Also, why is done like this:

loss = ops.sum(values)
loss /= ops.cast(
    ops.prod(ops.convert_to_tensor(ops.shape(values), dtype="int32")),
    loss.dtype,
)

if there's the clearer and shorter keras mean?
loss = ops.mean(values, axis=None, keepdims=False)

Please take it, sum_over_batch_size is confusing, I already implemented mean when I realized it was already done under a different name.

Edit: already undid formating while keeping the typo fixes see last commit: 19933d9

Copy link

google-cla bot commented Oct 14, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Undid ruff format while keeping typo fixes
@codecov-commenter
Copy link

codecov-commenter commented Oct 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.87%. Comparing base (713382b) to head (f81287f).
Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #20352   +/-   ##
=======================================
  Coverage   78.87%   78.87%           
=======================================
  Files         512      512           
  Lines       49264    49264           
  Branches     7953     7953           
=======================================
  Hits        38859    38859           
  Misses       8546     8546           
  Partials     1859     1859           
Flag Coverage Δ
keras 78.74% <ø> (ø)
keras-jax 62.40% <ø> (ø)
keras-numpy 57.42% <ø> (ø)
keras-tensorflow 63.64% <ø> (ø)
keras-torch 62.37% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -132,7 +133,7 @@ def reduce_values(values, reduction="sum_over_batch_size"):
):
return values
loss = ops.sum(values)
if reduction == "sum_over_batch_size":
if (reduction == "sum_over_batch_size") or (reduction == "mean"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if reduction in ("mean", "sum_over_batch_size"):

@@ -177,7 +178,7 @@ def apply_mask(sample_weight, mask, dtype, reduction):
"""Applies any mask on predictions to sample weights."""
if mask is not None:
mask = ops.cast(mask, dtype=dtype)
if reduction == "sum_over_batch_size":
if (reduction == "sum_over_batch_size") or (reduction == "mean"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

@google-ml-butler google-ml-butler bot added kokoro:force-run ready to pull Ready to be merged into the codebase labels Oct 15, 2024
@fchollet fchollet merged commit 813fbc5 into keras-team:master Oct 15, 2024
6 checks passed
@google-ml-butler google-ml-butler bot removed ready to pull Ready to be merged into the codebase kokoro:force-run labels Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants