Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross entropy loss is divided by the number of classes and not the batch size #34

Closed
cDenius opened this issue Sep 1, 2021 · 2 comments · Fixed by #38
Closed

Cross entropy loss is divided by the number of classes and not the batch size #34

cDenius opened this issue Sep 1, 2021 · 2 comments · Fixed by #38
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@cDenius
Copy link
Contributor

cDenius commented Sep 1, 2021

In the current cross entropy implementation, the loss value is divided by N. In principle, this should be the batch size. But since we are working with a batch size of one and thus call the function with a one-dimensional array, we are currently dividing by the number of classes which makes no sense at all.

Deleting the division would solve the problem for batch size one (= dividing by one). But to support different batch sizes in the future, a more sophisticated solution would be nice.

@cDenius cDenius added good first issue Good for newcomers invalid This doesn't seem right labels Sep 1, 2021
@eileen-kuehn
Copy link
Member

@all-contributors please add @cDenius for bug

@allcontributors
Copy link
Contributor

@eileen-kuehn

I've put up a pull request to add @cDenius! 🎉

@eileen-kuehn eileen-kuehn added bug Something isn't working and removed invalid This doesn't seem right labels Sep 1, 2021
@cDenius cDenius linked a pull request Sep 1, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants