Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error: backward before forward #32

Closed
chsasank opened this issue Apr 9, 2024 · 3 comments
Closed

Error: backward before forward #32

chsasank opened this issue Apr 9, 2024 · 3 comments

Comments

@chsasank
Copy link

chsasank commented Apr 9, 2024

Followed instructions from README and it throws the error:

(base) sasank@johnaic:~/code/sycl/llm.c$ ./train_gpt2
[GPT-2]
max_seq_len: 1024
vocab_size: 50257
num_layers: 12
num_heads: 12
channels: 768
num_parameters: 124439808
train dataset num_batches: 1192
val dataset num_batches: 128
num_activations: 73323776
val loss 5.252006
step 0: train loss 5.356172 (took 2658.361852 ms)
Error: must forward with targets before backward
@chsasank
Copy link
Author

chsasank commented Apr 9, 2024

Duplicate of #19

@chsasank chsasank closed this as completed Apr 9, 2024
@ent0n29
Copy link
Contributor

ent0n29 commented Apr 9, 2024

this is solved with these changes in the Makefile:
delete -Ofast
CFLAGS = -O3 -Wno-unused-result

or add this flag to disable -ffast-math
CFLAGS = -O3 -Ofast -fno-fast-math -Wno-unused-result

@chsasank
Copy link
Author

chsasank commented Apr 9, 2024

Yep, saw #21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chsasank @ent0n29