New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

when using auto_insert_landmarks, it appears to be view problems #12

Open

EzTan1 opened this issue Aug 1, 2023 · 1 comment

EzTan1 commented Aug 1, 2023

as mem_freq = 63, and seq_length 3535 can't divide 63, view[input_ids.shape[0], -1, mem_freq] would raise error?

Collaborator

mkrima commented Aug 1, 2023

yes, when using auto_insert_landmarks and triton kernel during training, sequence length should be divisible by mem_freq.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment