-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FD-based gradient calculation seems incorrect for Burgers (with code to verify) #20
Comments
Please use these three source files to reproduce the benchmark results. |
source (1).zip My modified code is attached. Here are the output from my running:
As you can see, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The issue
I am trying to understand how gradients are computed for Burgers, implemented by
FDM_Burgers()
intrain_utils/losses.py
, as pasted below:It is clear that you are using finite difference (FD) to compute$u_t$ , $u_x$ , you are computing it in the Fourier space as described in the paper. However, it seems to me that what you are doing here (i.e., only one round of FFT, wavenumber multiplication, and IFFT) is insufficient; for example, the pointwise activation functions are not included at all. I do not understand why $u_x$ and $u_{xx}$ could be computed in such a simple way.
ut = (u[:, 2:, :] - u[:, :-2, :]) / (2 * dt)
. ForBenchmark with
autograd
To understand this question, I check the results against
autograd
. These are what I have done:FDM_Burgers()
andPINO_loss()
intrain_utils/losses.py
to expose the gradient outputs;autograd
results in the training methodtrain_2d_burger()
intrain_utils/train_2d.py
;train_burgers.py
to run debug on CPU.For quick reference, this is my code for the FD vs
autograd
comparison in step 2:If you replace the original source files with the attached three files and run
you should be able to get some outputs similar to the following:
As we can see, the differences between FD and$u_t$ are quite small, as expected, which also imply that I am using $u_x$ are exceedingly large, which seems to support my doubt that $u_x$ .
autograd
forautograd
correctly intrain_2d_burger()
. However, the differences forFDM_Burgers()
is insufficent forThe text was updated successfully, but these errors were encountered: