-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Binary output : writing out ghost cells? #236
Comments
unfortunately, looping over slices if a lot slower. IT’s all the system calls. The ascii output option is horribly slow,
which is why I had to go to the binary format.
How about reading it all in in binary, including ghost cells and copying it in matlab, if you don’t want
it all. Doing it on the matlab side might not be as slow.
Marsha
… On Dec 26, 2018, at 8:15 AM, Donna Calhoun ***@***.***> wrote:
@mjberger <https://github.com/mjberger> requested a feature to read binary data in the Matlab graphics code. This is fairly straightforward to add, but would be even easier if the ghost cell data weren't written out.
In valout.f90 (line 172 in the 2d code), the relevant output line is
case(3)
! Note: We are writing out ghost cell data also
i = (iadd(num_eqn, num_cells(1) + 2 * num_ghost, &
num_cells(2) + 2 * num_ghost))
write(out_unit + 1) alloc(iadd(1, 1, 1):i)
It is obvious why writing out ghost cell data here is easier than trying to loop over interior data only. But it wouldn't be that hard to modify the code slightly so that the write statement is in a subroutine in which the data in alloc(istart:iend) is reshaped so that one can output only the interior slice . Something like :
call write_slice(mx, my,meqn,mbc,alloc(iadd(1,1,1):i),out_unit+1)
where
subroutine write_slice(mx, my,meqn,mbc,q,unitnum)
...
double precision q(meqn,1-mbc:mx+mbc,1-mbc:my+mbc)
write(unitium) q(1:meqn,1:mx,1:my)
end
Performance shouldn't be an issue, or in any case, I wouldn't expect it to be any slower than the looping that is done for the ascii output.
Would this be a useful addition to valout.f90 (both 2d and 3d versions)? Or, if someone sees an obvious problem with the proposed code above, please comment.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#236>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA1DC-51hOS5uavXW8wZGSHlsV9PjKQUks5u83Z1gaJpZM4Zhvfh>.
|
@mjberger Okay. I was hoping that using the F90 slicing feature would not be as slow as manually looping over i.j. It isn't that big a deal to strip out the ghost cells in Matlab. |
I would imagine the compiler could inline something like that. Have you tested the slow-down by chance? |
I am testing it now - and it seems that there is no slow-down, and in fact, writing out fewer values is faster (e.g. 0.044s vs. 0.059s). So it does seem that the compiler is able to handle the slicing efficiently. |
I will time myself too
But you can’t change it when he python expects the whole thing
From the car, mjb
… On Dec 26, 2018, at 11:39 AM, Donna Calhou ***@***.***> wrote:
I am testing it now - and it seems that there is no slow-down, and in fact, writing out fewer values is faster (e.g. 0.044s vs. 0.059s). So it does seem that the compiler is able to handle the slicing efficiently.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Reading binary output in Matlab is faster than reading the ascii, but this may not speed up the 3d plotting as much as one might hope. Rather, the code slows down as the number of 3d slices increases. At two different places in the code, I do basically an all-to-all comparison with slices at different orientations (x,y,z) to mask patches and to plot lines at slice intersections. The lines can be suppressed, but the masking is needed so the plotting looks okay. These bottlenecks can probably be sped up with more efficient algorithms. Anyone want to look into this? In the meantime, set as few slices as necessary, i.e. in
Note : users can loop over multiple slices using the
for more information on Clawpack Matlab graphics commands. |
On a related note, at one point I was experimenting with creating arrays in visclaw that included the ghost cells too when reading in the binary output, after making sure these were properly set at the time they are written out in |
I actually think it would be fairly easy to specify how many ghost cells to output. Of course, it would require an extra parameter - something like On the other hand, allowing the graphics to make use of the ghost cells is another story ... |
what about geoclaw output? Is it consistent with amrclaw?
— Marsha
… On Dec 26, 2018, at 6:18 PM, Donna Calhoun ***@***.***> wrote:
I've just issued a PR <#237> to AMRClaw and with my proposed changes for the binary output. I've also issued one for Visclaw PR <clawpack/visclaw#235> with the binary reader for Matlab.
Especially in 3d, not writing out the ghost cells seems to save a lot in both time writing output and saves in storage.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#236 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA1DCzR0Qc7oUobGNjm2rBA4KtlHDfACks5u9APcgaJpZM4Zhvfh>.
|
I haven't done anything with GeoClaw output. And I don't really support GeoClaw plotting in Matlab - mostly because I am guessing nobody uses Matlab with GeoClaw. But I could do something if it were useful. |
@mjberger requested a feature to read binary data in the Matlab graphics code. This is fairly straightforward to add, but would be even easier if the ghost cell data weren't written out.
In
valout.f90
(line 172 in the 2d code), the relevant output line isIt is obvious why writing out ghost cell data here is easier than trying to loop over interior data only. But it wouldn't be that hard to modify the code slightly so that the write statement is in a subroutine in which the data in
alloc(istart:iend)
is reshaped so that one can output only the interior slice . Something like :where
Performance shouldn't be an issue, or in any case, I wouldn't expect it to be any slower than the looping that is done for the ascii output.
Would this be a useful addition to
valout.f90
(both 2d and 3d versions)? Or, if someone sees an obvious problem with the proposed code above, please comment.The text was updated successfully, but these errors were encountered: