Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix invalid CUDA device function error in HPMG #813

Conversation

AlexanderSinn
Copy link
Member

@AlexanderSinn AlexanderSinn commented Nov 9, 2022

Replace template and if constexpr with good old overloading for NVCC 11.4

From Severin:

We found that the laser envelope crashes on Maxwell for the following input script:

my_constants.kp_inv = 10.e-6
my_constants.kp = 1. / kp_inv
my_constants.wp = clight * kp
my_constants.ne = wp^2 * m_e * epsilon0 / q_e^2

max_step = 50
hipace.dt = 10.*kp_inv/clight

hipace.verbose = 0

amr.n_cell = 512 512 1000

hipace.do_tiling = 0

geometry.prob_lo     = -25.6*kp_inv   -25.6*kp_inv   -13.*kp_inv
geometry.prob_hi     =  25.6*kp_inv    25.6*kp_inv    0.*kp_inv

laser.a0 = 2.5
laser.use_laser = 1
laser.position_mean = 0. 0. -4*kp_inv
laser.w0 = 3.2*kp_inv 3.2*kp_inv
laser.L0 = 2.26*kp_inv * .57 # .57 is a rough conversion factor between cos**2 and exp profiles
laser.lambda0 = 2*pi/34*kp_inv
laser.solver_type = multigrid

hipace.bxby_solver=explicit

amr.blocking_factor = 2
amr.max_level = 0

hipace.output_period = 1

hipace.numprocs_x = 1
hipace.numprocs_y = 1

# hipace.depos_order_xy = 2

geometry.coord_sys   = 0                  # 0: Cartesian
geometry.is_periodic = true  true  false  # Is periodic?

beams.names = no_beam

plasmas.names = plasma
my_constants.zramp = 20*kp_inv
plasma.density(x,y,z) = ne * if(z>zramp,1,z/zramp) * if(z>0,1,0)
plasma.ppc = 2 2
plasma.u_mean = 0.0 0.0 0.
plasma.element = electron
plasma.radius = 23.*kp_inv

diagnostic.diag_type = xz

with the following error:

amrex::Abort::0::GPU last error detected in file /home/diederse/software/hipace/build/_deps/fetchedamrex-src/Src/Base/AMReX_GpuLaunchFunctsG.H line 868: invalid device function !!!
SIGABRT
See Backtrace.0 file for details

This PR fixes the issue

  • Small enough (< few 100s of lines), otherwise it should probably be split into smaller PRs
  • Tested (describe the tests in the PR description)
  • Runs on GPU (basic: the code compiles and run well with the new module)
  • Contains an automated test (checksum and/or comparison with theory)
  • Documented: all elements (classes and their members, functions, namespaces, etc.) are documented
  • Constified (All that can be const is const)
  • Code is clean (no unwanted comments, )
  • Style and code conventions are respected at the bottom of https://github.com/Hi-PACE/hipace
  • Proper label and GitHub project, if applicable

@SeverinDiederichs
Copy link
Member

I confirm that this indeed fixes the issue.

@AlexanderSinn AlexanderSinn added bug Something isn't working GPU Related to GPU acceleration labels Nov 9, 2022
@SeverinDiederichs SeverinDiederichs merged commit 763998a into Hi-PACE:development Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working GPU Related to GPU acceleration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants