Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

last rank needs to resize to remove slipped beam particles from previous time step #426

Merged
merged 1 commit into from
Mar 19, 2021

Conversation

SeverinDiederichs
Copy link
Member

@SeverinDiederichs SeverinDiederichs commented Mar 16, 2021

This PR resolves #424.

In the last time step, the slipped beam particles need to be removed, because they don't belong to the current time step anymore.

Assuming we have 2 boxes (= 2 ranks) with 10 particles per box. In the last time step, 2 particles slip from the first to the second box. if that happened in a previous time step, the original 10 particles would be send to the next rank and then sorted there. In the last time step, the particles are not send. Now, when calculating the second box, the rank has its original 10 particles in that box + the two slipped particles, which are already one time step ahead. Despite being wrong, the simulation crashed in IO, because now the number of particles exceeded the total number of particles of the beam.

This PR fixes this problem: in Notify, the particles which should have been sent to the next rank are removed by resizing (in the same way they would have been removed, if they had been sent).

This problem was tested with the following input script:

amr.n_cell = 127 127 300

hipace.normalized_units=1
hipace.predcorr_B_mixing_factor = 0.05
hipace.predcorr_max_iterations = 30
hipace.predcorr_B_error_tolerance = 4e-2
hipace.verbose = 1

hipace.output_slice = 1
diagnostic.diag_type=xz
diagnostic.field_data = ExmBy Ez rho

amr.blocking_factor = 1
amr.max_level = 0

max_step = 30
hipace.output_period = 1
#hipace.do_adaptive_time_step = 1
hipace.dt=3

hipace.numprocs_x = 1
hipace.numprocs_y = 1

hipace.depos_order_xy = 2

geometry.coord_sys   = 0                  # 0: Cartesian
geometry.is_periodic = 1     1     0      # Is periodic?
geometry.prob_lo     = -12.   -12.   -5    # physical domain
geometry.prob_hi     =  12.    12.    5

beams.names = beam #beam2
beam.profile = gaussian
beam.injection_type = fixed_weight
beam.num_particles = 1000000
beam.density = 50.
beam.u_mean = 0. 0. 50 #0
beam.u_std = 0. 0. 0.
beam.position_mean = 0. 0. 0
beam.position_std = 0.1 0.1 1.2
#beam.do_symmetrize=1
#beam.do_z_push = 0

plasma.density = 1.
plasma.radius = 1.3
plasma.ppc = 5 5
plasma.u_mean = 0.0 0.0 0.

Running on 6 CPUs, it fails on development and gives the correct results with this PR.

  • Small enough (< few 100s of lines), otherwise it should probably be split into smaller PRs
  • Tested (describe the tests in the PR description)
  • Runs on GPU (basic: the code compiles and run well with the new module)
  • Contains an automated test (checksum and/or comparison with theory)
  • Documented: all elements (classes and their members, functions, namespaces, etc.) are documented
  • Constified (All that can be const is const)
  • Code is clean (no unwanted comments, )
  • Style and code conventions are respected at the bottom of https://github.com/Hi-PACE/hipace
  • Proper label and GitHub project, if applicable

@SeverinDiederichs SeverinDiederichs added bug Something isn't working component: beam About the beam species component: diagnostics About any types of diagnostics labels Mar 16, 2021
@MaxThevenet MaxThevenet merged commit 355617a into development Mar 19, 2021
@SeverinDiederichs SeverinDiederichs deleted the fix_beam_slippage branch May 7, 2021 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component: beam About the beam species component: diagnostics About any types of diagnostics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: simulation crashing with slipping beam particles in parallel runs
2 participants