Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregation + Append causes corrupt data #2482

Closed
pnorbert opened this issue Oct 6, 2020 · 0 comments
Closed

Aggregation + Append causes corrupt data #2482

pnorbert opened this issue Oct 6, 2020 · 0 comments
Labels

Comments

@pnorbert
Copy link
Contributor

pnorbert commented Oct 6, 2020

Describe the bug
Writing to an existing BP file with Append mode only works correctly if there is no aggregation (in either step).
If there is aggregation, the appended step's data will be corrupt (even if the same aggregation ratio is used).

To Reproduce

  1. in testing/adios2/engine/bp/TestBPWriteAppendReadADIOS2.cpp, lines 173 and 324 (after io.AddTransport)
  • add io.SetParameter("SubStreams", "1");
  1. run the test in build directory:
$ mpirun -n 4 bin/Test.Engine.BP.WriteAppendReadADIOS2.MPI --gtest_filter=BPWriteAppendReadTestADIOS2.ADIOS2BPWriteAppendRead2D2x4 BP4
  1. bpls shows that the second batch of steps (3..5) are corrupt
$ ./bin/bpls -la ADIOS2BPWriteAppendRead2D2x4Test.bp -d i16 -n 16
  int16_t         i16            6*{2, 16} = -509 / 543
    (0,0, 0)    513 514 -509 516 514 515 -508 517 515 516 -507 518 516 517 -506 519
    (0,1, 0)    -507 518 -505 520 -506 519 -504 521 -505 520 -503 522 -504 521 -502 523
    (1,0, 0)    517 518 -505 520 518 519 -504 521 519 520 -503 522 520 521 -502 523
    (1,1, 0)    -503 522 -501 524 -502 523 -500 525 -501 524 -499 526 -500 525 -498 527
    (2,0, 0)    521 522 -501 524 522 523 -500 525 523 524 -499 526 524 525 -498 527
    (2,1, 0)    -499 526 -497 528 -498 527 -496 529 -497 528 -495 530 -496 529 -494 531
    (3,0, 0)    525 526 -497 528 622 0 0 0 527 528 -495 530 622 0 0 0
    (3,1, 0)    -495 530 -493 532 28160 0 0 0 -493 532 -491 534 28160 0 0 0
    (4,0, 0)    529 530 -493 532 0 0 0 28160 531 532 -491 534 0 0 0 28160
    (4,1, 0)    -491 534 -489 536 0 0 0 0 -489 536 -487 538 0 0 0 0
    (5,0, 0)    533 534 -489 536 0 0 28160 0 535 536 -487 538 0 0 28160 0
    (5,1, 0)    -487 538 -485 540 0 0 0 1134 -485 540 -483 542 0 0 0 1134
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant