Reporter system, JSON library, and Geochemistry module help #15796

aeslaughter · 2020-09-03T00:48:56Z

aeslaughter
Sep 3, 2020

@WilkAndy I hoping you could help me with a problem I am seeing with the geochemistry module. I am working on creating a new system: Reporter system. This is a generalization of the Postprocessor and VectorPostprocessor system to allow for arbitrary types. This new system includes JSON output to handle complex/arbitrary types. To facility the JSON output I am changing the underlying library we use for JSON support.

I have attempted to update the geochemistry module to use this new library, but I have one test still failing and was hoping you could take a look. The offending test is a CSVDIFF and the values are completely different.

    DIFF FAILED (CSVDIFF) test:time_dependent_reactions.cooling

If you have time you can take a look? You can get the code here: https://github.com/aeslaughter/moose/tree/reporter-11323

Answered by WilkAndy

Sep 3, 2020

OK, the json problem is almost definitely that it's not parsing the "note" in the following correctly.

    "Al13O4(OH)24(7+)": {
      "species": {
        "Al+++": 13.0,
        "H+": -32.0,
        "H2O": 28.0
      },
      "charge": 7.0,
      "radius": 11.0,
      "molecular weight": 822.9328,
      "logk": [
        125.1868,
        98.7072,
        61.6358,
        19.2684,
        -33.6908,
        -86.65,
        -139.6092,
        -192.5684
      ],
      "note": "Missing array values in original database have been filled using a linear fit. Original values are [125.1868, 98.7072, 500.0000, 500.0000, 500.0000, 500.0000, 500.0000, 500.0000]"
    },

These lines of C++ are not wo…

View full answer

WilkAndy · 2020-09-03T07:59:13Z

WilkAndy
Sep 3, 2020

Sure, where do i see the diffs? I don't think i have time to pull from your branch and recompile - it takes my computer about an hour usually and i don't have that time to spare.

2 replies

aeslaughter Sep 3, 2020
Author

Here is the diff, unfortunately they are completely different. I will keep digging, but if you have any insight that would be great.

$ diff cooling_out.csv gold/cooling_out.csv 
4,31c4,31
< 10,11.54141898215,0.77688728905613,15.590522755031,8.4032251924934,290
< 20,8.200363308286,0,18.125665406182,9.9379916370605,280
< 30,6.0820051794106,0,19.449799930141,10.742118727918,270
< 40,4.7302780001717,0,20.294768075156,11.257647043208,260
< 50,3.9779908582935,0,20.765068476027,11.547403042803,250
< 60,3.5985924568425,0,21.002295439433,11.696687568705,240
< 70,3.4191948121143,0,21.114504785601,11.770578876017,230
< 80,3.3378110969368,0,21.165441499485,11.807420839502,220
< 90,3.3019145211899,0,21.187939575272,11.82692029575,210
< 100,3.2864158074652,0,21.197685017782,11.838460932358,200
< 110,3.2798558027864,0,21.201844567032,11.846312007676,190
< 120,3.2771504141633,0,21.203599298688,11.852361497845,180
< 130,3.2760886772454,0,21.204332542889,11.857432022236,170
< 140,3.2757210088493,0,21.204636559725,11.861883705647,160
< 150,3.2756398951534,0,21.204761802814,11.865880933997,150
< 160,3.275666891023,0,21.204813125078,11.869506386651,140
< 170,3.2757223595362,0,21.204834063846,11.872806044249,130
< 180,3.2757830691759,0,21.204842575292,11.875798942324,120
< 190,3.2759019342859,0,21.20484602432,11.878455598302,110
< 200,3.276323223148,0,21.20484741808,11.880616534481,100
< 210,3.2778256152027,0,21.204847979852,11.881758265474,90
< 220,3.282645136094,0,21.204848205703,11.880366121524,80
< 230,3.2968116661229,0,21.204848296257,11.872346226888,70
< 240,3.3356640026972,0,21.204848332454,11.847275733682,60
< 250,3.4356335479374,0,21.20484834687,11.780389215187,50
< 260,3.6754243418668,0,21.204848352583,11.618173590003,40
< 270,4.2010615890797,0,21.204848354828,11.261331280116,30
< 275,4.634744053004,0,21.204848355364,10.966626741676,25
---
> 10,18.94173279687,11.148792954471,4.9998037774527,2.0048258790889,290
> 20,17.924766315917,12.25305816618,5.0001605211656,2.011190510366,280
> 30,16.951632494155,13.309767095694,5.0004907970772,2.0186255929182,270
> 40,16.023798980675,14.31732819853,5.0007887751934,2.0267544239692,260
> 50,15.141901107302,15.275055487346,5.0010498588671,2.0352531945155,250
> 60,14.305960764259,16.182928567974,5.001271619614,2.0438577990397,240
> 70,13.515579338177,17.041380464231,5.0014542002143,2.0523642994991,230
> 80,12.77010288991,17.851116990883,5.0016001651923,2.0606243512705,220
> 90,12.068757946346,18.612969725984,5.0017139045309,2.0685373789824,210
> 100,11.410756583384,19.327783439827,5.0018007915981,2.0760412843555,200
> 110,10.795370054751,19.996338011507,5.0018663266297,2.0831030930646,190
> 120,10.221970976376,20.619304210294,5.001915455968,2.0897104594609,180
> 130,9.6900449650207,21.197232047494,5.0019521678553,2.0958645060896,170
> 140,9.1991736714807,21.73056955266,5.0019793678177,2.1015741448883,160
> 150,8.7489923680173,22.219708690819,5.0019989661355,2.1068518228424,150
> 160,8.3391266409887,22.665053697752,5.0020120826121,2.1117105413328,140
> 170,7.9691142411778,23.067105458274,5.0020192843129,2.1161619817061,130
> 180,7.6383196178186,23.426553885876,5.0020208021368,2.1202155889427,120
> 190,7.345849885228,23.744368868838,5.0020167035717,2.1238784846703,110
> 200,7.0904816533878,24.021879568356,5.0020070207392,2.1271560780064,100
> 210,6.8706079839728,24.260832022435,5.0019918424899,2.1300532155418,90
> 220,6.6842134448703,24.463416377556,5.0019713800385,2.1325756823356,80
> 230,6.5288826959783,24.632257726602,5.0019460121471,2.1347318754288,70
> 240,6.4018443707786,24.770368332619,5.0019163118179,2.1365345400957,60
> 250,6.3000477565976,24.881063558871,5.0018830536164,2.1380024971196,50
> 260,6.22026586412,24.967848471292,5.0018471994731,2.1391620745239,40
> 270,6.1592157092931,25.034286063106,5.0018098609538,2.140047441393,30
> 275,6.1347049796925,25.060970134853,5.0017910086353,2.1403997366763,25

aeslaughter Sep 3, 2020
Author

Here is the complete diff of the geochemistry module code, I am going through it again to see if I can figure out what I messed up.
reporter.log

WilkAndy · 2020-09-03T08:55:35Z

WilkAndy
Sep 3, 2020

By the way, you might like to run

cd geochemistry/test/tests/time_dependent_reactions/
cooling.i
cp cooling_out.csv gold/.
python cooling.py

to compare your results with the "gold standard" GWB software.

1 reply

aeslaughter Sep 3, 2020
Author

I did something wrong:

aeslaughter · 2020-09-03T14:50:56Z

aeslaughter
Sep 3, 2020
Author

On another note, the geochemistry/unit seems to have a segfault on upstream/next (166c9f3).

/Users/slauae/projects/moose2/modules/geochemistry/unit
$ lldb -- ./geochemistry-unit-devel
(lldb) target create "./geochemistry-unit-devel"
Current executable set to './geochemistry-unit-devel' (x86_64).
(lldb) b MPI_Abort
Breakpoint 1: where = libmpi.12.dylib`MPI_Abort, address = 0x000000000004fbb0
(lldb) r
Process 79850 launched: '/Users/slauae/projects/moose2/modules/geochemistry/unit/geochemistry-unit-devel' (x86_64)
Process 79850 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x2e0)
    frame #0: 0x0000000108c22959 libopenblasp-r0.3.8.dylib`dnrm2_ + 25
libopenblasp-r0.3.8.dylib`dnrm2_:
->  0x108c22959 <+25>: movq   0x2e0(%rax), %rax
    0x108c22960 <+32>: popq   %rbp
    0x108c22961 <+33>: jmpq   *%rax
    0x108c22963 <+35>: xorps  %xmm0, %xmm0
Target 0: (geochemistry-unit-devel) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x2e0)
  * frame #0: 0x0000000108c22959 libopenblasp-r0.3.8.dylib`dnrm2_ + 25
    frame #1: 0x000000010bea2a14 libopenblasp-r0.3.8.dylib`dlarfg_ + 68
    frame #2: 0x000000010be3a249 libopenblasp-r0.3.8.dylib`dgebd2_ + 505
    frame #3: 0x000000010be3adc6 libopenblasp-r0.3.8.dylib`dgebrd_ + 1766
    frame #4: 0x000000010be5140b libopenblasp-r0.3.8.dylib`dgesvd_ + 6667
    frame #5: 0x0000000102bb5018 libmesh_dbg.0.dylib`libMesh::DenseMatrix<double>::_svd_helper(char, char, std::__1::vector<double, std::__1::allocator<double> >&, std::__1::vector<double, std::__1::allocator<double> >&, std::__1::vector<double, std::__1::allocator<double> >&) + 680
    frame #6: 0x0000000102bb5377 libmesh_dbg.0.dylib`libMesh::DenseMatrix<double>::_svd_lapack(libMesh::DenseVector<double>&, libMesh::DenseMatrix<double>&, libMesh::DenseMatrix<double>&) + 311
    frame #7: 0x0000000102badedd libmesh_dbg.0.dylib`libMesh::DenseMatrix<double>::svd(libMesh::DenseVector<double>&, libMesh::DenseMatrix<double>&, libMesh::DenseMatrix<double>&) + 45
    frame #8: 0x00000001005232a0 libgeochemistry-devel.0.dylib`GeochemistrySpeciesSwapper::constructInverseMatrix(this=<unavailable>, mgd=<unavailable>, basis_index_to_replace=<unavailable>, eqm_index_to_insert=<unavailable>) at GeochemistrySpeciesSwapper.C:92:18 [opt]
    frame #9: 0x000000010050e68c libgeochemistry-devel.0.dylib`GeochemistrySpeciesSwapper::performSwap(this=0x00000001003af7f8, mgd=0x00000001003afca8, replace_this="Ca++", with_this="Calcite") at GeochemistrySpeciesSwapper.C:119:3 [opt]
    frame #10: 0x000000010050c45a libgeochemistry-devel.0.dylib`GeochemicalSystem::checkAndInitialize(this=<unavailable>, kin_name=<unavailable>, kin_initial_moles=<unavailable>) at GeochemicalSystem.C:175:16 [opt]
    frame #11: 0x000000010050db21 libgeochemistry-devel.0.dylib`GeochemicalSystem::GeochemicalSystem(this=0x00000001003b4170, mgd=<unavailable>, gac=<unavailable>, is=<unavailable>, swapper=<unavailable>, swap_out_of_basis=<unavailable>, swap_into_basis=size=1, charge_balance_species="H+", constrained_species=size=4, constraint_value=size=4, constraint_meaning=size=4, initial_temperature=25, iters_to_make_consistent=0, min_initial_molality=0.0000000000000000000099999999999999995, kin_name=size=0, kin_initial_moles=size=0) at GeochemicalSystem.C:136:3 [opt]
    frame #12: 0x00000001001c8ece libgeochemistry-unit-devel.0.dylib`_GLOBAL__sub_I_GeochemicalSystemTest.C [inlined] __cxx_global_var_init.788 at GeochemicalSystemTest.C:38:25 [opt]
    frame #13: 0x00000001001c8c0f libgeochemistry-unit-devel.0.dylib`_GLOBAL__sub_I_GeochemicalSystemTest.C at GeochemicalSystemTest.C:0 [opt]
    frame #14: 0x000000010002515d dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 539
    frame #15: 0x0000000100025582 dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
    frame #16: 0x000000010001fdc7 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 493
    frame #17: 0x000000010001fd32 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 344
    frame #18: 0x000000010001de58 dyld`ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 188
    frame #19: 0x000000010001def8 dyld`ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 82
    frame #20: 0x000000010000c6b1 dyld`dyld::initializeMainExecutable() + 199
    frame #21: 0x0000000100011cd0 dyld`dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 6797
    frame #22: 0x000000010000b227 dyld`dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLoaded const*, unsigned long*) + 453
    frame #23: 0x000000010000b025 dyld`_dyld_start + 37
(lldb) q

1 reply

WilkAndy Sep 3, 2020

Thanks for that. it needs a bit more digging to fix, which is a bit complicated since i don't get failures on any of my systems..... In geochemistry there are a tonne of unit tests, which really need to be checked by CIVET so ensure everything is OK with all architectures/compilers.

a

WilkAndy · 2020-09-03T20:26:12Z

WilkAndy
Sep 3, 2020

OK, the json problem is almost definitely that it's not parsing the "note" in the following correctly.

    "Al13O4(OH)24(7+)": {
      "species": {
        "Al+++": 13.0,
        "H+": -32.0,
        "H2O": 28.0
      },
      "charge": 7.0,
      "radius": 11.0,
      "molecular weight": 822.9328,
      "logk": [
        125.1868,
        98.7072,
        61.6358,
        19.2684,
        -33.6908,
        -86.65,
        -139.6092,
        -192.5684
      ],
      "note": "Missing array values in original database have been filled using a linear fit. Original values are [125.1868, 98.7072, 500.0000, 500.0000, 500.0000, 500.0000, 500.0000, 500.0000]"
    },

These lines of C++ are not working properly:

    if (_root["secondary species"][name].isMember("note"))
      _root["secondary species"].removeMember(name);
}

1 reply

aeslaughter Sep 3, 2020
Author

That did the trick. I was removing elements while iterating. You are my friend.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reporter system, JSON library, and Geochemistry module help #15796

{{title}}

Replies: 4 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Reporter system, JSON library, and Geochemistry module help #15796

aeslaughter Sep 3, 2020

Replies: 4 comments · 5 replies

WilkAndy Sep 3, 2020

aeslaughter Sep 3, 2020 Author

aeslaughter Sep 3, 2020 Author

WilkAndy Sep 3, 2020

aeslaughter Sep 3, 2020 Author

aeslaughter Sep 3, 2020 Author

WilkAndy Sep 3, 2020

WilkAndy Sep 3, 2020

aeslaughter Sep 3, 2020 Author

aeslaughter
Sep 3, 2020

Replies: 4 comments 5 replies

WilkAndy
Sep 3, 2020

aeslaughter Sep 3, 2020
Author

aeslaughter Sep 3, 2020
Author

WilkAndy
Sep 3, 2020

aeslaughter Sep 3, 2020
Author

aeslaughter
Sep 3, 2020
Author

WilkAndy
Sep 3, 2020

aeslaughter Sep 3, 2020
Author