Metaphysicl problem?

[makeclean]

Hi MOOSEers

I've got a problem which I'm running it started out as an IGA based problem, and now I've migrated to regular elements to debug whats going on to no avail. My problem is a fully non-linear thermo-mechanical problem, when I remove the mechanical aspects of the problem and make it fully thermal it runs & converges fine. However, when i run the coupled thermal-mechanical problem I get the following error

+----------------+---------------------+
| time           | average_temperature |
+----------------+---------------------+
|   0.000000e+00 |        0.000000e+00 |
+----------------+---------------------+


    Computing Initial Residual
      Computing Residual..                                                               [ 18.15 s] [  -48 MB]
    Finished Computing Initial Residual                                                  [ 18.15 s] [  -48 MB]

Performing automatic scaling calculation

/home/dc-davi4/rds/rds-ukaea-ap001/moose_dev/moose/scripts/../libmesh/installed/include/metaphysicl/dynamic_std_array_wrapper.h, line 125, compiled Sep 22 2022 at 23:53:34

The particular funciton is

  void resize(size_type new_size)
  {
    if (new_size > N)
      metaphysicl_error();
    _dynamic_n = new_size;
  }

and N is static const std::size_t N = NType::size so I understand why. What sets N, is this related to the derivative size, do I need to recompile and make the it size bigger? I'm already configuring with ./configure --with-derivative-size=200 --with-ad-indexing-type=global which has dealt with all of my problems so far.

Thanks

makeclean

[GiudGiud]

@recuero @dschwen if you could help on this please.
Thermo-mechanical seems to require very very large derivative sizes.

[recuero]

Coupling more variables will necessarily increase the required AD containers size. My suggestions would be to further increase the containers size you configure MOOSE with. If doing interface problems, do not make the primary surface mesh size finer.

I've had to go beyond 200 for 3D thermomechanical interface problems.

[GiudGiud]

Has someone run the number on why so many derivative dependences? Do every qp variable value end up relying on all the qps nearby for displacement for example?

[recuero]

I have run the number for a few of these problems and you can come up with a value of 300 or more depending on your problem dimensionality, number of coupled of variables, refinement of interface surfaces. Every qp will depend on all the element variables and then all the element variables involved in interface projections. MetaPhysicL was providing the right dependencies as far as I could tell.

We have some simple code in the heat conduction and contact modules that remove interior node derivatives when Lagrange elements are used (since interior nodes do not affect the interface problem in that case).

[makeclean]

Ok, recompiled with --with-derivative-size=300 and my problem is now running. Do you think it would be helpful to print an error message that maybe even states you should recompile with a larger derivative size, given we know the size of new_size one could even prompt the user what that size should be?

[recuero]

Is the "heat flux" (in your image) an interface problem or another boundary condition? I asked for the image because I was wondering if you could have a thermal pair in which the primary surface would be meshed finer than the secondary surface. Having this setup will add a significant number of dependencies to your AD objects. But I cannot see that situation in this image (or are the corresponding pairs not in the image?).

[makeclean]

Just another BC....

[recuero]

I can't immediately see why you need such a high AD container size value. Do you have any nonlocal computation in which you use displacement/temperature variables from a number of elements? Is the pressure BC just applying a constant pressure value to the entire sideset?

[makeclean]

Pressure is just applying constant pressure yeah over the entire sideset. Nothing non-local no.

[roystgnr]

given we know the size of new_size one could even prompt the user what that size should be?

Discussion starting over in roystgnr/MetaPhysicL#84 if you want to chime in. This is a great goal, but the trouble is that we only know new_size for the particular resize operation that first triggers failure, and that might not be the largest resize that ever gets performed for a particular problem.

its consistent between hex & IGA

This is what I would hope (assuming the same p for each). You get wider sparsity in the final matrix with IGA, but that's due to the subsequent application of constraint equations, after the sparsity is no longer in the hands of MetaPhysicL.

[makeclean]

Sorry, took a while to get here, but here is the GDB outtput,

122       void resize(size_type new_size)
123       {
124         if (new_size > N)
125           metaphysicl_error();
126         _dynamic_n = new_size;
127       }

new_size is 54 and N is 53, I get the same error from 2nd order Hex problem & lagrange and IGA with 2nd order bernstein. So I'm starting to think its something else. Here is my smallest mesh & input that causes the problem, I haven't started backing off the physics yet.

hex-metaphysicl.tar.gz

[GiudGiud]

this looks like another error. Unrelated to the AD size.
@lindsayad you seen this one before?

[lindsayad]

My best guess is derivative insertion during residual evaluation or access into uninitialized memory (which would be identifiable with valgrind)

[roystgnr]

That definitely looks like an OOB write. Not guaranteed to be caught with valgrind, but valgrind's the best chance to catch it.

[richmondodufisan]

Thanks, valgrind worked! Took me a while to figure it out but I mistakenly attempted to consume a material property with getParam (within a custom Material). So the error was from me indexing the material property at the [_qp]. Surprised this wasn't caught as a compile error but its fixed now.

[lindsayad]

great to hear!