Stack overflow checkpoint file with binary file

[JelmerPolman]

Bug description
In some cases, writing a checkpoint file in combination with a binary output file leads to a stack overflow. This issue is more likely to happen with an increasing TMax.

To Reproduce
I have been able to reproduce this problem with Test18.

1. Use the windows executable distributed with release v2.2.0, see OpenFAST exe files #361.
2. Run Test18 5MW_Land_DLL_WTurb, with TMax = 700, ChkptTime = 20 and OutFileFmt = 2 (or 3)
3. After 20 seconds, upon writing the checkpoint file the simulation crashes with stack overflow

Note that this might be system depended. It does not appear with my own linux-intel compiled executable.

Expected behavior
To my knowledge, the binary is written only at the end of the simulation. It is also much smaller in size than the text file, which is written during the simulation. If anything, I would expect that writing a text file in combination with the checkpoint file leads to a stack overflow, rather than the binary file. However, since the checkpoint files are only about 33 mb in size, I wouldn't expect a stack overflow at all.

Screenshots, if applicable

**************************************************************************************************
OpenFAST

Copyright (C)  National Renewable Energy Laboratory
Copyright (C)  Envision Energy USA LTD

This program is licensed under Apache License Version 2.0 and comes with ABSOLUTELY NO WARRANTY.
See the "LICENSE" file distributed with this software for details.
**************************************************************************************************

OpenFAST-v2.2.0
Compile Info:
 - Architecture: 64 bit
 - Precision: single
 - Date: Dec 20 2019
 - Time: 15:19:00
Execution Info:
 - Date: 03/03/2020
 - Time: 07:56:12+0100

OpenFAST input file heading:
    FAST Certification Test #18: NREL 5.0 MW Baseline Wind Turbine (Onshore)

Running ElastoDyn.
Running AeroDyn.
Running AirfoilInfo.
Running BEM.
Running UnsteadyAero.
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 1, Blade = 1
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 2, Blade = 1
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 3, Blade = 1
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 4, Blade = 1
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 1, Blade = 2
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 2, Blade = 2
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 3, Blade = 2
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 4, Blade = 2
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 1, Blade = 3
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 2, Blade = 3
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 3, Blade = 3
Warning: Turning off Unsteady Aerodynamics because C_nalpha is 0.  BladeNode = 4, Blade = 3
Running InflowWind.

   Reading a 31x31 grid (145 m wide, 17.5 m to 162.5 m above ground) with a characteristic wind
   speed of 12 m/s. This full-field file was generated by TurbSim (v1.06.00, 21-Sep-2012) on
   07-Jan-2014 at 12:50:45.

   Processed 1442 time steps of 20-Hz full-field data (72.05 seconds).
Running ServoDyn.
Running ServoDyn Interface for Bladed Controllers.
 Time: 0 of 1200 seconds.

FAST_Solution0:CalcOutputs_And_SolveForInputs:SolveOption2:SrvD_CalcOutput:Running with torque
and pitch control of the NREL offshore 5MW baseline wind turbine from DISCON.dll as written by J.
Jonkman of NREL/NWTC for use in the IEA Annex XXIII OC3 studies.
forrtl: severe (170): Program Exception - stack overflow 08:00:47.
Image              PC                Routine            Line        Source
OpenFAST_v2_2_0_S  00007FF6E28828C8  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6DFA87F3B  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6DFA795FC  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6DF9867C7  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6DF9865C1  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6DF97182F  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6E2765492  Unknown               Unknown  Unknown
OpenFAST_v2_2_0_S  00007FF6E2882AD4  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFFB0087974  Unknown               Unknown  Unknown
ntdll.dll          00007FFFB099A261  Unknown               Unknown  Unknown

[bjonkman]

I think this is the same problem as described in #99.

The binary files must store all the data for the output file so that it can scale at the end of the simulation and then write the whole file at one time. Text files don't store that data (it gets written at each time step). Stack overflow usually occurs because the code is doing some operation on an array that contains more data than is allowed on the stack at one time. The size of the stack varies by compiler/linker and build options, too.

I rewrote the routines where the stack overflow is likely taking place in #373, so you might try that version and see if it is still an issue.

[JelmerPolman]

Thank you for your reply.

I have never gotten myself to compiling OpenFAST with windows, only on linux. However, on linux the problem does not occure. Therefore I cannot try your version to see if its an issue.

On the other hand, it occurs rarely with long simulation times. A workaround is to choose the text file as output instead of the binary, or dismiss writing out checkpoint files. If it is expected behavior, than I would delete the Bug label and close this issue.