Automated checking against MD-F

[RyanDavies19]

It would be good to add automated tests that check against MDF, because OpenFAST has a pretty robust regression testing scheme. This would also ensure codes stay synchronized in terms of performance.

[sanguinariojoe]

I was checking that the other day. It would take me some time though

[RyanDavies19]

I have the beginnings of a script written in Python, it handles creating a MDF driver file and running both on the same input file as well as plotting outputs. I could clean it up and get it more functional for comparing output values, but I'm not super familiar with how to use github actions to run it.

[sanguinariojoe]

Upload it to a branch on your fork and head me there I think I can automatize it on the CI/CD

…

On Fri, 31 May 2024, 18:04 RyanDavies19, ***@***.***> wrote: I have the beginnings of a script written in Python, it handles creating a MDF driver file and running both on the same input file as well as plotting outputs. I could clean it up and get it more functional for comparing output values, but I'm not super familiar with how to use github actions to run it. — Reply to this email directly, view it on GitHub <#219 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMXKKCT7OJSUYFBP24SCBDZFCNSFAVCNFSM6AAAAABIRHIODOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBSGU3DCOBYGI> . You are receiving this because you commented.Message ID: ***@***.***>

[RyanDavies19]

@sanguinariojoe sounds good, I will work on this next week when I have some free time

[RyanDavies19]

@sanguinariojoe it took me a little longer than expected but see here: https://github.com/RyanDavies19/MoorDynC_ryan/tree/mdf_verify

I added some FIXME's where paths could/need to be adjusted based on what paths GH actions is using. I also added a test case that uses a lot of different features (aside from non-linear stiffness, bending stiffness, and water kinematics). Let's discuss when you have time to review, no rush

[sanguinariojoe]

Hey!

As far as I can see just a single regression test is ran. I thought more reg-tests were considered on MoorDynF, but apparently that is not the case...

https://github.com/OpenFAST/openfast/blob/6a7a543790f3cad4a65b87242a619ac5b34b4c0f/reg_tests/CTestList.cmake#L406

I am also seeing very ugly things loading the dylib... As long as we are using Python, I would rather apply the Python wrapper straight away. I think I can simplify this a lot.

One question I have is... When shall we run the MDF vs. MDC regression tests? On every dev commit/PR? Or just on the releases?

[sanguinariojoe]

I think I will start implementing an action just to clone and compile OpenFAST, and then run the MoorDyn regression tests

[RyanDavies19]

Hey!

As far as I can see just a single regression test is ran. I thought more reg-tests were considered on MoorDynF, but apparently that is not the case...

https://github.com/OpenFAST/openfast/blob/6a7a543790f3cad4a65b87242a619ac5b34b4c0f/reg_tests/CTestList.cmake#L406

I am also seeing very ugly things loading the dylib... As long as we are using Python, I would rather apply the Python wrapper straight away. I think I can simplify this a lot.

One question I have is... When shall we run the MDF vs. MDC regression tests? On every dev commit/PR? Or just on the releases?

I just added to the python driver in this framework. I previously didn't include it because it seemed simpler to interface with just the library rather than go through the wrapper.

OpenFAST also runs regression tests for fully coupled simulations: https://github.com/OpenFAST/r-test/tree/548bf2143633a7066f4582167fdf4180951fd7f6/glue-codes/openfast that include MoorDyn

[RyanDavies19]

I would say we should do it on every dev PR because releases are not extremely common and it would be best to catch differences early.

I only included one comparison test for now in this while we get this up and running. Once that is done we can add more that look at different features.

[RyanDavies19]

I think I will start implementing an action just to clone and compile OpenFAST, and then run the MoorDyn regression tests

If it helps I also included a function to compile the moordyn_driver from OpenFAST in this framework. We don't need to compile all of OpenFAST for this testing, just the MD driver.

[sanguinariojoe]

Fixed on #224