Build SU2 and run regressions on self hosted aarch64 runner scheduled #1619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

martin-g merged 97 commits into su2code:develop from martin-g:build-on-self-hosted-aarch64-runner-scheduled-2

Jun 30, 2022

.github/workflows/regression-arm64.yml

-Original file line number
+Diff line change
@@ -0,0 +1,13 @@
+    name: Regression tests running on Linux ARM64
+    on:
+      push:
+        branches:
+          - 'develop'
+    jobs:
+      build_on_arm64:
+        name: Run the jobs on Linux ARM64
+        uses: './.github/workflows/regression.yml'
+        with:
+          runner: 'ARM64'

.github/workflows/regression.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -9,6 +9,13 @@ on:
  
        branches: 

          - 'develop'

          - 'master'

      workflow_call:

        inputs:

          runner:

            description: 'The github runner to use'

            default: 'ubuntu-latest'

            required: false

            type: string

    jobs:

      build:

    @@ -36,25 +43,37 @@ jobs:
  
                flags: '-Denable-autodiff=true -Denable-normal=false -Dwith-omp=true -Denable-mixedprec=true -Denable-tecio=false --warnlevel=3 --werror'

              - config_set: ForwardOMP

                flags: '-Denable-directdiff=true -Denable-normal=false -Dwith-omp=true -Denable-mixedprec=true -Denable-tecio=false --warnlevel=3 --werror'

        runs-on: ubuntu-latest

        runs-on: ${{ inputs.runner || 'ubuntu-latest' }}

        steps:

          - name: Cache Object Files

            uses: actions/cache@v1

            uses: actions/cache@v3

            with:

              path: ccache

              key: ${{ matrix.config_set }}-${{ github.sha }}

              restore-keys: ${{ matrix.config_set }}

          - name: Pre Cleanup 

            uses: docker://ghcr.io/su2code/su2/build-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

          - name: Build 

            uses: docker://su2code/build-su2:20191105

            uses: docker://ghcr.io/su2code/su2/build-su2:220614-1237

            with:

              args: -b ${{github.ref}} -f "${{matrix.flags}}"

          - name: Compress binaries

            run: tar -zcvf install_bin.tgz install/bin

          - name: Upload Binaries 

            uses: actions/upload-artifact@v1

            uses: actions/upload-artifact@v3

            with:

              name: ${{ matrix.config_set }}

              path: install/bin

              path: install_bin.tgz

          - name: Post Cleanup

            uses: docker://ghcr.io/su2code/su2/build-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

      regression_tests:

        runs-on: ubuntu-latest

        runs-on: ${{ inputs.runner || 'ubuntu-latest' }}

        name: Regression Tests

        needs: build

        strategy:

    @@ -79,22 +98,44 @@ jobs:
  
              - testscript: 'hybrid_regression_AD.py'

                tag: OMP

        steps:

          - name: Download All artifact

            uses: actions/download-artifact@v2

          - name: Move Binaries

          - name: Pre Cleanup

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

          - name: Download All artifacts

            uses: actions/download-artifact@v3

          - name: Uncompress and Move Binaries

            run: |

              mkdir -p install/bin

              if [ -d "${{format('Base{0}', matrix.tag)}}" ];    then cp -r ${{format('Base{0}', matrix.tag)}}/.    install/bin/; fi

              if [ -d "${{format('Reverse{0}', matrix.tag)}}" ]; then cp -r ${{format('Reverse{0}', matrix.tag)}}/. install/bin/; fi

              if [ -d "${{format('Forward{0}', matrix.tag)}}" ]; then cp -r ${{format('Forward{0}', matrix.tag)}}/. install/bin/; fi

              chmod a+x install/bin/*

              BIN_FOLDER="$PWD/install/bin"

              mkdir -p $BIN_FOLDER

              ls -lah $BIN_FOLDER

              for type in Base Reverse Forward; do

                TYPE_FOLDER="${type}${{matrix.tag}}"

                echo "Processing '$TYPE_FOLDER' ..."

                if [ -d $TYPE_FOLDER ]; then

                  pushd $TYPE_FOLDER

                  ls -lah

                  tar -zxvf install_bin.tgz

                  ls -lah install/bin/

                  cp -r install/bin/* $BIN_FOLDER;

                  popd; 

                fi

              done

              chmod a+x $BIN_FOLDER/*

              ls -lahR $BIN_FOLDER

          - name: Run Tests in Container

            uses: docker://su2code/test-su2:20200303

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              # -t <Tutorials-branch> -c <Testcases-branch>

              args: -b ${{github.ref}} -t develop -c develop -s ${{matrix.testscript}}

          - name: Cleanup 

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

      unit_tests:

        runs-on: ubuntu-latest

        runs-on: ${{ inputs.runner || 'ubuntu-latest' }}

        name: Unit Tests

        needs: build

        strategy:

    @@ -109,26 +150,54 @@ jobs:
  
              - testdriver: 'test_driver_DD'

                tag: MPI

        steps:

          - name: Pre Cleanup 

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

          - name: Download Base

            uses: actions/download-artifact@v1

            uses: actions/download-artifact@v3

            with:

              name: ${{format('Base{0}', matrix.tag)}}

              path: ${{format('Base{0}', matrix.tag)}}

          - name: Download Reverse

            uses: actions/download-artifact@v1

            uses: actions/download-artifact@v3

            with:

              name: ${{format('Reverse{0}', matrix.tag)}}

              path: ${{format('Reverse{0}', matrix.tag)}}

          - name: Download Forward

            uses: actions/download-artifact@v1

            uses: actions/download-artifact@v3

            with:

              name: ${{format('Forward{0}', matrix.tag)}}

          - name: Move Binaries

              path: ${{format('Forward{0}', matrix.tag)}}

          - name: Uncompress and Move Binaries

            run: |

              mkdir -p install/bin

              cp -r ${{format('Base{0}', matrix.tag)}}/. install/bin/

              cp -r ${{format('Reverse{0}', matrix.tag)}}/. install/bin/

              cp -r ${{format('Forward{0}', matrix.tag)}}/. install/bin/

              chmod a+x install/bin/*

              ls -lah

              BIN_FOLDER="$PWD/install/bin"

              mkdir -p $BIN_FOLDER

              ls -laH $BIN_FOLDER

              for type in Base Reverse Forward; do

                TYPE_FOLDER="${type}${{matrix.tag}}"

                if [ -d $TYPE_FOLDER ]; then

                  echo "Processing '$TYPE_FOLDER' ..."

                  pushd $TYPE_FOLDER

                  ls -lah

                  tar -zxvf install_bin.tgz

                  ls -laH install/bin/

                  cp -r install/bin/* $BIN_FOLDER;

                  popd;

                else

                  echo "$TYPE_FOLDER does not exist!"

                fi

              done

              find $BIN_FOLDER -type f -exec chmod a+x '{}' \;

              ls -lahR $BIN_FOLDER

          - name: Run Unit Tests

            uses: docker://su2code/test-su2:20191031

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              entrypoint: install/bin/${{matrix.testdriver}}

          - name: Post Cleanup 

            uses: docker://ghcr.io/su2code/su2/test-su2:220614-1237

            with:

              entrypoint: /bin/rm

              args: -rf install install_bin.tgz src ccache ${{ matrix.config_set }}

TestCases/TestCase.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -27,6 +27,7 @@ @@
     from __future__ import print_function, division, absolute_import
     import time, os, subprocess, datetime, sys
     import difflib
+    import platform
     def print_vals(vals, name="Values"):
@@ Expand Down Expand Up / @@ -63,6 +64,9 @@ def __init__(self,tag_in): @@
             self.test_iter = 1
             self.ntest_vals = 4
             self.test_vals = []
+            self.test_vals_aarch64 = []
+            self.cpu_arch = platform.processor()
+            self.enabled_on_cpu_arch = ["x86_64", "aarch64"]
             # These can be optionally varied
             self.su2_exec    = "SU2_CFD"
@@ Expand All / @@ -71,10 +75,14 @@ def __init__(self,tag_in): @@
             # Options for file-comparison tests
             self.reference_file = "of_grad.dat.ref"
+            self.reference_file_aarch64 = ""
             self.test_file      = "of_grad.dat"
         def run_test(self):
+            if not self.is_enabled():
+                return True
             print('==================== Start Test: %s ===================='%self.tag)
             passed       = True
             exceed_tol   = False
@@ Expand Down Expand Up / @@ -109,6 +117,8 @@ def run_test(self): @@
                                                self.cfg_file,
                                                logfilename)
+            self.adjust_test_data()
             # Run SU2
             workdir = os.getcwd()
             os.chdir(self.cfg_dir)
@@ Expand Down Expand Up / @@ -208,6 +218,8 @@ def run_test(self): @@
             if iter_missing:
                 print('ERROR: The iteration number %d could not be found.'%self.test_iter)
+            print('CPU architecture=%s' % self.cpu_arch)
             if len(self.test_vals) != 0:
                 print('test_iter=%d' % self.test_iter)
@@ Expand All / @@ -225,13 +237,19 @@ def run_test(self): @@
             return passed
         def run_filediff(self):
+            if not self.is_enabled():
+                return True
             print('==================== Start Test: %s ===================='%self.tag)
             passed       = True
             timed_out    = False
             # Adjust the number of iterations in the config file
             self.adjust_iter()
+            self.adjust_test_data()
             # if root, add flag to mpirun
             if os.geteuid()==0:
                 if self.su2_exec.startswith('mpirun'):
@@ Expand Down Expand Up / @@ -305,6 +323,7 @@ def run_filediff(self): @@
             else:
                 passed = False
+            print('CPU architecture=%s'%self.cpu_arch)
             print('test duration: %.2f min'%(running_time/60.0))
             print('==================== End Test: %s ====================\n'%self.tag)
@@ Expand All / @@ -314,6 +333,9 @@ def run_filediff(self): @@
         def run_opt(self):
+            if not self.is_enabled():
+                return True
             print('==================== Start Test: %s ===================='%self.tag)
             passed       = True
             exceed_tol   = False
@@ Expand All / @@ -324,6 +346,8 @@ def run_opt(self): @@
             # Adjust the number of iterations in the config file
             self.adjust_opt_iter()
+            self.adjust_test_data()
             # Assemble the shell command to run SU2
             logfilename = '%s.log' % os.path.splitext(self.cfg_file)[0]
             command = "%s %s > %s 2>&1" % (self.su2_exec, self.cfg_file, logfilename)
@@ Expand Down Expand Up / @@ -434,6 +458,9 @@ def run_opt(self): @@
         def run_geo(self):
+            if not self.is_enabled():
+                return True
             print('==================== Start Test: %s ===================='%self.tag)
             passed       = True
             exceed_tol   = False
@@ Expand All / @@ -446,6 +473,8 @@ def run_geo(self): @@
             found_twist  = False
             found_chord  = False
+            self.adjust_test_data()
             # if root, add flag to mpirun
             if os.geteuid()==0:
                 if self.su2_exec.startswith('mpirun'):
@@ Expand Down Expand Up / @@ -562,14 +591,19 @@ def run_geo(self): @@
             return passed
         def run_def(self):
+            if not self.is_enabled():
+                return True
             print('==================== Start Test: %s ===================='%self.tag)
             passed       = True
             exceed_tol   = False
             timed_out    = False
             iter_missing = True
             start_solver = True
+            self.adjust_test_data()
             # if root, add flag to mpirun
             if os.geteuid()==0:
                 if self.su2_exec.startswith('mpirun'):
@@ Expand Down Expand Up / @@ -760,3 +794,20 @@ def disable_restart(self): @@
             os.chdir(workdir)
             return
+        def is_enabled(self):
+            is_enabled = self.cpu_arch in self.enabled_on_cpu_arch
+            if not is_enabled:
+                print('Ignoring test "%s" because it is not enabled for the current CPU architecture: %s' % (self.tag, self.cpu_arch))
+            return is_enabled
+        def adjust_test_data(self):
+            if self.cpu_arch == 'aarch64':
+                if len(self.test_vals_aarch64) != 0:
+                    self.test_vals = self.test_vals_aarch64
+                if len(self.reference_file_aarch64) != 0:
+                    self.reference_file = self.reference_file_aarch64

TestCases/cont_adj_euler/naca0012/of_grad_cd_disc_aarch64.dat.ref

-Original file line number
+Diff line change
@@ -0,0 +1,39 @@
+    VARIABLES="VARIABLE"      , "GRADIENT"      , "FINDIFF_STEP"
+, -2656.96        , 0.001
+, -12995.9        , 0.001
+, -21781.3        , 0.001
+, -27675.3        , 0.001
+, -30437.9        , 0.001
+, -30443.9        , 0.001
+, -28343.1        , 0.001
+, -24794.2        , 0.001
+, -20329.4        , 0.001
+, -15376.6        , 0.001
+, -10418.0        , 0.001
+, -6201.67        , 0.001
+, -3878.08        , 0.001
+, -4910.93        , 0.001
+, -10573.5        , 0.001
+, -20610.7        , 0.001
+, -30110.4        , 0.001
+, -26294.5        , 0.001
+, -62393.8        , 0.001
+, -2797.26        , 0.001
+, -832.067        , 0.001
+, -668.081        , 0.001
+, -1773.25        , 0.001
+, -5718.86        , 0.001
+, -13717.8        , 0.001
+, -25699.0        , 0.001
+, -40061.9        , 0.001
+, -53729.8        , 0.001
+, -62415.6        , 0.001
+, -61293.4        , 0.001
+, -46505.9        , 0.001
+, -17841.7        , 0.001
+, 18183.8         , 0.001
+, 48447.5         , 0.001
+, 62322.0         , 0.001
+, 64139.5         , 0.001
+, 54568.2         , 0.001
+, 64856.8         , 0.001

TestCases/cont_adj_euler/naca0012/of_grad_directdiff_aarch64.dat.ref

-Original file line number
+Diff line change
@@ -0,0 +1,4 @@
+    VARIABLES="VARIABLE"      , "DRAG"          , "EFFICIENCY"    , "FORCE_X"       , "FORCE_Y"       , "FORCE_Z"       , "LIFT"          , "MOMENT_X"      , "MOMENT_Y"      , "MOMENT_Z"      , "SIDEFORCE"
+, 0.2253591473    , -105.6097088    , 0.2588459007    , -1.5322178      , 0.0             , -1.537499867    , 0.0             , 0.0             , 1.202899757     , 0.0
+, 0.3835809166    , -173.3502205    , 0.4363886002    , -2.415957492    , 0.0             , -2.424902327    , 0.0             , 0.0             , 1.053347497     , 0.0
+, 0.5151776249    , -228.9760041    , 0.5835870252    , -3.129538494    , 0.0             , -3.141524632    , 0.0             , 0.0             , 0.6540715539    , 0.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build SU2 and run regressions on self hosted aarch64 runner scheduled #1619

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!