Skip to content

Hybrid parallel boundary conditions for turbulence and compressible flow solvers #975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pcarruscag merged 42 commits into from
May 28, 2020
Merged

Hybrid parallel boundary conditions for turbulence and compressible flow solvers #975

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
1e7a5ea
Merge branch 'linsol_output' into hybrid_parallel_bcs
pcarruscag May 9, 2020
520ae25
thread-safe BC's for turbulent solvers
pcarruscag May 9, 2020
6a1ab58
thread-safe compressible solver BC's
pcarruscag May 9, 2020
496c5f5
avoid using the small member arrays of CSolver also in its methods
pcarruscag May 10, 2020
e70d5fb
pick one boundary numerics object per thread in CIntegration (to avoi…
pcarruscag May 10, 2020
4a21bcf
fix for turbo boundaries
pcarruscag May 10, 2020
266ee30
un-fix CSolver::SetRotatingFrame_GCL as one restarted case was diverging
pcarruscag May 11, 2020
c46166e
Smatrix and Cvector no longer needed
pcarruscag May 11, 2020
6ecc03c
CNSSolver::BC_HeatFlux_Wall
pcarruscag May 11, 2020
3e8ba1b
cleanup isothermal boundary condition and remove duplication between …
pcarruscag May 11, 2020
a95105a
simple fix
pcarruscag May 11, 2020
885ea02
cleanup wall functions
pcarruscag May 11, 2020
33f99e3
fix for sym plane BC
pcarruscag May 11, 2020
f6935a4
delete a few useless lines from CConfig
pcarruscag May 12, 2020
a59c7ff
Merge branch 'develop' into hybrid_parallel_bcs
pcarruscag May 13, 2020
2b5f72f
Merge branch 'develop' into hybrid_parallel_bcs
pcarruscag May 13, 2020
4f987cd
small tweak of CSysMatrix preconditioner-allocation logic
pcarruscag May 15, 2020
67a1051
Merge branch 'develop' into hybrid_parallel_bcs
pcarruscag May 15, 2020
faf1382
add hybrid test configuration
pcarruscag May 15, 2020
02be287
add regression somewhere else
pcarruscag May 15, 2020
ae54af3
try again
pcarruscag May 15, 2020
a2b1317
try again
pcarruscag May 15, 2020
f6a3571
logic fix for SU2 kind
pcarruscag May 15, 2020
5596a5d
bloody hell
pcarruscag May 15, 2020
2e5c477
wrong script name
pcarruscag May 15, 2020
ac3981b
set reference residuals for hybrid tests
pcarruscag May 15, 2020
76bd9f6
missing change
pcarruscag May 15, 2020
32e7391
few testcase tweaks
pcarruscag May 15, 2020
e483760
Merge branch 'linsol_output' into hybrid_parallel_bcs
pcarruscag May 18, 2020
46b87e1
cleanup regression script
pcarruscag May 18, 2020
ca3e637
Simplified regression.yml script
talbring May 22, 2020
30bbd90
Merge branch 'develop' into hybrid_parallel_bcs
pcarruscag May 22, 2020
beb3da9
Small fix
talbring May 22, 2020
f88024c
Merge branch 'hybrid_parallel_bcs' of https://github.com/su2code/SU2 …
talbring May 22, 2020
98c9a0d
Merge branch 'feature_mixed_precision' into hybrid_parallel_bcs
pcarruscag May 25, 2020
53bd556
put mixed precision under regression
pcarruscag May 25, 2020
c8cbf90
fix bug, update hybrid regressions
pcarruscag May 25, 2020
d5f033a
Merge remote-tracking branch 'upstream/develop' into hybrid_parallel_bcs
pcarruscag May 27, 2020
6413241
fix pywrapper
pcarruscag May 27, 2020
0e86465
fix unit tests
pcarruscag May 27, 2020
c05943b
fix pywrapper, again
pcarruscag May 27, 2020
ef9eac3
give tests more time
pcarruscag May 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 13 additions & 20 deletions .github/workflows/regression.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
strategy:
fail-fast: false
matrix:
config_set: [BaseMPI, ReverseMPI, ForwardMPI, BaseNoMPI, ReverseNoMPI, ForwardNoMPI]
config_set: [BaseMPI, ReverseMPI, ForwardMPI, BaseNoMPI, ReverseNoMPI, ForwardNoMPI, BaseOMP]
include:
- config_set: BaseMPI
flags: '-Denable-pywrapper=true -Denable-tests=true --werror'
Expand All @@ -30,6 +30,8 @@ jobs:
flags: '-Denable-autodiff=true -Denable-normal=false -Dwith-mpi=disabled -Denable-pywrapper=true -Denable-tests=true --werror'
- config_set: ForwardNoMPI
flags: '-Denable-directdiff=true -Denable-normal=false -Dwith-mpi=disabled -Denable-tests=true --werror'
- config_set: BaseOMP
flags: '-Dwith-omp=true -Denable-mixedprec=true -Denable-tecio=false --werror'
runs-on: ubuntu-latest
steps:
- name: Cache Object Files
Expand All @@ -54,7 +56,7 @@ jobs:
strategy:
fail-fast: false
matrix:
testscript: ['tutorials.py', 'parallel_regression.py', 'parallel_regression_AD.py', 'serial_regression.py', 'serial_regression_AD.py']
testscript: ['tutorials.py', 'parallel_regression.py', 'parallel_regression_AD.py', 'serial_regression.py', 'serial_regression_AD.py', 'hybrid_regression.py']
include:
- testscript: 'tutorials.py'
tag: MPI
Expand All @@ -66,31 +68,22 @@ jobs:
tag: NoMPI
- testscript: 'serial_regression_AD.py'
tag: NoMPI
- testscript: 'hybrid_regression.py'
tag: OMP
steps:
- name: Download Base
uses: actions/download-artifact@v1
with:
name: ${{format('Base{0}', matrix.tag)}}
- name: Download Reverse
uses: actions/download-artifact@v1
with:
name: ${{format('Reverse{0}', matrix.tag)}}
- name: Download Forward
uses: actions/download-artifact@v1
with:
name: ${{format('Forward{0}', matrix.tag)}}
- name: Download All artifact
uses: actions/download-artifact@v2
- name: Move Binaries
run: |
mkdir -p install/bin
cp -r ${{format('Base{0}', matrix.tag)}}/. install/bin/
cp -r ${{format('Reverse{0}', matrix.tag)}}/. install/bin/
cp -r ${{format('Forward{0}', matrix.tag)}}/. install/bin/
chmod a+x install/bin/*
if [ -d "${{format('Base{0}', matrix.tag)}}" ]; then cp -r ${{format('Base{0}', matrix.tag)}}/. install/bin/; fi
if [ -d "${{format('Reverse{0}', matrix.tag)}}" ]; then cp -r ${{format('Reverse{0}', matrix.tag)}}/. install/bin/; fi
if [ -d "${{format('Forward{0}', matrix.tag)}}" ]; then cp -r ${{format('Forward{0}', matrix.tag)}}/. install/bin/; fi
chmod a+x install/bin/*
- name: Run Tests in Container
uses: docker://su2code/test-su2:20200303
with:
args: -b ${{github.ref}} -t develop -c develop -s ${{matrix.testscript}}

unit_tests:
runs-on: ubuntu-latest
name: Unit Tests
Expand Down Expand Up @@ -125,7 +118,7 @@ jobs:
cp -r ${{format('Base{0}', matrix.tag)}}/. install/bin/
cp -r ${{format('Reverse{0}', matrix.tag)}}/. install/bin/
cp -r ${{format('Forward{0}', matrix.tag)}}/. install/bin/
chmod a+x install/bin/*
chmod a+x install/bin/*
- name: Run Unit Tests
uses: docker://su2code/test-su2:20191031
with:
Expand Down
8 changes: 6 additions & 2 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,15 @@ env:
- CONFIGURE_COMMAND="./meson.py build --prefix=$TRAVIS_BUILD_DIR -Denable-pywrapper=true"
TEST_SCRIPT=parallel_regression.py

# Hybrid-parallel build and test
- CONFIGURE_COMMAND="./meson.py build --prefix=$TRAVIS_BUILD_DIR -Dwith-omp=true -Denable-mixedprec=true -Denable-tecio=false"
TEST_SCRIPT=hybrid_regression.py

# Serial build and test for AD
- CONFIGURE_COMMAND="./meson.py build --prefix=$TRAVIS_BUILD_DIR -Denable-pywrapper=true -Dwith-mpi=disabled -Denable-autodiff=true -Denable-directdiff=true"
TEST_SCRIPT=serial_regression_AD.py

# Parallel build and test for AD:
# Parallel build and test for AD
- CONFIGURE_COMMAND="./meson.py build --prefix=$TRAVIS_BUILD_DIR -Denable-pywrapper=true -Denable-autodiff=true"
TEST_SCRIPT=parallel_regression_AD.py

Expand All @@ -51,7 +55,7 @@ before_install:
- sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
# Install the necessary packages using apt-get with sudo
- sudo apt-get update -qq
- sudo apt-get install -qq build-essential python3-numpy python3-scipy libopenmpi-dev openmpi-bin swig python3-mpi4py
- sudo apt-get install -qq build-essential python3-numpy python3-scipy libopenmpi-dev openmpi-bin swig python3-mpi4py
# to avoid interference with MPI
- test -n $CC && unset CC
- test -n $CXX && unset CXX
Expand Down
29 changes: 4 additions & 25 deletions Common/include/CConfig.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@
#include <assert.h>

#include "./option_structure.hpp"
#include "./datatype_structure.hpp"

#ifdef HAVE_CGNS
#include "cgnslib.h"
Expand Down Expand Up @@ -1282,7 +1281,7 @@ class CConfig {
* \brief Constructor of the class which takes an istream buffer containing the config options.
*/
CConfig(istream &case_buffer, unsigned short val_software, bool verb_high);

/*!
* \brief Constructor of the class which reads the input file and uses default options from another config.
*/
Expand All @@ -1302,7 +1301,7 @@ class CConfig {
* \brief Destructor of the class.
*/
~CConfig(void);

/*!
* \brief Initialize common fields of the config structure.
*/
Expand Down Expand Up @@ -8009,8 +8008,8 @@ class CConfig {
/*!
* \brief Set the config file parsing.
*/
void SetConfig_Parsing(istream &config_buffer);
void SetConfig_Parsing(istream &config_buffer);

/*!
* \brief Set the config file parsing.
*/
Expand Down Expand Up @@ -8340,26 +8339,6 @@ class CConfig {
*/
unsigned long GetNonphysical_Reconstr(void) const { return Nonphys_Reconstr; }

/*!
* \brief Given arrays x[1..n] and y[1..n] containing a tabulated function, i.e., yi = f(xi), with
x1 < x2 < . . . < xN , and given values yp1 and ypn for the first derivative of the interpolating
function at points 1 and n, respectively, this routine returns an array y2[1..n] that contains
the second derivatives of the interpolating function at the tabulated points xi. If yp1 and/or
ypn are equal to 1 × 1030 or larger, the routine is signaled to set the corresponding boundary
condition for a natural spline, with zero second derivative on that boundary.
Numerical Recipes: The Art of Scientific Computing, Third Edition in C++.
*/
void SetSpline(vector<su2double> &x, vector<su2double> &y, unsigned long n, su2double yp1, su2double ypn, vector<su2double> &y2);

/*!
* \brief Given the arrays xa[1..n] and ya[1..n], which tabulate a function (with the xai’s in order),
and given the array y2a[1..n], which is the output from spline above, and given a value of
x, this routine returns a cubic-spline interpolated value y.
Numerical Recipes: The Art of Scientific Computing, Third Edition in C++.
* \return The interpolated value of for x.
*/
su2double GetSpline(vector<su2double> &xa, vector<su2double> &ya, vector<su2double> &y2a, unsigned long n, su2double x);
Comment on lines -8351 to -8361
Copy link
Member Author

@pcarruscag pcarruscag May 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code was unused and duplicated in CGeometry.


/*!
* \brief Start the timer for profiling subroutines.
* \param[in] val_start_time - the value of the start time.
Expand Down
0 Common/include/ad_structure.hpp → Common/include/basic_types/ad_structure.hpp
View file
Open in desktop
File renamed without changes.
0 Common/include/datatype_structure.hpp → ...nclude/basic_types/datatype_structure.hpp
View file
Open in desktop
File renamed without changes.
5 changes: 2 additions & 3 deletions Common/include/blas_structure.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
*
* SU2 Project Website: https://su2code.github.io
*
* The SU2 Project is maintained by the SU2 Foundation
* The SU2 Project is maintained by the SU2 Foundation
* (http://su2foundation.org)
*
* Copyright 2012-2020, SU2 Contributors (cf. AUTHORS.md)
Expand All @@ -29,7 +29,6 @@

#pragma once

#include "datatype_structure.hpp"
#include "CConfig.hpp"

/* LIBXSMM include files, if supported. */
Expand All @@ -39,7 +38,7 @@

/*!
* \class CBlasStructure
* \brief Class, which serves as an interface to the BLAS functionalities needed.
* \brief Class, which serves as an interface to the BLAS functionalities needed.
* \author: E. van der Weide
* \version 7.0.4 "Blackbird"
*/
Expand Down
2 changes: 1 addition & 1 deletion Common/include/fem/geometry_structure_fem_part.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

#pragma once

#include "../datatype_structure.hpp"
#include "../basic_types/datatype_structure.hpp"

#include <vector>
#include <algorithm>
Expand Down
2 changes: 1 addition & 1 deletion Common/include/interface_interpolation/CInterpolator.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
*/
#pragma once

#include "../../include/datatype_structure.hpp"
#include "../../include/basic_types/datatype_structure.hpp"
#include "../../include/toolboxes/C2DContainer.hpp"
#include <vector>

Expand Down
2 changes: 1 addition & 1 deletion Common/include/linear_algebra/CSysMatrix.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ class CSysMatrix {
void Initialize(unsigned long npoint, unsigned long npointdomain,
unsigned short nvar, unsigned short neqn,
bool EdgeConnect, CGeometry *geometry,
CConfig *config, bool needTranspPtr = false);
const CConfig *config, bool needTranspPtr = false);

/*!
* \brief Sets to zero all the entries of the sparse matrix.
Expand Down
2 changes: 1 addition & 1 deletion Common/include/mpi_structure.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
#include <map>
#endif

#include "./datatype_structure.hpp"
#include "basic_types/datatype_structure.hpp"
#include <stdlib.h>
#ifndef _MSC_VER
#include <unistd.h>
Expand Down
1 change: 0 additions & 1 deletion Common/include/toolboxes/C1DInterpolation.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
#include <cmath>
#include <vector>
#include<fstream>
#include "../datatype_structure.hpp"
#include "../option_structure.hpp"

using namespace std;
Expand Down
2 changes: 1 addition & 1 deletion Common/include/toolboxes/C2DContainer.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#pragma once

#include "allocation_toolbox.hpp"
#include "../datatype_structure.hpp"
#include "../basic_types/datatype_structure.hpp"

#include <utility>
#include <type_traits>
Expand Down
2 changes: 1 addition & 1 deletion Common/include/toolboxes/printing_toolbox.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
#include <sstream>
#include <cmath>
#include <algorithm>
#include "../datatype_structure.hpp"
#include "../basic_types/datatype_structure.hpp"

namespace PrintingToolbox {

Expand Down
2 changes: 1 addition & 1 deletion Common/lib/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ lib_sources = \
../src/grid_adaptation_structure.cpp \
../src/grid_movement_structure.cpp \
../src/mpi_structure.cpp \
../src/ad_structure.cpp \
../src/basic_types/ad_structure.cpp \
../src/fem/fem_gauss_jacobi_quadrature.cpp \
../src/geometry/CGeometry.cpp \
../src/geometry/CPhysicalGeometry.cpp \
Expand Down
Loading