New Discrete Adjoint Multizone Capabilities (e.g. Adjoint CHT) #774

oleburghardt · 2019-08-26T13:22:29Z

Proposed Changes

This PR introduces the discrete adjoint counterpart for the multizone driver. It also comes with some heat solver-related additions for the new input/output structure as CHT (conjugate heat transfer) cases are the primary application so far.

The new driver adapts many of the well-known routines for registering variables, recording, initializing and extracting adjoints. The main difference is the handling of multiple zones.

The adjoint (fixed point-) iteration approach in CDiscAdjMultizoneDriver is to initialize them per-zone in an outer loop and to extract adjoints with respect to dependent variables from all zones, thereby inherently capturing the cross dependencies.
Multiple successive evaluations of adjoints with respect to the same zone (equivalent to inner iterations in the multizone driver) are possible.

In order not to have to re-record a tape in the (outer) zone-loop as it can be quite costly, this branch includes some new functionalities in the underlying AD-structure which make it possible to record a "full" tape that holds all derivative information and that can be accessed flexibly for different input and output variables across zones.
This also overcomes the restriction that adjoints have to be evaluated in the same order as their variables had been registered.

This is the first version of the driver and my intention is to continue contributing to it in terms of both performance and coverage of different kinds of physical couplings in SU2.

PR Checklist

A regression test for a heated cylinder array with conjugate heat transfer is added to TestCases/coupled_cht/disc_adj_incomp_2d.

The very same case is used in a tutorial showcasing the new driver and CHT sensitivity analysis, see Heated cylinders test case.

I'll check carefully for compiler warnings during test phase.

I am submitting my contribution to the develop branch.
My contribution generates no new compiler warnings (try with the '-Wall -Wextra -Wno-unused-parameter -Wno-empty-body' compiler flags).
My contribution is commented and consistent with SU2 style.
I have added a test case that demonstrates my contribution, if necessary.

…ional graph for gradient evaluations.

…rnally stored indices.

…ed indices.

… solver level.

…multizone.cpp).

… coordinates within the CPoint class (one needs derivatives with respect to these e.g. in moving mesh mesh problems or in FSI).

…e adjoints.

…sing it yet.

…e discrete adjoint functionalities.

…to sc_develop

oleburghardt · 2019-10-08T09:52:53Z

Ok, I am ~halfway through and I have mostly nitpicking stuff. In general it is really tidy and I am looking forward to use everything you added. Rest will follow tomorrow.

Thanks a lot Tobias! I think I made all the changes that you suggested, it's just the ouput that you and Pedro do not like. I'll think about what can be done .. (though part of it might be due to the fact that the switch-thing is more apparent now, same thing can be found in the solvers as well).

I already did the tutorial you uploaded but I'll redo everything as I couldn't reproduce the nice gradients you show in the tutorial. I'll report to you... maybe I somewhere wrong

Could be please try to run SU2_DOT_AD in serial? (If this doesn't help and if you've got the time, also SU2_CFD_AD?)
That really must work .. it's quite important to me as I invested so much time so make it available for all. Could you send me your adjoint solution files?

TobiKattmann · 2019-10-09T08:20:15Z

Could be please try to run SU2_DOT_AD in serial? (If this doesn't help and if you've got the time, also SU2_CFD_AD?)
That really must work .. it's quite important to me as I invested so much time so make it available for all. Could you send me your adjoint solution files?

@oleburghardt Yep, Single core for both did the trick and I now get the same gradient results. I will double check with 28 cores to see if it still produces differing results.

…rom config.

oleburghardt · 2019-10-09T11:43:01Z

@oleburghardt Yep, Single core for both did the trick and I now get the same gradient results. I will double check with 28 cores to see if it still produces differing results.

Do they really look different or do we rather see these kind of noisy spots where physical boundaries intersect with MPI boundaries? (In that case, it's likely that it's not an adjoint problem but that the driver computes accurate sensitivities based on a somewhere flawed primal code, i.e. the sensitivities are just unveiling a primal problem.)

oleburghardt · 2019-10-09T14:42:15Z

SU2_CFD/src/interfaces/cht/CConjugateHeatInterface.cpp

+  Donor_Variable[0] = Twall*donor_config->GetTemperature_Ref();
+  Donor_Variable[1] = heat_flux_density*donor_config->GetHeat_Flux_Ref();


@economon Could you look over this and changes in the CHeatSolverFVM::Heat_Fluxes routine to check whether these usages of GetHeat_Flux_Ref meet your intention behind it?

See previous comment above..

oleburghardt · 2019-10-09T14:44:44Z

SU2_CFD/src/interfaces/cht/CConjugateHeatInterface.cpp

+    su2double Gamma         = donor_config->GetGamma();
+    su2double Gas_Constant  = donor_config->GetGas_ConstantND();
+    su2double Cp            = (Gamma / (Gamma - 1.0)) * Gas_Constant;
+
+    Prandtl_Lam             = donor_config->GetPrandtl_Lam();
+    laminar_viscosity       = donor_solution->node[Point_Donor]->GetLaminarViscosity(); // TDE check for consistency
+    Cp                      = (Gamma / (Gamma - 1.0)) * Gas_Constant;


@economon This is now completely analogue to the code in CNSSolver::Friction_Forces. Do you think it's okay that way?

Perhaps it is best to have the flow solvers compute the information that you need here just once and store it (maybe we can use what is computed in Friction_Forces() or something similar), and only have the interface class access it from the corresponding solver (compressible vs incompressible). It could be dangerous to have two implementations of the same thing - if one changes and the other doesn't, the functionality breaks. It would be a lot easier to maintain, and simpler, to define it just once. Also, these quantities can be different between compressible and incompressible.

Yes it's just about the compressible case.
For the incompressible one, I get a non-dimensionalized thermal conductivity from donor_solution->node[iPoint]->GetThermalConductivity() to then compute everything and finally use donor_config->GetHeat_Flux_Ref() to dimensionalize.
I am just not so much familiar with the compressible solver, but if it's sure that I could e.g. get the thermal conductivity always from FluidModel I could replace the computation (which is unsafe, as you pointed out) there by such a solver call.

pcarruscag · 2019-10-10T10:49:53Z

@oleburghardt, looks like this is taking a turn to also address primal CHT topics, but all comments on the main dish have been addressed. Can we merge this to shorten the dependency chain between PR's while you continue working on ironing out minor kinks?

oleburghardt · 2019-10-10T11:41:18Z

@oleburghardt, looks like this is taking a turn to also address primal CHT topics, but all comments on the main dish have been addressed. Can we merge this to shorten the dependency chain between PR's while you continue working on ironing out minor kinks?

True, but that was about it. They just have fit in so well. As for the main part, we just have to keep in mind the CDiscAdjMultizoneDriver::SetObjFunction that two of you didn't like. But as I said, it has much to do with other changes that should be done before.

pcarruscag · 2019-10-10T12:27:02Z

Great! Let's merge then, I'll merge this with #753 today, please have a look at the changes affecting the new fields you added to CVariable on my next commit.

pcarruscag · 2019-10-10T12:41:01Z

SU2_CFD/src/interfaces/cht/CConjugateHeatInterface.cpp

  }
  else if (incompressible_flow) {

-    iPoint = donor_geometry->vertex[Marker_Donor][Vertex_Donor]->GetNode();


@oleburghardt just in the process of merging I think this line was removed by mistake? iPoint is used right after

Yes, it's a mistake. If you are in the merge window right now, can you revert that change/add that line in your branch? That should be the quickest solution to fix it in develop. Thanks in advance..

pcarruscag · 2019-10-10T18:06:10Z

SU2_CFD/src/iteration_structure.cpp


  /*--- Compute coupling between flow and turbulent equations ---*/
-
+  solver[iZone][iInst][MESH_0][FLOW_SOL]->Preprocessing(geometry[iZone][iInst][MESH_0], solver[iZone][iInst][MESH_0], config[iZone], MESH_0, NO_RK_ITER, RUNTIME_FLOW_SYS, true);


@oleburghardt this call is duplicated, can one of them be removed?

I'll check and eventually delete it in #798 Thanks!

oleburghardt and others added 30 commits November 14, 2018 10:57

Adding routines to provide direct access to the indices of a computat…

b44ab34

…ional graph for gradient evaluations.

Adapted doxygen information on AD extensions.

fce0218

Added variable data structures to set and extract gradients with exte…

6ce7ef8

…rnally stored indices.

First changes on discrete adjoint solver level to use explicitly stor…

4e158db

…ed indices.

Added missing (important) lines for first changes on discrete adjoint…

c357bdc

… solver level.

Simple data structures to update adjoint solutions for multiple zones.

c1df4d2

Update travis for developping.

458ac29

Discrete adjoint multizone driver file (counterpart of driver_direct_…

72f36ff

…multizone.cpp).

Updating travis for development.

349b25e

Adding the possibility to store (adjoint vector) indices of the point…

344f936

… coordinates within the CPoint class (one needs derivatives with respect to these e.g. in moving mesh mesh problems or in FSI).

Merge branch 'develop' into feature_ad_extensions

e077902

Missing initialisation.

7b20aab

Allocation fix.

dda2de5

Merge branch 'develop' into feature_ad_extensions

2168034

Improved naming of new AD routines.

8609b53

Merge branch 'develop' into feature_discadj_multiphysics

f1e6106

Merge branch 'develop' into feature_discadj_multiphysics

a8808a0

Merge branch 'feature_ad_extensions' into feature_discadj_multiphysics

9c4dde2

Some important corrections for the new AD functions.

8d03108

Some overall important corrections and bugfixes for multizone discret…

c1eceba

…e adjoints.

Code for saving coordinate indices within point variables, too. Not u…

edd432b

…sing it yet.

Multizone Discrete Adjoint Driver code.

f0a66cd

Replaced ZoneSpecific-booleans by Multizone booleans for the multizon…

ab45a8e

…e discrete adjoint functionalities.

Two further small corrections.

0372841

Initialising the multizone discrete adjoint driver.

8a4f8cc

Merge feature_discadj_multiphysics

ff65507

Bugfix.

9d12d85

Merge remote-tracking branch 'origin/feature_discadj_multiphysics' in…

f76d3ba

…to sc_develop

Added output routine for adjoint multiphysics

fa13349

Fixed output pointer

9da3efe

oleburghardt added 3 commits October 8, 2019 11:43

Doxygen and grammar fixes.

a74efbd

Adding override identifiers, some other fixes.

8833f43

Nitpicky part.

7f7c3c4

oleburghardt added 2 commits October 9, 2019 13:35

Reference value (temperature and heat fluxes) cleanup for heat solver.

9adcc03

Conjugate heat interface cleanup (part 1/2), using reference values f…

cda8a25

…rom config.

Conjugate heat interface cleanup (part 2/2).

3ec8fd0

oleburghardt commented Oct 9, 2019

View reviewed changes

pcarruscag merged commit b5ba8e6 into develop Oct 10, 2019

pcarruscag deleted the sc_develop branch October 10, 2019 12:27

pr-triage bot added PR: merged and removed PR: unreviewed labels Oct 10, 2019

pcarruscag restored the sc_develop branch October 10, 2019 12:27

pcarruscag reviewed Oct 10, 2019

View reviewed changes

pr-triage bot added PR: unreviewed and removed PR: merged labels Oct 10, 2019

talbring mentioned this pull request Oct 10, 2019

Various fixes #798

Merged

3 tasks

pcarruscag reviewed Oct 10, 2019

View reviewed changes

pcarruscag mentioned this pull request Oct 12, 2019

airfoil_fsi_rbf failing travis #801

Closed

rsanfer mentioned this pull request Oct 15, 2019

Re-enable Discrete Adjoint FSI Test Cases #803

Merged

4 tasks

talbring added the changelog:feature label Nov 6, 2019

talbring changed the title ~~Discrete adjoint multizone driver~~ New Discrete Adjoint Multizone Capabilities (e.g. Adjoint CHT) Nov 8, 2019

pr-triage bot added PR: merged and removed PR: unreviewed labels Nov 8, 2019

		Donor_Variable[0] = Twall*donor_config->GetTemperature_Ref();
		Donor_Variable[1] = heat_flux_density*donor_config->GetHeat_Flux_Ref();


		/--- Compute coupling between flow and turbulent equations ---/

		solver[iZone][iInst][MESH_0][FLOW_SOL]->Preprocessing(geometry[iZone][iInst][MESH_0], solver[iZone][iInst][MESH_0], config[iZone], MESH_0, NO_RK_ITER, RUNTIME_FLOW_SYS, true);

New Discrete Adjoint Multizone Capabilities (e.g. Adjoint CHT) #774

New Discrete Adjoint Multizone Capabilities (e.g. Adjoint CHT) #774

Uh oh!

Conversation

oleburghardt commented Aug 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

PR Checklist

Uh oh!

oleburghardt commented Oct 8, 2019

Uh oh!

TobiKattmann commented Oct 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oleburghardt commented Oct 9, 2019

Uh oh!

oleburghardt Oct 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

economon Oct 9, 2019

Choose a reason for hiding this comment

Uh oh!

oleburghardt Oct 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

economon Oct 9, 2019

Choose a reason for hiding this comment

Uh oh!

oleburghardt Oct 10, 2019

Choose a reason for hiding this comment

Uh oh!

pcarruscag commented Oct 10, 2019

Uh oh!

oleburghardt commented Oct 10, 2019

Uh oh!

pcarruscag commented Oct 10, 2019

Uh oh!

pcarruscag Oct 10, 2019

Choose a reason for hiding this comment

Uh oh!

oleburghardt Oct 10, 2019

Choose a reason for hiding this comment

Uh oh!

pcarruscag Oct 10, 2019

Choose a reason for hiding this comment

Uh oh!

oleburghardt Oct 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

oleburghardt commented Aug 26, 2019 •

edited

Loading

TobiKattmann commented Oct 9, 2019 •

edited

Loading

oleburghardt Oct 9, 2019 •

edited

Loading

oleburghardt Oct 9, 2019 •

edited

Loading

oleburghardt Oct 11, 2019 •

edited

Loading