-
Notifications
You must be signed in to change notification settings - Fork 918
Multiphysics interpolation new features and improvements #914
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| AccessorImpl& operator= (AccessorImpl&& other) noexcept \ | ||
| { \ | ||
| if(m_data!=nullptr) free(m_data); \ | ||
| MemoryAllocation::aligned_free<Scalar_t>(m_data); \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also fixes a potential memory leak for mac and windows.
…titioning of markers
…erpolation_improvements
…erpolation_improvements
pcarruscag
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR turned into a bit of a bug fixing campaign, my solution for one of them is less straightforward than I would like. It would be great if someone who is very familiar with how markers are partitioned could have a look at this.
The problem
The displacement boundary conditions of CFEASolver (also used for mesh deformation in FSI problems) would sometimes break when running in parallel.
For example this would happen:

files.zip
For 3D FSI the mesh deformation is almost guaranteed to break due to every boundary being of this type, and we would get this type of artifact:

The cause (?)
For displacement boundary conditions we need to impose the solution of the linear system at some nodes, even on partitions where these nodes are halos otherwise matrix-vector products will be incorrect on that partition.
It seems that some partitions do not seem to receive enough of some markers to cover those halo points.
(A) solution
To get correct matrix vector products it is sufficient to communicate the linear system solution before solving the systems. This gives the halos the correct values, and since the matrix vector product (and all linear preconditioners) ignore the "halo rows" of the matrix, those values stay correct.
However, to maintain the symmetry of the stiffness matrix, we eliminate both the row and column associated with these nodes, while the row only needs to be eliminated on the partition for which the node is a domain point, the column needs to be eliminated on all partitions (thus updating the right hand side on any row with a non-zero block on the target column), this process was failing due to the "incomplete marker" problem.
Again, this is only a problem if we want to maintain strict symmetry which is required to use symmetric factorizations, but seemingly irrelevant for all our normal algorithms.
My current solution is to detect these very few problem nodes (10 or so on a case with 50k+ boundary nodes) and do extra row-column eliminations just before solving the linear system...
Ideally we would just have "enough marker" for these nodes to be covered during the normal boundary condition handling, but at the moment I have no idea how to go about fixing the root cause of the problem.
This does not seem to affect the FVM solvers, at least the results do not change with number of ranks...
|
|
||
| } | ||
|
|
||
| void CFEASolver::Set_VertexEliminationSchedule(CGeometry *geometry, const vector<unsigned short>& markers) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the method to detect problem nodes.
| Jacobian.InitiateComms(LinSysSol, geometry, config, SOLUTION_MATRIX); | ||
| Jacobian.CompleteComms(LinSysSol, geometry, config, SOLUTION_MATRIX); | ||
|
|
||
| SU2_OMP(section) | ||
| for (auto iPoint = nPointDomain; iPoint < nPoint; iPoint++) | ||
| LinSysSol.SetBlock_Zero(iPoint); | ||
| for (auto iPoint : ExtraVerticesToEliminate) { | ||
| Jacobian.EnforceSolutionAtNode(iPoint, LinSysSol.GetBlock(iPoint), LinSysRes); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And this is what we need to do before solving the system.
|
I'll merge this in with #833... |
Proposed Changes
Improvements and general cleanup / restructuring of CInterpolator and co. consisting of:
New features:
RADIAL_BASIS_FUNCTION_PRUNE_TOLERANCEto drop relatively small coefficients.NUM_NEAREST_NEIGHBORS(because "why not").Enhancements:
Maintenance:
Other enhancements:
DEFORM_STIFF_LAYER_SIZE = [0]whenDEFORM_STIFFNESS_TYPE = WALL_DISTANCE. This allows refined regions around a wing tip to be dragged instead of sheared (which happens with the simple 1/x variation).Other fixes:
I know, there's way too much stuff in this PR, but it was all required to make 3D FSI more reliable, and hey, it works!

PR Checklist