Bug? Strange spot in turbulence field on the wall at the edge of the mpi partition

[bigfooted]

I did some more simulations on my 90 degree pipe bend and I noticed some spots in y+, tke and omega. This is a test for wall functions by the way, where I impose the values for tke and omega.
A picture is here:
https://github.com/bigfooted/su2cases/blob/master/validation/sudo_pipe_bend/paraview_screenshot_hotspot_tke.png

It looks like these spots appear at the edge of the mpi partitions. A further examination showed that these high tke values only appear on halo nodes, where we have:
(!geometry->nodes->GetDomain(iPoint))

Not sure how to get rid of these spots. Since they are on halo nodes, do they then not affect the solution and it is only a visualization thing?

[pcarruscag]

We talked about this in the context of a fix by @maxaehle (illegal access to normal neighbor IIRC)
Do you get exactly the same solution in serial and parallel?

[bigfooted]

Finally had some more time to look at this... The issue was mentioned in #1430.
It looks like it is related to this issue. It also looks like it is more than a visualization issue, since tke downstream is affected by the spots in the tke at the edges of the mpi partitions. The origin of the problem is YPlus, UTau and EddyViscWall, so the computations in CIncNSSolver::SetTauWall_WF

My understanding of do's and don't wrt halo nodes is limited. In CTurbSSTSolver::SetTurbVars_WF for instance, we do not skip halo nodes, which we do skip in the computation of the Y+ in CIncNSSolver::SetTau_Wall_WF. This at least seems inconsistent. So we have to skip halo nodes in SetTurbVars_WF? But don't we need to set something at the halo nodes?

[bigfooted]

So, if I compute y+ etc at halo nodes in SetTauWall_WF ( I commented the line if (!geometry->nodes->GetDomain(iPoint)) continue; ) then there are no spots anymore in the solution.
But this would then be a problem for full MG problems?

[pcarruscag]

If the information is all there to compute on halo nodes then that is the best option.
You can skip halos only on coarse levels, since it should not matter for the final solution, or for robustness it may even be better to not use wall functions on coarse levels, just setting safe values.

[bigfooted]

OK, I will then compute on halo nodes as well. Why would it be not robust to use wall functions on coarse levels? With turbulence, I'd say wall functions would be good on coarse levels as well since the y+ is larger on the coarse levels.

Is this (potential issues with too coarse mesh near the wall) not the reason that turbulence uses SingleGrid_Iteration?

[pcarruscag]

You can check MG_Level of the solver for MESH_0, no need to pass iMesh around.
Anything that requires gradients is a robustness problem on coarse levels because the mesh is bad.
No, I think the reason is simplicity.