The new intellisense parsing, VS2022 dies to StackOverflow in CheckDeclarations.fs

[Thorium]

After latest F# and VS2022 batches the Visual Studio has went unreliable and just disappears sometimes when writing new text, leaving Windows Eventlog a devenv.exe error entry.

I attached another Visual Studio instance to my Visual Studio to see why it dies.

I have an F# file of 9313 lines, and meanwhile writing to anywhere really, it seems to be that compiler intellisense throws stack-overflow and that ends-up killing the whole VisualStudio 2022. Maybe the code after error, the unknown-part, is too big for the compiler to try to parse?

Here the "Cannot evaluate expression. Unknown error 0x80040014 from .NET debugging services." makes totally sense, as meanwhile typing the source-code cannot be parsed, I'm in a middle of a word!

The stackoverflow seems to come from CheckDeclarations.fs function TcMutRecDefns_Phase2:

fsharp/src/Compiler/Checking/CheckDeclarations.fs

Line 2110 in c910057

with RecoverableException exn -> errorRecovery exn scopem; [], envMutRec

Callstack:

FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcMutRecDefns_Phase2(FSharp.Compiler.CheckBasics.TcFileState cenv, FSharp.Compiler.CheckBasics.TcEnv envInitial, FSharp.Compiler.Text.Range mBinds, FSharp.Compiler.Text.Range scopem, Microsoft.FSharp.Core.FSharpOption<System.Tuple<Microsoft.FSharp.Core.FSharpOption<FSharp.Compiler.TypedTree.Entity>, Microsoft.FSharp.Core.FSharpRef<FSharp.Compiler.TypedTree.ModuleOrNamespaceType>>> mutRecNSInfo, FSharp.Compiler.CheckBasics.TcEnv envMutRec, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.CheckDeclarations.MutRecShape<FSharp.Compiler.CheckDeclarations.MutRecDefnsPhase2DataForTycon, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.CheckExpressions.RecDefnBindingInfo>, System.Tuple<FSharp.Compiler.CheckDeclarations.MutRecDefnsPhase2DataForModule, FSharp.Compiler.CheckBasics.TcEnv>>> mutRecDefns, bool isMutRec) Line 2097 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcDeclarations.TcMutRecDefinitions(FSharp.Compiler.CheckBasics.TcFileState cenv, FSharp.Compiler.CheckBasics.TcEnv envInitial, FSharp.Compiler.TypedTree.ParentRef parent, Microsoft.FSharp.Collections.FSharpSet typeNames, FSharp.Compiler.CheckBasics.UnscopedTyparEnv tpenv, FSharp.Compiler.Text.Range m, FSharp.Compiler.Text.Range scopem, Microsoft.FSharp.Core.FSharpOption<System.Tuple<Microsoft.FSharp.Core.FSharpOption<FSharp.Compiler.TypedTree.Entity>, Microsoft.FSharp.Core.FSharpRef<FSharp.Compiler.TypedTree.ModuleOrNamespaceType>>> mutRecNSInfo, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.CheckDeclarations.MutRecShape<FSharp.Compiler.Syntax.SynTypeDefn, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Syntax.SynBinding>, FSharp.Compiler.Syntax.SynComponentInfo>> mutRecDefns, bool isMutRec) Line 4558 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5093-1.Invoke(System.Threading.CancellationToken ct) Line 5113 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5093-18.Invoke(System.Threading.CancellationToken ct) Line 5093 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementsNonMutRec(FSharp.Compiler.CheckBasics.TcFileState cenv, FSharp.Compiler.TypedTree.ParentRef parent, Microsoft.FSharp.Collections.FSharpSet typeNames, FSharp.Compiler.Text.Range endm, Microsoft.FSharp.Collections.FSharpList<System.Tuple<Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.TypedTree.ModuleOrNamespaceContents>, Microsoft.FSharp.Collections.FSharpList<Microsoft.FSharp.Core.FSharpFunc<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Core.Unit>>, Microsoft.FSharp.Collections.FSharpList<System.Tuple<System.AttributeTargets, FSharp.Compiler.TypedTree.Attrib>>>> defsSoFar, FSharp.Compiler.CheckBasics.TcEnv env, FSharp.Compiler.CheckBasics.TcEnv envAtEnd, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Syntax.SynModuleDecl> moreDefs, System.Threading.CancellationToken ct) Line 5337 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5469-5.Invoke(System.Threading.CancellationToken ct) Line 5469 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5450-8.Invoke(System.Threading.CancellationToken ct) Line 5450 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5196-15.Invoke(System.Threading.CancellationToken ct) Line 5196 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5093-18.Invoke(System.Threading.CancellationToken ct) Line 5093 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementsNonMutRec(FSharp.Compiler.CheckBasics.TcFileState cenv, FSharp.Compiler.TypedTree.ParentRef parent, Microsoft.FSharp.Collections.FSharpSet typeNames, FSharp.Compiler.Text.Range endm, Microsoft.FSharp.Collections.FSharpList<System.Tuple<Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.TypedTree.ModuleOrNamespaceContents>, Microsoft.FSharp.Collections.FSharpList<Microsoft.FSharp.Core.FSharpFunc<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Core.Unit>>, Microsoft.FSharp.Collections.FSharpList<System.Tuple<System.AttributeTargets, FSharp.Compiler.TypedTree.Attrib>>>> defsSoFar, FSharp.Compiler.CheckBasics.TcEnv env, FSharp.Compiler.CheckBasics.TcEnv envAtEnd, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Syntax.SynModuleDecl> moreDefs, System.Threading.CancellationToken ct) Line 5337 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5469-5.Invoke(System.Threading.CancellationToken ct) Line 5469 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5450-8.Invoke(System.Threading.CancellationToken ct) Line 5450 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5285-17.Invoke(System.Threading.CancellationToken ct) Line 5285 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5093-18.Invoke(System.Threading.CancellationToken ct) Line 5093 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementsNonMutRec(FSharp.Compiler.CheckBasics.TcFileState cenv, FSharp.Compiler.TypedTree.ParentRef parent, Microsoft.FSharp.Collections.FSharpSet typeNames, FSharp.Compiler.Text.Range endm, Microsoft.FSharp.Collections.FSharpList<System.Tuple<Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.TypedTree.ModuleOrNamespaceContents>, Microsoft.FSharp.Collections.FSharpList<Microsoft.FSharp.Core.FSharpFunc<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Core.Unit>>, Microsoft.FSharp.Collections.FSharpList<System.Tuple<System.AttributeTargets, FSharp.Compiler.TypedTree.Attrib>>>> defsSoFar, FSharp.Compiler.CheckBasics.TcEnv env, FSharp.Compiler.CheckBasics.TcEnv envAtEnd, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Syntax.SynModuleDecl> moreDefs, System.Threading.CancellationToken ct) Line 5337 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5469-5.Invoke(System.Threading.CancellationToken ct) Line 5469 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5450-8.Invoke(System.Threading.CancellationToken ct) Line 5450 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.CheckOneImplFile@5718-2.Invoke(System.Threading.CancellationToken ct) Line 5718 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CheckDeclarations.CheckOneImplFile@5696-1.Invoke(System.Threading.CancellationToken ct) Line 5718 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.ParseAndCheckInputs.CheckOneInput@1396-15.Invoke(System.Threading.CancellationToken ct) Line 1396 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.ParseAndCheckInputs.CheckOneInput@1390-16.Invoke(System.Threading.CancellationToken ct) Line 1390 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.ParseAndCheckInputs.CheckOneInput@1320-2.Invoke(System.Threading.CancellationToken ct) Line 1390 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.ParseAndCheckInputs.CheckOneInput@1319-17.Invoke(System.Threading.CancellationToken ct) Line 1319 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.ParseAndCheckInputs.CheckOneInputAndFinish@1464-1.Invoke(System.Threading.CancellationToken ct) Line 1464 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3245-7.Invoke(System.Threading.CancellationToken ct) Line 3245 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3242-6.Invoke(System.Threading.CancellationToken ct) Line 3245 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3232-8.Invoke(System.Threading.CancellationToken ct) Line 3232 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3230-11.Invoke(System.Threading.CancellationToken ct) Line 3230 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3215-3.Invoke(System.Threading.CancellationToken ct) Line 3230 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3213-2.Invoke(System.Threading.CancellationToken ct) Line 3215 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.ParseAndCheckFile.CheckOneFile@3192-1.Invoke(System.Threading.CancellationToken ct) Line 3213 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.CodeAnalysis.FSharpCheckFileResults.CheckOneFile@3665-13.Invoke(System.Threading.CancellationToken ct) Line 3665 C#
FSharp.Compiler.Service.dll!Internal.Utilities.Library.Cancellable.toAsync@75-1<FSharp.Compiler.CodeAnalysis.FSharpCheckFileResults>.Invoke(System.Threading.CancellationToken ct) Line 75 C#
FSharp.Compiler.Service.dll!FSharp.Compiler.BuildGraph.GraphNode<System.Tuple<FSharp.Compiler.CodeAnalysis.FSharpParseFileResults, FSharp.Compiler.CodeAnalysis.FSharpCheckFileResults, long, System.DateTime>>.GetOrComputeValue@67-5.Invoke(Microsoft.FSharp.Core.Unit _arg2) Line 72 C#
FSharp.Editor.dll!Microsoft.VisualStudio.FSharp.Editor.WorkspaceExtensions.CheckerExtensions.FSharpChecker-ParseAndCheckDocumentWithPossibleStaleResults@389-4.Invoke(System.Threading.CancellationToken ct) Unknown
FSharp.Editor.dll!Microsoft.VisualStudio.FSharp.Editor.WorkspaceExtensions.CheckerExtensions.FSharpChecker-ParseAndCheckDocumentWithPossibleStaleResults@387-3.MoveNext() Unknown

Event-log:

Faulting application name: devenv.exe, version: 17.11.35303.130, time stamp: 0x66d77afb
Faulting module name: clr.dll, version: 4.8.9261.0, time stamp: 0x667a1925
Exception code: 0xc00000fd
Fault offset: 0x0000000000062d57
Faulting process id: 0x0x3C9C
Faulting application start time: 0x0x1DB055063C73BE6
Faulting application path: C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\devenv.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: 933f8128-ee82-4d71-b12d-a1746671f9fe

Environment:
Microsoft Visual Studio Professional 2022 (64-bit) - Current Version 17.11.3
Windows 11 - Dotnet 8.0.400

I don't need intellisense outside my screen, so there is no reason to try to parse the whole file through!
And in case of any parsing error, it should just give-up and not parse.

[vzarytovskii]

I don't need intellisense outside my screen, so there is no reason to try to parse the whole file through!
And in case of any parsing error, it should just give-up and not parse.

It doesn't work like that unfortunately, to do anything it should typecheck everything dependant.

Regardless, it should be fixed in #17654

[Thorium]

Ok, hope to get the fix soon to live. It's VS2022 release we need to wait for and not FSharp.Core, right?

[vzarytovskii]

Ok, hope to get the fix soon to live. It's VS2022 release we need to wait for and not FSharp.Core, right?

Yes, 17.12

[Thorium]

Tried it with current 17.12 preview-2 released 4 days ago, and I can confirm this is not fixed.
The VS 2022 17.12 still dies as the old one.

Faulting application name: devenv.exe, version: 17.12.35309.182, time stamp: 0x3e83f198
Faulting module name: VCRUNTIME140_1_CLR0400.dll, version: 14.32.31326.0, time stamp: 0x8f307fd5
Exception code: 0xc00000fd
Fault offset: 0x00000000000016a8
Faulting process id: 0x0x84AC
Faulting application start time: 0x0x1DB06D20D3D8FEE
Faulting application path: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\devenv.exe
Faulting module path: C:\Windows\SYSTEM32\VCRUNTIME140_1_CLR0400.dll
Report Id: 73482dc8-d1fd-496f-aeca-e8ac49d319ba


Problem signature:
P1: devenv.exe
P2: 17.12.35309.182
P3: 3e83f198
P4: VCRUNTIME140_1_CLR0400.dll
P5: 14.32.31326.0
P6: 8f307fd5
P7: c00000fd
P8: 00000000000016a8

The current source-code seems to match the lines of the Github main.

[vzarytovskii]

It is not in preview2 yet, p2 was done (frozen) when fix was merged. Either 3 or 4.

It was just inserted into vs 4 days ago: #17693 preview builds are usually done days before release, so p2 is likely ~10d old, maybe even more.

[vzarytovskii]

@T-Gro could you please publish a vsix here on Monday, so @Thorium can install it to 17.2 locally and test it?

[T-Gro]

fsharp_vsix.zip

This is a .vsix built now.
Do not forget to uninstall it once a new VS update comes in, otherwise it would stay in overwriting the version coming together with VS.

[Thorium]

Hi,
It's a bit slow, intellisense takes 14 seconds (I measured, on my laptop, 13th Gen i9-13900H, 2600 Mhz, 14 Core(s), 20 Logical Processor(s)).

But the main thing confirmed: the Visual Studio is not randomly dying anymore.
Which is excellent.

[vzarytovskii]

Hi, It's a bit slow, intellisense takes 14 seconds (I measured, on my laptop, 13th Gen i9-13900H, 2600 Mhz, 14 Core(s), 20 Logical Processor(s)).

Cold start is kinda expected to be slow on big files. We hope to address it once we migrate to new out-of-proc LSP-based extension in future. Or rather we will be able to do more, now we are tied to net472 hosting-in-process in VS, which brings a lot of limitations :(

[Thorium]

Sadly this wasn't even a cold-start... But the code uses multiple type-providers and query-expressions, etc, and I think major part of slowness is (in the ecosystem, but) outside FSharp core itself. The slowness (and also full memory-footprint reduction) could be addressed by trying to optimize e.g. the the ProvidedTypes.fs, for example: If type-providers are accepted to be large amount of independent type entities, why does the interface expect (linked-)lists and not arrays? Many F# libraries seem to use recklessly ConcurrentDictionaries as caches, one per type. And so on.

[Thorium]

Generic off-topic question: These are top 20 by count items in my memory snapshot of devenv.exe of an F# project. Would it help to make thise FSharpLists to arrays:

If I look it by Inclusive Byes, then it's all about FSharpAsync<...> and MruCache of code-analysis results.

[vzarytovskii]

Generic off-topic question: These are top 20 by count items in my memory snapshot of devenv.exe of an F# project. Would it help to make thise FSharpLists to arrays:

If I look it by Inclusive Byes, then it's all about FSharpAsync<...> and MruCache of code-analysis results.

It's case by case, we use immutable nature of lists a lot, as well as destructuring and pattern matching.

[T-Gro]

In case they are not being reused, we could move some of them from lists to ImmutableArrays.
However, that comes at a readability cost in idiomatic F# (pattern matching support most notably), so we would need to find a usage which fits this tradeoff.

[xperiandri]

Which release is it expected to appear?

[vzarytovskii]

17.12

[xperiandri]

@T-Gro I tried the VSIX above and autocomplete execution takes 40 seconds for popup to appear

[T-Gro]

I am sorry about that.
That was a local built for which I did not do native NGEN, but I am surprised the difference is that bad.

Was it the first autocomplete (possibly causing a lot of .dll loading and JITing), or is it that slow all the time?