LSP should maybe ignore .json files by default

[bartlomieju]

User reports that using a 8.4Mb JSON file causes a slow down in the LSP. We should investigate how this file in used in the project (is it imported somewhere?) and figure out how to speed it up. Maybe we should disable analysis of JSON files altogether?

[marvinhagemeister]

Wondering if it would make sense to parse that file in rust, generate a simple definition file in rust and then hand it over to TSC. Looking at the JSON file the type would be:

export default Array<[number, {
  key: number,
  value: {
    id: number,
    first: string,
    last: string,
    age: number
  }
}]>

[nhrones]

The app repo is https://github.com/nhrones/BuenoCache
The json file is /workers/hundredk.json
It's imported in /workers/buildWorker.js

Note: This is not a Deno project!
Deno is enabled in this project to support my custom Deno-dev-server -- (HotServe).](https://github.com/nhrones/Hot)

[nhrones]

This has happened once again with the same (unprotected) JSON file.
Note: I never got any indication of excessive CPU usage with this event.
When this has happened in the past, my laptop fan would go full blast, and the UI would be slow to respond just prior to the notification.
Again, if I use disablePaths setting for this JSON file, this does not happen.
Also note that this is on a new but very inexpensive HP laptop with fixed 8GB memory. (Win11)

VS Code OUT:

Starting Deno language server...
  version: 1.38.4 (release, x86_64-pc-windows-msvc)
  executable: C:\Users\nhron\.deno\bin\deno.EXE
Connected to "Visual Studio Code" 1.84.2
Enabling import suggestions for: https://deno.land
Server ready.

<--- Last few GCs --->

[15020:0000016B9E4D7510]   165736 ms: Scavenge 3036.3 (3098.4) -> 3033.3 (3099.9) MB, 16.26 / 0.00 ms  (average mu = 0.981, current mu = 0.979) allocation failure; 
[15020:0000016B9E4D7510]   165765 ms: Scavenge 3038.1 (3100.1) -> 3035.9 (3104.1) MB, 8.38 / 0.00 ms  (average mu = 0.981, current mu = 0.979) allocation failure; 
[15020:0000016B9E4D7510]   165801 ms: Scavenge 3041.7 (3104.1) -> 3038.9 (3121.9) MB, 16.65 / 0.00 ms  (average mu = 0.981, current mu = 0.979) allocation failure; 


<--- JS stacktrace --->


#
# Fatal JavaScript out of memory: Reached heap limit
#

[Info  - 12:58:34 PM] Connection to server got closed. Server will restart.
Starting Deno language server...
  version: 1.38.4 (release, x86_64-pc-windows-msvc)
  executable: C:\Users\nhron\.deno\bin\deno.EXE
Connected to "Visual Studio Code" 1.84.2
Enabling import suggestions for: https://deno.land

<--- Last few GCs --->

[11844:00000161077D3E80]    22092 ms: Scavenge 3036.7 (3098.9) -> 3033.4 (3100.6) MB, 30.12 / 0.00 ms  (average mu = 0.969, current mu = 0.954) allocation failure; 
[11844:00000161077D3E80]    22126 ms: Scavenge 3038.4 (3100.6) -> 3036.0 (3104.9) MB, 14.18 / 0.00 ms  (average mu = 0.969, current mu = 0.954) allocation failure; 
[11844:00000161077D3E80]    22160 ms: Scavenge 3041.9 (3104.9) -> 3038.9 (3122.1) MB, 14.60 / 0.00 ms  (average mu = 0.969, current mu = 0.954) allocation failure; 


<--- JS stacktrace --->


#
# Fatal JavaScript out of memory: Reached heap limit
#

[Info  - 12:58:57 PM] Connection to server got closed. Server will restart.
Starting Deno language server...
  version: 1.38.4 (release, x86_64-pc-windows-msvc)
  executable: C:\Users\nhron\.deno\bin\deno.EXE
Connected to "Visual Studio Code" 1.84.2
Enabling import suggestions for: https://deno.land

<--- Last few GCs --->

[23104:0000020475D8AB80]    22375 ms: Scavenge 3037.3 (3099.6) -> 3034.0 (3101.6) MB, 16.00 / 0.00 ms  (average mu = 0.965, current mu = 0.947) allocation failure; 
[23104:0000020475D8AB80]    22427 ms: Scavenge 3039.0 (3101.6) -> 3036.5 (3104.9) MB, 19.70 / 0.00 ms  (average mu = 0.965, current mu = 0.947) allocation failure; 
[23104:0000020475D8AB80]    22463 ms: Scavenge 3042.6 (3105.1) -> 3039.7 (3123.1) MB, 15.99 / 0.00 ms  (average mu = 0.965, current mu = 0.947) allocation failure; 


<--- JS stacktrace --->


#
# Fatal JavaScript out of memory: Reached heap limit
#

[Info  - 12:59:19 PM] Connection to server got closed. Server will restart.
Starting Deno language server...
  version: 1.38.4 (release, x86_64-pc-windows-msvc)
  executable: C:\Users\nhron\.deno\bin\deno.EXE
Connected to "Visual Studio Code" 1.84.2
Enabling import suggestions for: https://deno.land
Server ready.

<--- Last few GCs --->

[4304:000002747704CD50]    30155 ms: Scavenge 3034.4 (3096.4) -> 3031.4 (3098.1) MB, 13.61 / 0.00 ms  (average mu = 0.972, current mu = 0.953) allocation failure; 
[4304:000002747704CD50]    30189 ms: Scavenge 3036.2 (3098.4) -> 3034.0 (3102.1) MB, 8.20 / 0.00 ms  (average mu = 0.972, current mu = 0.953) allocation failure; 
[4304:000002747704CD50]    30225 ms: Scavenge 3039.9 (3102.1) -> 3037.1 (3120.1) MB, 15.51 / 0.00 ms  (average mu = 0.972, current mu = 0.953) allocation failure; 


<--- JS stacktrace --->


#
# Fatal JavaScript out of memory: Reached heap limit
#

[Info  - 12:59:50 PM] Connection to server got closed. Server will restart.
Starting Deno language server...
  version: 1.38.4 (release, x86_64-pc-windows-msvc)
  executable: C:\Users\nhron\.deno\bin\deno.EXE
Connected to "Visual Studio Code" 1.84.2
Enabling import suggestions for: https://deno.land
Server ready.

<--- Last few GCs --->

[25616:00000174A3B53660]    25818 ms: Scavenge 3035.9 (3097.9) -> 3032.6 (3099.4) MB, 11.76 / 0.00 ms  (average mu = 0.970, current mu = 0.956) allocation failure; 
[25616:00000174A3B53660]    25860 ms: Scavenge 3037.6 (3099.6) -> 3035.1 (3102.9) MB, 19.81 / 0.00 ms  (average mu = 0.970, current mu = 0.956) allocation failure; 
[25616:00000174A3B53660]    25897 ms: Scavenge 3041.1 (3103.1) -> 3038.2 (3121.1) MB, 14.38 / 0.00 ms  (average mu = 0.970, current mu = 0.956) allocation failure; 


<--- JS stacktrace --->


#
# Fatal JavaScript out of memory: Reached heap limit
#

[Error - 1:00:16 PM] The Deno Language Server server crashed 5 times in the last 3 minutes. The server will not be restarted. See the output for more information.

[nhrones]

Heads up:
Even with the following settings disabling this file, I still have issues.

{
   "deno.enable": true,
   "deno.lint": true,
   "deno.unstable": true,
   "deno.disablePaths": ["./hundredk.json", "./.vscode"],
}

If I actually open the file in VS Code, I get the OOM crash.

#
# Fatal JavaScript out of memory: Reached heap limit
#

Note: I rarely open this file, It was auto generated to be used for DB testing.

[nhrones]

LSP-Crash-Test

SEE: https://github.com/nhrones/LSP-test

LSP OOM crash after about 3 minutes.

Small two file test app, one being a large JSON data file.

deno.disablePaths not working?

[nayeemrmn]

JSON files already aren't preloaded by default unless they are imported, in which case their types need to be analysed. We determined this wasn't the cause for most users, just the isolated case linked here. Closing.

[nhrones]

Still getting OOM crash, even without any large files. Seems related to long edit sessions. This on a wimpy 8GIG mem Win11 laptop. I've learned to just expect it and simply restart VS Code. Error states something to the effect failed with OOM three times in ... Server will not be restarted!

[nayeemrmn]

Still getting OOM crash, even without any large files.

So that's not related to the JSON file, right? I think this particular issue isn't worth pursuing.

[nhrones]

Still getting OOM crash, even without any large files.

So that's not related to the JSON file, right? I think this particular issue isn't worth pursuing.

Yes I agree. In my apps with large json files, I've been able to isolate this issue with your recommended fix

{
   "deno.enable": true,
   "deno.disablePaths": ["./dist","./.vscode", "./data"],
}

I only wanted to provide feedback that on my laptop, I'm still getting random LSP crash.
The logs report that the LSP has used 3.2GB (all available memory), and resets. After the third reset (hours of editing in same project), The report is LSP-OOM - LSP will not be reset!
At that point I just restart VS Code.

I would agree that this issue should be closed.

[nayeemrmn]

The above report wasn't related to JSON files