C/C++ intellisense extremely slow with large codebases (due to populating file name cache)

[kitswas]

Environment

• OS and Version: Windows 11 (Windows_NT x64 Build 10.0.22631)
• VS Code Version: 1.86.2 (system setup)
• C/C++ Extension Version: v1.19.9
• If using SSH remote, specify OS of remote machine: Output of uname -a
Linux hp-PowerEdge-T440 6.5.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Bug Summary and Steps to Reproduce

Bug Summary:

Large codebases like ns-3 take 2~5 min after opening and ~10s after that for whichever file you open to prepare syntax highlighting and IntelliSense.

Steps to reproduce:

1. Git clone https://gitlab.com/nsnam/ns-3-dev.git
2. Checkout any release tag. I am using v3.40. git checkout tags/ns-3.40
3. Open the folder in VSCode.
4. Open any file.
5. Watch the syntax highlighting and IntelliSense take ages to load. (Build all CMake targets first if it fails)
6. The problem is more pronounced when using third-party modules. (More code in the contrib directory)

Expected behaviour:

1. Speed up the process, please. If VSCode needs more processes, it can have them.
2. If direct speedup is not possible, VSCode should at least prepare IntelliSense for the files open in the current workspace in the background after the working file is ready.

Configuration and Logs

c_cpp_properties.json

{
    "configurations": [
        {
            "name": "Linux",
            "includePath": [
                "${workspaceFolder}/**",
                "${workspaceFolder}/build/include"
            ],
            "defines": [],
            "compilerPath": "/usr/bin/gcc",
            "cStandard": "c17",
            "cppStandard": "gnu++17",
            "intelliSenseMode": "linux-gcc-x64",
            "configurationProvider": "ms-vscode.cmake-tools"
        }
    ],
    "version": 4
}

Other Extensions

No response

Additional context

htop shows only a single CPU core at 100% when a new file is opened. All the others are idle. (<10%)

[trustytrojan]

IMHO, it doesn't seem like performance is this extension's priority, because there are a ton of issues titled similarly to this. I also wish it could be faster, but I ended up resorting to the clangd extension instead, which somehow leaves cpptools in the dust when parsing large codebases. In every other aspect though, this extension is nicer.

[bobbrow]

@kitswas I just tried cloning your project and configured with CMake on Windows. Is there a particular file that you are having problems with? I wasn't able to repro a performance problem with the few files I tried.

ns-3-dev.mp4

This is on a local machine though. You seem to indicate that you're connecting to a remote machine? It's not WSL, is it? Can you share more details about your remote setup? Our extension doesn't manage communication back and forth between machines, but it's possible that something is configured sub-optimally that is resulting in delays. It looked like there were only about 3300 source/header files in this repo and our extension can normally handle a project of that size ok.

@trustytrojan We feel your pain and have been experimenting with alternate approaches to deal with the performance issues you are referring to. These efforts have not yielded the wholesale improvements we're looking for yet, but we have been scaling up the investment here and hope to announce more improvements in the future. Our most recent improvements were released in version 1.19 (progressive IntelliSense updates, and a rewrite of the symbol search feature).

[sean-mcmanus]

@kitswas Also, if you can set C_Cpp.loggingLeve to "Debug", the logging might indicate what is taking so long to process. Also, with 1.20.0 (pre-release), you could attach a debugger and provide the call stacks (for the thread that appears to be using the CPU).

[kitswas]

@bobbrow

I just tried cloning your project and configured with CMake on Windows. Is there a particular file that you are having problems with? I wasn't able to repro a performance problem with the few files I tried.

ns3 can only run/compile in a Linux environment. To use ns-3 on Windows, you need the Windows Subsystem for Linux (WSL) or the Msys2/MinGW64 toolchain.

Did you wait for all the symbols to get highlighted and the popup-on-hover to start working?
The C++ and the 🔥(IntelliSense) spinners sometimes stop early.

This is on a local machine though. You seem to indicate that you're connecting to a remote machine? It's not WSL, is it? Can you share more details about your remote setup? Our extension doesn't manage communication back and forth between machines, but it's possible that something is configured sub-optimally that is resulting in delays.

I am connecting to an Ubuntu 22.04 machine via ssh over LAN.

It looked like there were only about 3300 source/header files in this repo and our extension can normally handle a project of that size ok.

That's the base ns3 repo with no third-party modules. My current repo has 4177 C++ source and header files.
Let's see if I can get you something closer to what I am working with now.

[kitswas]

@sean-mcmanus
Here's the log (level Debug) after switching to a different file.

after_editor_change.log

It contains the lines:

Populating file name cache...
Done populating filename cache. Elapsed time: 15585 ms
...
Update IntelliSense time (sec): 9.954

That's a total of 25 seconds for Intellisense to become fully functional.

[sean-mcmanus]

@kitswas Yes, it's stuck populating the file name cache. This is a performance issue we've are aware of and I believe another member on our team is actively working on fixing that (unless I'm mistaken). One prior bug is #11954 , although that doesn't mention IntelliSense being blocked, so we can use this issue to track the IntelliSense case.

Well, that's for the 15 sec, but the 10 sec updating time might be improvied via #3628 or #5362 or other performance work that is in progress.

[bobbrow]

As @sean-mcmanus mentions, the filename cache is an area we're working to improve. Soon, this will only delay IntelliSense the first time you open a workspace, not every time you open the same workspace. For me with all files local to the machine, this doesn't take long.

Done populating filename cache. Elapsed time: 139 ms

Did you wait for all the symbols to get highlighted and the popup-on-hover to start working?

I didn't have to wait long. I'm not sure why your machine takes 10 seconds and mine takes 1 second.

Update IntelliSense time (sec): 0.879

We have heard of issues with remote work taking longer, but it's hard to know. Are all of your files/compilers/headers local to the remote machine, or are there any network drives involved?

All of my remotes are dead right now, but I was able to spin up a Codespace for this repo and saw some numbers closer to yours. filename cache took 3~5 seconds to populate. IntelliSense took 3~8 seconds depending on the source file.

ns3 can only run/compile in a Linux environment. To use ns-3 on Windows, you need the Windows Subsystem for Linux (WSL) or the Msys2/MinGW64 toolchain.

I didn't try to build on Windows. I just configured with CMake to get IntelliSense configurations and tested the IntelliSense. The Codespace is linux.

That's the base ns3 repo with no third-party modules. My current repo has 4177 C++ source and header files.
Let's see if I can get you something closer to what I am working with now.

I was looking for instructions, but didn't see anything about submodules or anything. If you have more info on how to set this up, I can add the extra files and see if it makes a difference. Also, the file in your log wasn't in the default repo (contrib is empty for me), so I couldn't test with that file.

[kitswas]

Thank you for your patience.

We have heard of issues with remote work taking longer, but it's hard to know. Are all of your files/compilers/headers local to the remote machine, or are there any network drives involved?

All my files are on a drive local to the remote machine (Dell PowerEdge t440).

I was looking for instructions, but didn't see anything about submodules or anything. If you have more info on how to set this up, I can add the extra files and see if it makes a difference. Also, the file in your log wasn't in the default repo (contrib is empty for me), so I couldn't test with that file.

Try this repo. https://gitlab.com/kitswas/coexistence-wigig-5gnr/-/tree/master/scratch?ref_type=heads#3rd-party-modules
Please follow the instructions to set up contrib.
The ns-3 project does not use git submodules yet.

[heartacker]

same here
and any source file editing make c-language spinning(bottom status bar show me this)， and block the formatting util the unspinning

[XuPengfei-1020]

Same issue.
Vscode remote mode(client on mac or windows, server on a pertty high performance linux server):
The IntelliSense is rather not soomth when I open a cpp file.
Whenever I open a C++ file(cpp or h), the 'IntelliSense scan' and 'parsing workspace' process will be launched.
It will take about a minute, and I can not use the functions such as 'Go to definition' or 'Go to implemention' which are dependent on IntelliSense funtion until the process is finished.
And If I go to a new cpp file, the scan and parsing process need run again before I can 'Go to definition'...

Vscode local mode:

1. run on mac:
   IntelliSense Performance issue is sill there.
2. run on windows:
   IntelliSense Performance is far better than on mac(local mode) and on linux(Remote mode).

[sean-mcmanus]

@kitswas @trustytrojan @XuPengfei-1020 We've made improvements with https://github.com/microsoft/vscode-cpptools/releases/tag/v1.22.0 . Let us know if it's still an issue for you with that version.

[lamnguyenx]

IMHO, it doesn't seem like performance is this extension's priority, because there are a ton of issues titled similarly to this. I also wish it could be faster, but I ended up resorting to the clangd extension instead, which somehow leaves cpptools in the dust when parsing large codebases. In every other aspect though, this extension is nicer.

As of today, I've updated C/C++ Extension to v1.22.9 for VSCode 1.93.1 and still got very high disk I/O when opening C++ project on remote.

My source code is not that large, but linked to PyTorch and CUDA.

Anyway, after switching to Clangd, it's so much faster. It also tells me about unused header files.

[anoban]

29th November 2024

VSCode info ::

Version: 1.95.3 (system setup)
Commit: f1a4fb101478ce6ec82fe9627c43efbf9e98c813
Date: 2024-11-13T14:50:04.152Z
Electron: 32.2.1
ElectronBuildId: 10427718
Chromium: 128.0.6613.186
Node.js: 20.18.0
V8: 12.8.374.38-electron.0
OS: Windows_NT x64 10.0.22631

Extension info ::

Name: C/C++
Id: ms-vscode.cpptools
Description: C/C++ IntelliSense, debugging, and code browsing.
Version: 1.22.11
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools

Configs ::

{
    "C_Cpp.autocompleteAddParentheses": true,
    "C_Cpp.clang_format_style": "",
    "C_Cpp.codeAnalysis.clangTidy.args": [],
    "C_Cpp.codeAnalysis.clangTidy.codeAction.showDisable": false,
    "C_Cpp.codeAnalysis.clangTidy.enabled": true,
    "C_Cpp.codeFolding": "disabled",
    "C_Cpp.default.compilerArgs": [
        "/C",
        "/DDEBUG",
        "/D_DEBUG",
        "/D__STDC_WANT_SECURE_LIB__",
        "/I./",
        "/I./include/",
        "/Zc:__cplusplus",
        "/Zc:preprocessor",
        "/EHsc",
        "/TP",
        "/Wall",
        "/Od",
        "/std:c++20"
    ],
    "C_Cpp.default.cStandard": "c23",
    "C_Cpp.default.cppStandard": "c++20",
    "C_Cpp.doxygen.generateOnType": false,
    "C_Cpp.errorSquiggles": "enabled",
    "C_Cpp.formatting": "clangFormat",
    "C_Cpp.inlayHints.autoDeclarationTypes.enabled": true,
    "C_Cpp.inlayHints.autoDeclarationTypes.showOnLeft": true,
    "C_Cpp.inlayHints.parameterNames.enabled": true,
    "C_Cpp.inlayHints.referenceOperator.enabled": true,
    "C_Cpp.markdownInComments": "enabled",
    "C_Cpp.suggestSnippets": false,
    "[cuda-cpp]": {
        "editor.wordBasedSuggestions": "allDocuments",
        "editor.suggest.insertMode": "replace",
        "editor.semanticHighlighting.enabled": true
    },
    "extensions.closeExtensionDetailsOnViewChange": true,
    "extensions.ignoreRecommendations": true,
    "files.associations": {
        ".clang-format": "yaml",
        ".clang-tidy": "yaml",
        ".clangd": "yaml",
        "*.cu": "cuda-cpp",
        "*.cuh": "cuda-cpp",
        "*.h": "c",
        "*.c": "c"
    },
    "C_Cpp.default.intelliSenseMode": "windows-msvc-x64",
    "C_Cpp.caseSensitiveFileSupport": "enabled",
    "C_Cpp.preferredPathSeparator": "Backslash",
    "C_Cpp.vcpkg.enabled": false,
    "C_Cpp.configurationWarnings": "disabled",
    "C_Cpp.refactoring.includeHeader": "ifNeeded",
    "C_Cpp.inactiveRegionOpacity": 0.25,
    "C_Cpp.intelliSenseUpdateDelay": 2000,
    "C_Cpp.renameRequiresIdentifier": false,
    "C_Cpp.workspaceParsingPriority": "low",
    "C_Cpp.vcFormat.indent.caseContents": false,
    "C_Cpp.vcFormat.indent.lambdaBracesWhenParameter": false,
    "C_Cpp.vcFormat.indent.namespaceContents": false,
    "C_Cpp.vcFormat.newLine.beforeCatch": false,
    "C_Cpp.vcFormat.newLine.beforeElse": false,
    "C_Cpp.vcFormat.space.afterComma": false,
    "C_Cpp.vcFormat.space.afterKeywordsInControlFlowStatements": false,
    "C_Cpp.vcFormat.space.beforeBlockOpenBrace": false,
    "C_Cpp.vcFormat.space.beforeConstructorColon": false,
    "C_Cpp.vcFormat.space.beforeInheritanceColon": false,
    "C_Cpp.vcFormat.space.groupSquareBrackets": false,
    "C_Cpp.vcFormat.space.removeBeforeSemicolon": false,
    "C_Cpp.vcFormat.space.removeAroundUnaryOperator": false,
    "C_Cpp.vcFormat.space.removeAroundMemberOperators": false,
    "C_Cpp.vcFormat.space.preserveInInitializerList": false,
    "C_Cpp.vcFormat.space.withinInitializerListBraces": false,
    "C_Cpp.simplifyStructuredComments": false,
    "C_Cpp.codeAnalysis.clangTidy.codeAction.showClear": "None",
    "C_Cpp.sshTargetsView": "disabled",
    "C_Cpp.intelliSenseCacheSize": 0,
    "C_Cpp.intelliSenseMemoryLimit": 1024,
    "C_Cpp.maxMemory": 1024,
    "C_Cpp.maxSymbolSearchResults": 500,
}

The same issue on Windows 11. Intellisense is horribly slow and never finishes (for a source file https://github.com/anoban/cpp23/blob/main/rvopt.cpp with just 42 lines of C++ code it takes about 12 minutes immediately after cold start). On the other hand clangd with VSCode insiders does everything in a fraction of a second. I'm not trying to downplay the usefulness of this extension because this is the only extension that provides decent C, C++ debugging experience on Windows but it has become obnoxiously sluggish. Is there a way to fix this from my end? I used to allow it to use memory liberally for caching but it never helped and sometimes consumed up to 2 GiBs of memory so I've now limited those options.

[bobbrow]

@anoban this issue is closed, so I moved your report to #13023