| T O P I C R E V I E W |
| kornman00 |
Posted - Jan 29 2021 : 5:57:14 PM
A project at work switched to CMake last year. Ever since then, when I open the project's folder in VS2019, it will eventually crash soon after when I have VAX enabled. If I disable VAX, VS2019 does not crash.
With VAX disabled, VS2019 sits at around 800MB of used memory with this project open and idle.
With VAX enabled, I can watch VS2019 continue to grow in Task Manager until it ends up crashing. Usually around 2785MB or so (since VS2019 is 32-bit, this suggests to me a possible OOM)
I enabled logging in VAX and ended up with a 21.7MB va.log file. Inspecting this file, there's an enormous amount of CollectFolderProjs logs (231013 instances), including the last line itself.
I suspect the problem relates to CollectFolderProjs not respecting the "ExcludedItems" array in VSWorkspaceSettings.json. This is what we change to make VS not populate folders or file patterns in the solution explorer.
Our project's root folder has a CMakeList.txt, and so the process is to open the root. However, there are many sub-projects (in sub-folders) which are *not* part of the CMake setup. So their root sub-folders are in "ExcludedItems".
These sub-projects include not only code (still on sln/vcxproj) but GBs of tools and source content in thousands of folders (they're individual games).
I suspect that since CollectFolderProjs isn't ignoring the sub-projects/folders we list in "ExcludedItems" that it ends up not only wasting cycles enumerating those folders, but is allocating memory from those scan results and the crash itself may be an OOM. For most other crashes in VS2019, I get a .dmp file in AppData\Local\CrashDumps. However, with this crash with VAX enabled there's no dump generated.
Another thing I tried was first disabling VAX, opening the project folder, then re-enabling VAX with the folder still opened. This doesn't result in a crash, because it doesn't automatically enumerate the project to gather intellisense data upon re-enabling.
Given all of this, whenever I want to work in the CMake project, I currently have to disable VAX altogether. Which is annoying to (remember) to do, but also means I'm left without my VAX-based Intellisense/code highlighting safety blanket. |
| 16 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Aug 15 2023 : 06:30:55 AM
I did find this page, which while a little old, does clearly state that this file was never supposed to be user editable, at least as far as Visual Studio is concerned:
https://github.com/MicrosoftDocs/visualstudio-docs/pull/3499
Visual Studio Code seems to be a different situation. |
| edl_si |
Posted - Aug 14 2023 : 1:54:24 PM
We don't use VS code as IDE for building we use it more a browser I suppose you'd say.
As far as I can tell when using a cmake project with VS the VSWorkspaceSettings.json is standard and works within VS for which files it displays. |
| feline |
Posted - Aug 14 2023 : 09:23:34 AM
Checking the case, it turns out that the advice from Microsoft is that the best, and most future proof method of excluding files from being included in Visual Studio is to use the ".vscode\settings.json" file, which is what we already support.
Out of interest, why do you want VS Code to show files that are not being compiled? It makes sense if you are editing them, but why edit files if editing them has no effect? |
| edl_si |
Posted - Aug 14 2023 : 07:02:39 AM
Hi,
We are just switching to cmake/open folder and are hitting the same issue. Just wanted to add our companies support to having VAX check the VSWorkspaceSettings.json file - we don't want to add VS code exclusions as we want VS code to show everything.
We also don't really want to have to duplicate our exclusion list.
Thanks,
Ed. |
| jussi |
Posted - Jul 06 2023 : 10:51:48 AM
quote:
Originally posted by feline
In the base directory you are using, since you may not be using a SLN file, can you please create the directory ".vscode", and then inside this directory create a new text file called "settings.json"
You need to give the file the following content:
{
"files.exclude": {
"cmake-build-debug": true
}
}
and then turn On the setting:
VA Options -> Performance -> Do not parse files excluded by .vscode\settings.json (requires restart)
which will require an IDE restart to take effect. Does this exclude these files? It should exclude this directory and everything under it.
Hey, thanks! This worked well.
|
| feline |
Posted - Jul 05 2023 : 12:17:40 PM
In the base directory you are using, since you may not be using a SLN file, can you please create the directory ".vscode", and then inside this directory create a new text file called "settings.json"
You need to give the file the following content:
{
"files.exclude": {
"cmake-build-debug": true
}
}
and then turn On the setting:
VA Options -> Performance -> Do not parse files excluded by .vscode\settings.json (requires restart)
which will require an IDE restart to take effect. Does this exclude these files? It should exclude this directory and everything under it. |
| jussi |
Posted - Jun 30 2023 : 12:57:38 PM
Hi!
I suspect I am hitting the same issue, maybe? Instead of getting my source files, I get all cmake dependencies.
Thanks |
| feline |
Posted - Mar 29 2022 : 11:28:51 AM
Apologies for the bad experience you have had here, I did reply to your email in case 144389, you are not the only person who does not seem to be getting emails support is sending, not that this helps you, or us for that matter.
Adding support to excluding files and directories from being parsed is down as a very high priority case, but I don't currently have an estimate for when this will be done, but the sooner the better. Clearly though, until this is done, VA is not working well for you. |
| kornman00 |
Posted - Mar 29 2022 : 03:30:46 AM
A Renewals Representative "in the middle of forecasting to the management" emailed me asking if I intended to renew my license.
Since I was ghosted *in direct emails with support* about a fix for case 144389 in Feb 2021, I had to decline. I even reheated the email chain earlier this month (March 15th), before I was emailed by renewals, and yes: still silence.
VS2022 may be 64-bit, but that the extra memory still doesn't solve VAX burning *days* enumerating, let's see what I said on Feb 03 2021 in this thread:
quote:
As for how large my directory tree is, this is what I have mapped down from Perforce (this is going to include build artifacts and such, plus a chunk of game-ready data for a couple of games):
2,829,261 Files, 321,181 Folders
Actually, that should be about 200k+ files less today (since I personally removed that many files from the CMake project directory in the past year).
Even after it (presumably) finishes enumeration of 2 million+ files (the bulk of them, not source or even text files), things like "Find Symbol" or "Open File In Solution" provide no results. |
| feline |
Posted - Feb 05 2021 : 1:05:37 PM
Thank you for pointing me at the Visual Studio Code exclude files settings, that does indeed exclude files from both Visual Studio 2019 and Visual Studio 2017, so I have put in a feature request for VA to watch for, and respect this setting if found:
case=144389 |
| feline |
Posted - Feb 05 2021 : 09:38:56 AM
Knowing that you are using Unreal Engine adds in another factor. Visual Assist does have built in support for Unreal Engine, but so far I have only ever used this when working with a Visual Studio SLN file. To try and help this along, one of the new features in our latest build is:
Added support for reading configuration information from a new Visual Assist specific section of Unreal Engine solution files. (case=143077)
which isn't going to work if you are not using UE solution files, even if you are using a version of UE that adds this information to its solution files.
Part of our detection for the presence of Unreal Engine is a project inside your solution called "UE4", again solution and project based thinking. |
| kornman00 |
Posted - Feb 04 2021 : 6:08:24 PM
It's a fair question. I personally haven't tried to generate VS solutions from the CMake configs. For additional context, we use CMake+Ninja+UBT(UE4) for building the project. CMake+Ninja build some shared non-UE4 specific code and libraries, then CMake invokes UBT to build the actual UE4 project. I think there's also desire to setup the UE4 project so it is also fully CMake+Ninja and instead only invokes UHT beforehand, but I'm not sure of the if/when's of that.
For an example VSWorkspaceSettings.json based on what is in use, check this out: https://gist.github.com/kornman00/494971efa7916969c9a9bb10f1a74eb8
I've been informed that VS itself also supports the .vscode/settings.json file, which can exclude things via "files.exclude" https://code.visualstudio.com/docs/getstarted/settings. The VSWorkspaceSettings.json ExcludeItems setup is still a supported method in Visual Studio proper (seems things have changed since 2019), but "files.exclude" may be the more obvious solution to target instead (if not targeting both).
Thanks for the junction points workaround suggestion! I'll look at setting aside some time next week to give it a try |
| feline |
Posted - Feb 04 2021 : 08:02:21 AM
First, this may be a "silly" question, but if you are using CMake along side Visual Studio, have you considered using CMake to generate a Visual Studio solution, containing just the parts of the directory tree that you want to work with?
In the "VSWorkspaceSettings.json" file what syntax are you using to exclude a directory? I have a fairly large test case here, but so far trying to exclude directories from showing up in Solution Explorer isn't working for me. Given that all documentation for this file has been removed, it does suggest that this isn't a solid long term approach, but it does work for now.
As a temporary work around, would making a "fake" root folder be an option?
Assuming you have something like this:
C:\all_codeC:\all_code\old_game_oneC:\all_code\old_game_oneC:\all_code\live_code_oneC:\all_code\utility_code
and only want VA to pick up the live and utility code, create a new "fake" root directory:
C:\all_code\va_fake_root
and then use Windows file system junction points to add just the directory trees you want into the fake root directory. The junction point makes the directory appear to be in two different places at the same time, similar to a shortcut, and no extra storage is used. Unlike a shortcut, when you move into the junction directory, the directory really does appear to exist in this new location.
In this example, open a command prompt in the directory "C:\all_code\va_fake_root\" and run the commands:
mklink /J live_code_one "C:\all_code\live_code_one"
mklink /J utility_code "C:\all_code\utility_code"
this will cause the directory structure:
C:\all_code\va_fake_root\live_code_oneC:\all_code\va_fake_root\utility_code
to appear, along with all of the sub-directories. |
| feline |
Posted - Feb 03 2021 : 08:20:16 AM
It is "good" to see our efforts to limit our memory usage are working, but clearly they are not enough here. I am going to set up a much bigger stress test here, this really should not be happening. |
| kornman00 |
Posted - Feb 03 2021 : 12:11:48 AM
Thanks, feline! I was on VA 2393. I'm noticing now that 2399 is out so I'm in the process of getting that and will attempt to repro the issue again. Even though I had "Automatically check for new version" enabled, 2393 wasn't notifying me there was a new version for some reason.
On 2393, Open File in Solution was reporting 0 out of 0 files up to the point of crashing (I was opening it then closing periodically after VAX's 'Reading solution projects...' message box came up then went away).
As for how large my directory tree is, this is what I have mapped down from Perforce (this is going to include build artifacts and such, plus a chunk of game-ready data for a couple of games):
2,829,261 Files, 321,181 Folders
I see that VSWorkspaceSettings.json was removed from docs in this PR from June 2019: https://github.com/MicrosoftDocs/visualstudio-docs/pull/3499 . "The file was never intended to be user-editable"
I'm guessing the direction they intend to go is with CppProperties.json? https://docs.microsoft.com/en-us/cpp/build/open-folder-projects-cpp?view=msvc-160
Although, VSWorkspaceSettings.json is called out in the Workspace settings docs too (which I'm guessing is how VAX would try and read "ExcludedItems", if that's the fix) https://docs.microsoft.com/en-us/visualstudio/extensibility/workspaces?view=vs-2019
I also need to see if setting a "files.exclude" in a .vs\settings.json file also gets picked up (that may also be the direction they're wanting to go, to align with VS Code?) https://code.visualstudio.com/docs/getstarted/settings.
Okay, I got 2399 setup and running while typing this up. It took a lot longer to crash (5-6mins, instead of <2mins maybe?). In Task Manager before it crashed (while I was writing this), I was seeing the overall VS2019 memory sitting around 3200MB (and counting) while the Visual Studio subnode itself is 2980MB (and counting). It was also heavily exercising disk IO that entire time, probably trying to process/parse nearly 3mil files (of course, not all of them source or even text). |
| feline |
Posted - Feb 01 2021 : 05:49:22 AM
This does indeed sound like an out of memory problem. Which version of VA are you using? VA 2393 introduced several changes to try and reduce our memory usage with large solutions, so if you are using an older version it is worth trying the current version to see if any of these changes help.
http://www.wholetomato.com/downloads/default.asp
Are you able to load this directory / CMake project with VA enabled long enough to use VA's Open File in Solution dialog to get a count of the number of files VA thinks are in the "solution"? I have a large test solution, an actual .SLN file, that holds 94,000 files, but since this uses a SLN file, and it probably does not have as complex a directory structure, I am not sure how well it compares to your situation.
I am also wondering if this is anywhere near the size of your directory tree.
VA's Open File in Solution dialog not respecting the ExcludedItems setting defined in VSWorkspaceSettings.json is a known bug:
case=141644
so it makes sense that VA is doing some parsing of all of these files and directories, exactly as the log suggests. |
|
|
