| T O P I C R E V I E W |
| OmegaNemesis28 |
Posted - Mar 03 2022 : 2:59:34 PM
I have Unreal Engine 5 source and a fairly massive project for it.
I'm on Visual Studio Pro 2019 16.11.9 (Nov 2021 version) and Visual Assist 2021.5
VA_X.dll file version 10.9.2440.0 built 2021.11.17
DevEnv.exe version 16.11.32106.194 Professional
msenv.dll version 16.0.32002.118
Comctl32.dll version 6.10.19041.1110
Windows 10
64 processors (x86-64, WOW64)
Language info: 1252, 0x409
I am using an AMD Threadripper PRO 3995WX 64 Core CPU. I have 128 threads.
When Visual Assist needs to index lots of new files or a whole new solution (creating a new Unreal project or something from source), or say if I change the specific "Unreal Engine Index Unreal Engine plugins" setting from "None" to "All", it queues up all those files to index and will start processing them. The bottom of my Visual Studio will show "VA: Parsing <file>".
What I've found is that even if I set "Limit maximum number of concurrent threads" to 32 out of the 128 I have, my Windows apps including Unreal Engine 5 especially, will hang frequently on simple operations. For example, in Unreal Engine 5 if I open up a blueprint file in the editor executable running from binaries I've previously built from VS (not debugging), it will freeze basically the whole OS intermittently and can take 10+ minutes to open that blueprint file. But if I shutdown or disable VSX, it will instantly work. Even Chrome itself will stall out frequently, especially plugins that seem to be using cache files.
I am not sure if this is some kind of IO bound issue or contention somewhere on the OS level? Even if I do not limit concurrent threads in VSX, my disk usage is minimal! The Nvme SSD is at a measly 5% utilization while VSX parses. And my Threadripper CPU will never reach above 40% utilization even with Unreal Engine open simultaneously.
There are quite many files to parse in Unreal, but even at half or max threads, VSX has been indexing for over 5 hours when I enable "All" plugins parsing. I've since changed it to 2 threads and the OS and UE itself feels more responsive, but I do not know how much longer it will take to parse files because I've limited it to 2 threads.
For completeness these are my settings right now:
-Enable multithreaded parsing
--Limit max # of concurrent threads 2
--Thread priority Normal
-Disable Watch for externally modified files
-Enable Parse All files when opening a project
-Disable use alternate dir for symbols
-Disable logging
-Enable AutoRecovery
-Enable support for UE4 Always
-Enable Enhanced Syntax Coloring in UE4 macros
-Suggest VA snippets inside UE4 macros
-Index UE plugins: All
-Index generated code
|
| 8 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Mar 16 2022 : 11:36:58 AM
Thank you for taking the time to check this. Seeing system interrupts so high is rather worrying, and isn't something that is happening here. I had to look this up, just to see what might be going on:
https://www.howtogeek.com/271400/what-is-the-system-interrupts-process-and-why-is-it-running-on-my-pc/
this page, if it applies here, suggests this is a hardware effect.
If you close Visual Studio does this drop the CPU usage from system interrupts back to normal, tiny levels, or does it remain high?
I have not been able to find a reliable pattern yet for the performance problems I can reproduce here, but junction points are certainly not a guaranteed trigger for me.
Have you used earlier versions of Visual Assist on your system? I am asking because I would like you to try a test profile, so we can try and find out if the trigger is somehow related to your default profile, and any other extensions you might have installed. But this test will use an older version of VA at first, but we can move to the current version if that is important.
Can you please close all instances of the IDE and then run the command:
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\devenv.exe" /RootSuffix VATest
this will create and load a new, default profile for Visual Studio, so you will be asked which basic settings you want to use, and you should only have the standard, default IDE extensions installed. Visual Assist will not be installed into this profile, so you will need to go into the dialog:
IDE Extensions menu -> Manage Extensions
and install Visual Assist from the online extension store, to install VA for this test profile.
To load your normal, default profile just load the IDE normally. To return to this test profile again, pass the /RootSuffix command line switch when loading the IDE. You can run both profiles at the same time, next to each other. In VS2019 and VS2022 the profile name will be shown just under the close button, in the top right hand corner of the main IDE window. If you export your IDE settings from your main profile you can them import them into the test profile.
Do you see the same performance problem with this test profile? |
| OmegaNemesis28 |
Posted - Mar 13 2022 : 8:10:03 PM
quote:
Originally posted by OmegaNemesis28
Apologies, havent gotten around to testing it yet. But something I noticed that may (or not) be related is that even if there is no more parsing files and VSX works as intended, if I reboot Visual Studio, sometimes VSX will decide to start parsing \Engine\ files again. Which do not change content and do not move either. That's Unreal Engine source files which, maybe I'll modify one day, but they're definitely not being modified now. I find this peculiar.
The bottom bar will flash "VA: Parsing C:\depot\<project>\Engine\..." files. I see "\Plugins\" in particular show up a lot again despite no changes. Perhaps there is a bug there with indexing? I can't use VSX while it is doing this unfortunately, such as find references, so every reboot of Visual Studio is usually about a 10-20 minute wait before VSX becomes responsive again.
I will try to test without the junction soon.
Ok ive went and started a new blank UE5 project with brand new preview 2 code all on my D: drive. No junctions. Unfortunately, same/similar results when Visual Studio and VSX is indexing. With no limits on the threads, Windows stalls out frequently. Whether I'm browsing files in explorer or using programs like Notepad or Chrome. Task Manager does not indicate many threads are under high use, but yet it feels like it.
I've noticed that on extra long hangs, Task Manager will only show "System interrupts" using the CPU heavily. When it finally is responsive again, I see Visual Studio using most the of the CPU again and it's only roughly 20% with under less than 1GB of memory and hardly any disk usage. I'm seeing it bounce between 0.5 to 1 MB/s on my SSD. "System" will also show about 1MB/s sometimes. I definitely do more for other things without the hanging.
And of course the second I shutdown Visual Studio everything is fine again. And if I reboot it, the issue will happen as long as Visual Studio with VSX is enabled. If I disable VSX, restart Visual Studio, it will not occur anymore. Or if I enable VSX, limit to 3 threads, and reboot Visual Studio. This is okay for the time being, I just can't measure how much slower it is to fully index.
|
| OmegaNemesis28 |
Posted - Mar 11 2022 : 4:22:55 PM
Apologies, havent gotten around to testing it yet. But something I noticed that may (or not) be related is that even if there is no more parsing files and VSX works as intended, if I reboot Visual Studio, sometimes VSX will decide to start parsing \Engine\ files again. Which do not change content and do not move either. That's Unreal Engine source files which, maybe I'll modify one day, but they're definitely not being modified now. I find this peculiar.
The bottom bar will flash "VA: Parsing C:\depot\<project>\Engine\..." files. I see "\Plugins\" in particular show up a lot again despite no changes. Perhaps there is a bug there with indexing? I can't use VSX while it is doing this unfortunately, such as find references, so every reboot of Visual Studio is usually about a 10-20 minute wait before VSX becomes responsive again.
I will try to test without the junction soon. |
| feline |
Posted - Mar 04 2022 : 5:17:53 PM
First test results here suggest that GPT isn't the problem, which is reassuring. Also suggest that a junction point, by its self, isn't the problem.
Are you able to test this without using the junction point, by running Unreal Engine editor via its "real" path rather than the junction point path?
If you have the time, can you please try creating a new, default Unreal Engine project, and see if you get the same parsing problem there or not?
Can you please try running Windows Performance Monitor and see if this gives you any clues as to where the bottleneck is? It shouldn't be CPU or IO by the sounds of it, but is the IDE using a lot of RAM? |
| OmegaNemesis28 |
Posted - Mar 04 2022 : 4:27:46 PM
Both disks are flagged as GPT |
| feline |
Posted - Mar 04 2022 : 10:45:07 AM
Can you please open a windows command prompt window, running as Admin, and then run the command:
diskpart
this will change the command prompt to "DISKPART>". Now run the command:
list disk
do either, or both of your hard drives have a * in the "Gpt" column?
Exit a couple of times will close the command prompt window normally.
On the machine where I am currently studying a terrible performance problem I have a 2 non Gpt hard drives, and 1 Gpt hard drive. The problem is happening with a junction point on the GPT hard drive, pointing to a non GPT hard drive.
I don't yet know if this is a factor or not, but its what I am currently testing, since its new, and the problem has just appeared.
I have come across a report that scanning files via a junction point can introduce a massive performance problem, but until now I have never seen this happen myself. The report mentioning the problem didn't provide any extra details beyond "junction point bad", along with benchmarks proving it was bad. The bench marks had nothing to do with the IDE or VA either, but the description in that report does match what we are both seeing. |
| OmegaNemesis28 |
Posted - Mar 04 2022 : 06:49:22 AM
It is from GitHub and indeed, I have a junction from C:\project to D:\project for source code and the engine.
Both drives are identical SSDs to be clear too. |
| feline |
Posted - Mar 04 2022 : 06:44:07 AM
This sounds similar to a problem I have just run into here, which I am currently trying to find the trigger for.
Where, which drive and path, do you have the UE source code installed?
Have you used junction points in the Windows file system, to make the UE source code appear to be somewhere else? This is a standard workaround I suggest if UE source code is not installed under:
"C:\Program Files\"
Did you get the UE source code via the Epic Launcher or did you download it from github? |
|
|
