| Author |
 Topic  |
|
|
rikonen
New Member
7 Posts |
 Posted - Oct 12 2009 : 09:40:07 AM
|
When Visual Assist X is installed the memory consumption of Visual Studio keeps on growing while coding, quickly hitting about 1.5 gigabytes, at which point VS stops responding properly and needs to be killed in order to get it working again. User ColinBell described similar sounding issue in topic http://forum.wholetomato.com/forum/topic.asp?TOPIC_ID=8043 but that didn't seem to contain any resolution. The memory consumption often hits 1.5 gigabytes after writing only few hundred lines of code so I often need to keep on killing Visual Studio after just half an hour of use. The memory consumption only seems to be growing when actually doing something (i.e. coding), just keeping Visual Studio open does not cause any problems.
The version of Visual Studio we're using is 2005 SP1. Our main solution is relatively large, about 70 projects with 4,000 files altogether and about million lines of code. However, the size of the project cannot be the sole cause of the problem because we have other people using Visual Assist X and they are not experiencing the same problem, and I didn't experience the problem myself either before switching OS. Our solution uses mostly plain Win32 API but also a bit of MFC and some miscellaneous include files from various libraries like OpenSSL and such. I don't have any AV installed, all VS patches have been installed and there aren't any applications installed that would seem likely to conflict with Visual Assist X.
So far what has happened is this:
I originally had Windows XP Professional x86 with Visual Assist X purchased in June 2006. Not sure about the exact version. I didn't experience any problems in this environment. We purchased an upgrade to the latest version in January 2008 and still no problems. Again, I'm note sure what the version was exactly but presumably build 1626.
This summer I needed to switch computer and the new one came with Windows Vista Ultimate x86. I installed all necessary VS patches and the latest version of Visual Assist X that we were allowed. Unfortunately I don't have that exact version anymore but I'm guessing it was build 1711. After this I immediately started experiencing the problem: after writing any non-trivial amount of code Visual Studio would start consuming so much memory that Windows started suggesting closing other applications and eventually the memory consumption went up to 1.5 gigabytes and VS stopped practically working and needed to be killed.
Few weeks ago I switched to Windows 7 Ultimate x64 (clean install) and installed the latest version of Visual Assist X (we had to purchase yet another upgrade in hope that the problem would be resolved). That version was 1736. I immediately hit the same memory leak problem. I then tried uninstalling Visual Assist X and the problem went away. Just to ensure the problem wasn't related to my old personal settings I tried re-installing Visual Assist X build 1738 without restoring my old settings and that didn't work any better so the problem wouldn't seem to be caused by my setting changes.
Is there any workaround? Any additional information I could provide to you? I have a colleague who's running Windows 7 Ultimate x64 with the same version of Visual Assist X and Visual Studio is consuming a lot of memory for him as well but it doesn't seem to be growing without bounds and isn't really causing any problems; it maxes at 1 gigabyte and doesn't seem to go beyond that. Another coder is using Windows XP and doesn't have any problems. So, this might be something Vista/7 related although there has to be something else to this as well because other people haven't had quite as bad problems with Vista or 7.
To answer some questions presented in topic 8043, I haven't seen any "VAX: Parsing ..." text flashing in my status bar. I haven't tried rebuilding symbol database but I wouldn't think that can have an effect given that the problem reproduced in a clean system.
Any suggestions are welcome. The current situation is quite frustrating because we have paid fair amount for the software and also for upgrades that we have hoped would resolve the situation and right now the application is almost unusable for me. |
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 12 2009 : 10:45:30 AM
|
I have seen a few reports of this problem over time, and they have all been caused by a corrupt VA symbol database. Normally though it is a one off problem. Tell VA to rebuild its symbol database, or manually delete it with all IDE instances closed, reload the IDE, let VA rebuild its symbol database, and the problem should be fixed.
It sounds like you are getting the same problem on a regular basis, which should not happen.
Can you start by pressing the button:
VA Options -> Performance -> Rebuild symbol databases
and restarting your IDE please. Does this help? If so, for how long?
If something is encouraging the symbol database to become corrupted then we need to try and find this trigger so we can hopefully fix it. |
zen is the art of being at one with the two'ness |
 |
|
|
rikonen
New Member
7 Posts |
Posted - Oct 14 2009 : 06:01:53 AM
|
I rebuilt the symbol database and also cleared the cache and then restarted VS. The initial msdev.exe memory consumption after restart was about 180 MB. I've been doing some bug fixing so the amount of code I've written since has been quite minimal, maybe 40 lines or so. Thus far the memory consumption has jumped up to 380 megabytes. I haven't done anything special in addition to opening the solution; just done incremental build maybe 10 or so times and opened (and then closed) roughly 50 different cpp/h files.
So as of yet I don't know how high the memory consumption is going to get but all the signs so far are identical to the typical case where the consumption shoots up to 1.5 gigs after few hundred more lines of code and I'll have to kill VS. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 14 2009 : 08:42:12 AM
|
I was afraid of this, given how many times you have seen the problem.
Can you try turning off:
VA Options -> Performance -> Keep symbols in memory for fast response after Alt+Tab
VA Options -> Performance -> watch for externally modified files and reparse when necessary
and see if this makes any difference?
Is there any obvious difference between your system and the systems of other people who do not have this problem?
Different anti-virus? Different IDE plugin's installed?
Can you try disabling the IDE's intellisense scanner as explained here:
http://docs.wholetomato.com?W133
I doubt that the IDE's intellisense scanner is a problem, but disabling it means there is one less thing going on while VA is parsing your solution. |
zen is the art of being at one with the two'ness |
|
|
|
RHunt
New Member
7 Posts |
Posted - Oct 14 2009 : 09:45:24 AM
|
| A complete shot in the dark but many builds ago (over a year) I had similar issues and one thing that helped was turning off hyperthreading in the bios. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 14 2009 : 8:18:55 PM
|
Yes, I remember this as well. There was a problem that would sometimes show up on machines with 2 or more processors, but that has been fixed for a couple of years now.
Still, it is a good point. rikonen are there any major differences in the type of machines and their configuration, yours and your colleges? |
zen is the art of being at one with the two'ness |
|
|
|
Michal Puczynski
Ketchup Master
Poland
85 Posts |
Posted - Oct 15 2009 : 06:38:46 AM
|
Haven't seen that since 1624, although I am using source code with over 3000 files.
The build just after 1624 solved that. On 1624 and earlier builds my studio was easily reaching 2GB and even more when I switched my devenv.exe to have "/3GB aware" flag. |
|
|
|
rikonen
New Member
7 Posts |
Posted - Oct 15 2009 : 10:13:20 AM
|
I tried turning of the "Keep symbols in memory..." setting. The "Watch for externally modified ..." setting wasn't enabled. These didn't seem to make any difference.
I observed the memory usage in a bit more detail and it seems that Visual Assist X is leaking roughly 1 megabyte of memory per every single letter I type. However, this only happens if I have the "Include suggestions in listboxes" setting enabled. If I turn off this setting the memory leak seems to go away. This isn't very good workaround because I really would like to use this feature but this might help figuring out the cause of the problem. You can also imagine this is quite obtrusive since I only need to write some 1500 letters to get to the point where things fail completely.
I haven't yet tried disabling intellisense. I can try that later when I get a moment.
None of the machines we're using Visual Assist X on have AV software installed. None of them have any other plug-ins, just Visual Assist X. Other software differs a bit but there aren't any applications that would hook themselves very deeply into the OS.
The hardware on the machines is very much different. My machine is Lenovo Thinkpad T61p, the other machines are Dell Optiplex desktops. I don't have exact specs for the other machines but they have some multicore Intel chip at least but probably none of the components between any of the machines are exactly the same. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 15 2009 : 12:36:45 PM
|
Unless there is some major and obvious difference between the machines then I am not sure this matters.
Can you try turning on both suggestions and VA logging, type a little bit, and then turn off both suggestions and logging. If you could also note down the memory used at the start and end of this test as well that would be helpful.
Please see this FAQ for details of turning on VA's logging, and sending us the log files
http://docs.wholetomato.com?W305
Suggestions are not supposed to cause a memory leak. I am just thinking, perhaps the "type" of listbox is a factor. Are you running anything like window blinds, or any "clever" utilities that came with your graphics card to make windows transparent, etc? I am wondering if something is hooking into our listbox windows and causing this problem. |
zen is the art of being at one with the two'ness |
|
|
|
rikonen
New Member
7 Posts |
Posted - Oct 16 2009 : 09:06:07 AM
|
I haven't installed any drivers that Windows didn't automatically install except Xerox global print driver but that shouldn't be relevant because the problem reproduced before installing it. And then there are of course the drivers installed by Virtual PC. I'll attach a screenshot that shows all installed and running applications along with the log files when sending separate request through http://www.wholetomato.com/support/contact.asp
Below is the list of steps that I took. This time memory consumption seemed to increase somewhat less than yesterday. It might be related to the fact that I just restarted Visual Studio before starting the test and the leak maybe somehow increases over time. I also did close all other applications before starting the test.
14:01:30 Started Visual Studio and enabled VAX logging. devenv.exe memory usage 140 MB
14:03:30 Intellisense finished. devenv.exe memory usage 159 MB
14:05:20 Enabled "Include suggestions in listboxes" setting. devenv.exe memory usage 159 MB
14:05:xx - 14:06:50 Wrote "pMessage" 3 times and erased the word after every time. devenv.exe memory usage 176 MB
14:08:00 devenv.exe memory usage dropped to 166 MB on its own (I was writing down previous note and wasn't using it)
14:08:50 - 14:10:40 Wrote "pMessage" roughly 10 times and erased the word after every time. Initially the memory usage didn't grow but after writing the word a few times it started increasing, ending up to 182 MB
14:11:50 devenv.exe memory usage dropped to 178 MB on its own
14:12:30 Disabled "Include suggestions in listboxes" setting. devenv.exe memory usage 178 MB
14:12:40 - 14:13:20 Wrote "pMessage" roughly 10 times and erased the word after every time. devenv.exe memory usage 178 MB
14:14:50 Zipped va.log and VAssist.log
14:16:00 Zipped Startup.log, errors.log did not exist
It was very clear that the memory leak was taking place every time the suggestions listbox was shown. The leak also took place even if the box was already visible and my typing didn't change its contents or cause any other visible change either. I tried a lot of different combination of various settings related to the listbox and what is shown in it but none of them seemed to make any difference; if the listbox was shown under any circumstance the memory leak always occurred, no matter how many or few entries there were and whether or not the entries changed while I typed more text. The same leak took place also when the listbox showing class members was shown.
The memory leaked during my test wasn't especially large but the problem definitely is still there because after continuing normal coding for just 15 minutes after finishing the test the memory usage was already 240 MB. |
|
|
|
accord
Whole Tomato Software
United Kingdom
3287 Posts |
Posted - Oct 16 2009 : 4:43:12 PM
|
| We got the log file, and will look into it, thank you. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 17 2009 : 09:23:46 AM
|
Unfortunately the log files are not helping much so far.
Which programming language or languages are you working in?
Can you try disabling the IDE's intellisense parser and see if this makes any difference?
If this does not help, can you try downloading and running Process Explorer please:
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
then select the IDE process in the list and press CTRL-A. This should save out a text file with details of the process, including all of the dll's in use by the IDE. We have a second user reporting a very similar memory leak, so I am wondering if you will have any unusual dll's in common with him that might be a factor.
Can you also try reinstalling / downgrading to VA 1626 and see if this fixes the problem? We are trying to find out if the problem only shows up with newer versions of VA, or if there is something about your system that triggers this problem in every version of VA.
You can download older builds of VA from this page:
http://www.wholetomato.com/support/history.asp |
zen is the art of being at one with the two'ness |
|
|
|
rikonen
New Member
7 Posts |
Posted - Oct 19 2009 : 02:02:02 AM
|
We're doing C++ development, mostly Win32 API as mentioned earlier but also a bit of other stuff like MFC and some third-party libraries.
I disabled Intellisense by renaming feacp.dll. That had no effect.
The requested Process Explorer files have been sent to you.
I tried version 1626 and some other old versions and here are the results:
1626 - OK, no memory leak
1646 - leaks memory
1640 - leaks memory
1632 - leaks memory
1626 - installed again to validate result, still no leak observed
So 1626 is clearly the last good version and 1632 introduced the leak. Hope this helps. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 20 2009 : 11:57:40 AM
|
Thank you for the clear results, this proves something is going on, and it tells us when it started. Now we just need to try and figure out what is causing it.
Do you have deep macro parsing enabled, as described here:
http://docs.wholetomato.com?W363
this might be a factor.
If you install VA 1738 and make a new, default C++ solution, do you still see the memory leak behaviour as you type? It is possible there is something about your main solution that is a factor here.
In your main solution, can you try making / opening a new blank .cpp file and then typing in it? If the file is blank when you start and you just type some general / random code do you still see the memory leak problem?
Having suggestions enabled and tying will cause VA to use memory, since it has to hold symbols to suggest in memory, but the memory usage should only increase so far, and over time the memory should be freed. So seeing memory usage drift up a bit does not prove you have seen the problem, but clearly memory usage should never hit 1.5gig and stop the IDE from working.
I am looking at the process explorer files now, thank you for these:
case=33968 |
zen is the art of being at one with the two'ness |
|
|
|
rikonen
New Member
7 Posts |
Posted - Oct 21 2009 : 07:35:01 AM
|
I haven't changed any settings so macro parsing is in whatever state it is by default. I checked registry and HKCU\\Software\\Whole Tomato\\Visual Assist X\\LimitMacro didn't seem to exist. HKCU\\Software\\Whole Tomato\\Visual Assist X\\VANet8\\LimitMacroParsing was present and the value was 01.
My latest leak test consisted of writing and erasing a single word over and over again for 30 seconds and then letting VS sit idle for bit over a minute. With all the problematic versions I could easily make the memory consumption increase by 40 megabytes during the 30 seconds I kept writing and the consumption did not come down at all during the 1 minute period I let VS sit idle. Given that there were only a few dozen suggestions that were displayed and they stayed the same all the time I highly doubt Visual Assist X required that much memory to cache the suggestions, especially given that with version 1626 I got the very same suggestions but memory consumption didn't increase at all.
I just switched to develop a Java based product for a while and don't have time to run tests with version 1738 right now. I can get back to those later when I have some time to spare if still necessary. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 21 2009 : 11:07:58 AM
|
When you have the time can you try making sure all instances of the IDE are closed, and then use regedit to set the following value:
HKEY_CURRENT_USER \\ Software \\ Whole Tomato \\ Visual Assist X \\ VANet8 \\ ListboxFlags = ffffffff
and see if this has any effect on the memory leak?
There were some changes to listboxes after VA 1626, so this might have an effect. |
zen is the art of being at one with the two'ness |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 28 2009 : 2:02:34 PM
|
So far we cannot find any obvious memory leak inside VA its self. Can you please run Process Explorer, and then right click on the column headings and go into "Select Columns" and turn on the following columns in the "Process Memory" tab:
Working Set Size - this is the total memory being used
GDI Objects
USER Objects
Can you then watch how these three numbers change as you reproduce the problem? The memory leak should show up in the Working Set Size column, but are you also seeing GDI and USER objects being leaked?
If GDI or USER objects are being leaked it is possible the problem is with your graphics card drivers. Is the same graphics card being used on machines with the problem and machines that do not have this problem?
Are there newer graphics card drivers available for your machine? If so does installing them make any difference?
If GDI and USER objects are not being leaked can you please download and run the VMMap utility from sysinternals:
http://technet.microsoft.com/en-us/sysinternals/dd535533.aspx
When you run the program it will present you with a list of running processes, please select the IDE process. Then when the information is present save out (CTRL-S) a .mmp file. Can you then reproduce the memory leak, refresh the memory information with F5 and save out a second .mmp file. Hopefully the difference in these two files will give us some clues. |
zen is the art of being at one with the two'ness |
|
|
|
rikonen
New Member
7 Posts |
Posted - Oct 30 2009 : 09:51:40 AM
|
I uninstalled the old version of Visual Assist X and installed 1738 to make the tests you suggested.
I created an empty Win32 project, added one file (main.cpp), included windows.h, declared WinMain. After this I saved VMMap file devenv_before.mmp. Then I typed for 60 seconds. The suggestions listbox was continuously shown (I was typing M -> MessageBox -> M -> MessageBox ...). During the 60 second period Working Set increased by roughly 90 megabytes, USER and GDI objects showed no noticeable change. I then waited a while to see that the memory consumption stays high and repeated the test. This time Working Set increased by 200 megabytes while USER and GDI objects still stayed stable. At this point I saved devenv_after.mmp. I still performed the test for the third time and Working Set increased by another 230 megabytes and GDI and User objects still shows no change. This state was saved into devenv_final.mmp. Based on this it looks like the leak might be increasing over time. The solution being used doesn't appear to make any difference.
The other machines where we're using Visual Assist X are desktop machines while I'm using a laptop. I don't know what graphics cards they have exactly but they almost certainly don't have the same one. My laptop has NVIDIA Quadro FX 570M. The driver date is 5th of September this year so they are quite recent -- this is what Windows 7 installed automatically. I tried getting the latest driver from NVIDIA website but for some reason they refused to install claiming that no supported hardware was found although I'm sure I picket the right driver package. Go figure.
I also tried setting ListboxFlags to ffffffff. That had no effect.
I'll email the mmp files to you shortly. |
|
|
|
feline
Whole Tomato Software
United Kingdom
19021 Posts |
Posted - Oct 30 2009 : 8:58:52 PM
|
| Thank you for the files, we are having a look at them. |
zen is the art of being at one with the two'ness |
|
|
|
sean
Whole Tomato Software
USA
2817 Posts |
Posted - Nov 11 2009 : 4:06:19 PM
|
| We took this up via email. case=35543 |
|
|
|
tandr
Senior Member
43 Posts |
Posted - Nov 19 2009 : 12:24:09 PM
|
Folks, may we get an update on this issue? Unfortunately, I have to confirm the same issue -- it does leak. So restart VS every morning due to the fact that eats more then half gig memory, and become dog slow.
Thing is that if I run stress test overnight with application in debug mode, it should not touch VA, but since there a leak, it affects my tests.
We have Purify license, so I can try to plug it to VS itself (not sure if it will work though, but we can try) and see where it is leaking. I will need some .pdb for VAX dlls files in order to get any useful stack traces out for all allocated objects. Please let me know if you are interested.
Regards,
t.
EDIT: sorry, info attached
VA_X.dll file version 10.5.1738.0 built 2009.10.01
Licensed to:
VA X: XXX Support ends 2010.02.04
DevEnv.exe version 9.0.30729.1
msenv.dll version 9.0.30729.1
Font: Consolas 11(Pixels)
Comctl32.dll version 6.0.2900.5512
Windows XP 5.1 Build 2600 Service Pack 3
2 processors (x86)
Platform: Win32
Stable Includes:
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\include;
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\atlmfc\\include;
D:\\Program Files\\Microsoft SDKs\\Windows\\v6.1\\include;
D:\\Program Files\\Microsoft SDKs\\Windows\\v6.1\\include;
E:\\1Repos\\svn.boost.org\\svn\\boost\\tags\\release\\Boost_1_40_0;
Other Includes:
Stable Source Directories:
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\atlmfc\\src\\mfc;
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\atlmfc\\src\\mfcm;
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\atlmfc\\src\\atl;
D:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\crt\\src;
|
Edited by - tandr on Nov 19 2009 12:30:36 PM |
|
|
|
sean
Whole Tomato Software
USA
2817 Posts |
Posted - Nov 19 2009 : 1:28:12 PM
|
We don't have a fix but do have a workaround that worked for rikonen. The leak appears to have some sort of system dependency (drivers, utilities, phase of the moon?), but for rikonen we isolated some behaviors to disable that fixed the leak he was seeing. Please use the contact form, referencing this thread, to get an updated dll and instructions:
http://www.wholetomato.com/support/contact.asp |
|
|
|
support
Whole Tomato Software
5566 Posts |
Posted - Jan 31 2010 : 1:23:31 PM
|
| case=35543 is fixed in build 1810 |
Whole Tomato Software, Inc. |
|
|
| |
Topic |
|
Visual Assist X leaking huge amounts of memory
