This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | Class leak in build system | ||
---|---|---|---|
Product: | projects | Reporter: | _ gtzabari <gtzabari> |
Component: | Ant | Assignee: | Petr Nejedly <pnejedly> |
Status: | CLOSED FIXED | ||
Severity: | blocker | CC: | issues, jglick, pnejedly, rkubacki |
Priority: | P3 | Keywords: | PERFORMANCE |
Version: | 4.x | ||
Hardware: | PC | ||
OS: | Windows XP | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: |
build.xml causing problems
Updated build.xml |
Description
_ gtzabari
2004-07-25 06:50:58 UTC
No this message is not related to described problem. If you still have the problem and can reproduce it create a couple of thread dumps and describe some steps how to reproduce it (+HW/SW config) Radim, Issue seems to be related to a pretty large memory leak in the build engine. That is, every time the project is build Netbeans leaks more memory until around 20 builds later it allocates over 200MB of RAM and goes into an endless loop trying to garbage collect. I assume this is because Netbeans caps its own memory at around the 200MB mark. I've reopened the issue. If this sounds like another issue you guys are currently investigating let me know, otherwise this is something you should definately investigate. Current limit for heap size is 160MB. (and of course you can set it to a larger value.) This means that every time when the VM cannot allocate space for a new object it runs GC and enlarges the heap (that is set to 24MB at the begining). This growth is not a leak yet. If you reach this size and the IDE can't reclaim the memory (you can try to manualy force GC by clicking on memory toolbar) then there is likely a leak. It can be usefull to check that the time is spent in GC using tool like jvmstat (http://developers.sun.com/dev/coolstuff/jvmstat/). Then it might be good to describe what project do you have (IDE project, custom build script, size, HW/SW config) so we can move further. Also you may have a big trouble if your JVM does not fit into your physical memory (size of program + data including heap and internal structures) and the OS starts to swap. Then heap management and OS swapping will be really in effective. Radim, I can still reproduce this issue in dev build 20040911800 but it seems specific to using Hibernate under Netbeans. I'll attach my build.xml for your examination. After building 10-20 times the heap grows to 200MB and cannot be garbage collected, even forcibly. Created attachment 17380 [details]
build.xml causing problems
If you need more information please let me know. This issue looks to me related to issue #43839 which was supposidly closed a month ago. Maybe it wasn't fully-fixed after all? Petr can you check what is leaking here? issue #43839 was fixed and fixed correctly. I can try to simulate your environment, but that means downloading and setting up xdoclet and so on. Can you please do a simple test yourself regarding class leaking? Start NB with -J-verbose:class (preferabely redirecting th eoutput to a file) and count class loads vs. class unloads for running your script/performing gc. Hmm, I've reproduced the problem here using your (modified) build.xml :-( Heap usage is OK, but perm generation goes through roof. Investigating more. Seems like the beans Introspector is the culprit here. I found two classes (log4j.Layout and PatternLayout) root-referenced from the java.beans.Introspector's methods cache. OK, if you track it down for sure please (1) file a priority JDK bug, (2) reassign to me to implement yet another Ant module hack to clear the cache. :-) OK, the problem is that the author of java.beans.Introspector don't know how to use WeakHashMap. The workaround would be to call Introspector.flush() after finishing a build script. The fix would be (on the JDK side) to use also weak values, not only weak keys. I've just also checked the surces of the JDK1.5 and it seems that one of the leak paths (through Introspector.declaredMethodCache) was fixed, but the other path (through Introspector.beanInfoCache) wasn't. Worth a JDK bug and also a workaround on our side. OK, thanks for the investigation. > OK, if you track it down ...
Sure. I've patched the Introspector to use WeakRefs and it doesn't
leak anymore.
Petr, Great work! Can you also post a link to the BugParade issue after you file it? I want to track this one. I filed JDK bug 5102804. Will appear on BugParade in a few days, I presume. Seems that the problem only occurs if the introspected class has an explicit BeanInfo. Will just try calling Introspector.flushCaches() after every Ant run. Unfortunate, since some caching is legitimate and helpful for performance. committed Up-To-Date 1.23 ant/src-bridge/org/apache/tools/ant/module/bridge/impl/BridgeImpl.java Reopening. I can still reproduce the problem in dev build 200409201800. In which JDK? JDK 1.5 RC OK Petr could you recheck it? Perhaps the fix did not do what it is supposed to, or perhaps there is a different leak... > I can still reproduce the problem in dev build 200409201800
With the same build.xml or have you added some other tool in the mean
time? Please note that this class of problems is specific to used
taskdefs.
Petr, I'm going to attach my updated build.xml. I doubt much has changed within it. I am pretty sure I have not added any new taskdefs. Created attachment 17896 [details]
Updated build.xml
Seeems that I can reproduce some leak in the newest build under JDK1.5. It may be different, but certainly leads to OOME. Can't reproduce with the FCS of JDK1.5. The problem I was seeing was already fixed leak in ZipFile. Try installing newest JDK and if the problem persists, checke whether it is really this problem by using jstat -class <pid> and jstat -gcpermcapacity Look, I'm seeing a definate leak here. I'm trying to use jstat but the numbers mean nothing to me. How do I know if this is the same issue or some other leak? Either way, after about 10-20 rebuilds of my project I run out of memory and must restart Netbeans. Often I find out I am out of memory too late and Netbeans hangs with major cpu usage and must be terminated which sometimes leads to data loss. All very upsetting :) I'd like to reopen this issue if possible until this is resolved. jvmstat -class shows the number of loaded and unloaded classes. If you repetitively build your sources, both number should grow by about the same amount, so the number of classes in memory should be constant (and not much over 10000 after long usage) Note: class unloading is asynchronous during gc, so use gc action before checking number of unloaded classes. If the number of classes in memory grows repeatedly during builds, it is a class leak, otherwise it is something different, see e.g. issue 50454. This issue was solved long time ago. Because nobody has reopened it neither added comments, we are verifying/closing it now. If you are still able to reproduce the problem, please reopen. Thanks in advance. |