This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 46532 - Class leak in build system
Summary: Class leak in build system
Status: CLOSED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Ant (show other bugs)
Version: 4.x
Hardware: PC Windows XP
: P3 blocker (vote)
Assignee: Petr Nejedly
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2004-07-25 06:50 UTC by _ gtzabari
Modified: 2006-03-24 09:58 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
build.xml causing problems (5.05 KB, text/plain)
2004-09-06 02:33 UTC, _ gtzabari
Details
Updated build.xml (5.73 KB, text/plain)
2004-09-27 19:27 UTC, _ gtzabari
Details

Note You need to log in before you can comment on or make changes to this bug.
Description _ gtzabari 2004-07-25 06:50:58 UTC
dev build 200407151800
JDK 5.0 beta3

  I get the following in messages.log:

INFORMATIONAL *********** Exception occurred
************ at 1:40 AM on Jul 25, 2004
[catch]java.lang.IllegalStateException: Warning -
the node "AddImage2.java"
[org.netbeans.modules.java.JavaNode] is trying to
fire the property modified which is not included
in its property sets. This is illegal. See IZ
#31413 for details.
	at
org.openide.nodes.Node.firePropertyChange(Node.java:828)
	at
org.openide.loaders.DataNode.access$700(DataNode.java:38)
	at
org.openide.loaders.DataNode$3.run(DataNode.java:578)
	at
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
	at
java.awt.EventQueue.dispatchEvent(EventQueue.java:461)
	at
java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:234)
	at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:163)
	at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:157)
	at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:149)
	at
java.awt.EventDispatchThread.run(EventDispatchThread.java:110)


   and memory usage peaks at around 200MB and
Netbeans goes into some endless loop (cpu usage is
high and never ends). I'm not sure if this message
is specifically related to the endless loop but I
suspect it might be.
Comment 1 _ rkubacki 2004-07-26 10:32:36 UTC
No this message is not related to described problem. 

If you still have the problem and can reproduce it create a couple of
thread dumps and describe some steps how to reproduce it (+HW/SW config)
Comment 2 _ gtzabari 2004-07-27 08:25:55 UTC
Radim,

  Issue seems to be related to a pretty large memory leak in the build
engine. That is, every time the project is build Netbeans leaks more
memory until around 20 builds later it allocates over 200MB of RAM and
goes into an endless loop trying to garbage collect. I assume this is
because Netbeans caps its own memory at around the 200MB mark. I've
reopened the issue. If this sounds like another issue you guys are
currently investigating let me know, otherwise this is something you
should definately investigate.
Comment 3 _ rkubacki 2004-08-02 14:15:41 UTC
Current limit for heap size is 160MB. (and of course you can set it to
a larger value.) This means that every time when the VM cannot
allocate space for a new object it runs GC and enlarges the heap (that
is set to 24MB at the begining). This growth is not a leak yet.

If you reach this size and the IDE can't reclaim the memory (you can
try to manualy force GC by clicking on memory toolbar) then there is
likely a leak. It can be usefull to check that the time is spent in GC
using tool like jvmstat
(http://developers.sun.com/dev/coolstuff/jvmstat/). Then it might be
good to describe what project do you have (IDE project, custom build
script, size, HW/SW config) so we can move further.

Also you may have a big trouble if your JVM does not fit into your
physical memory (size of program + data including heap and internal
structures) and the OS starts to swap. Then heap management and OS
swapping will be really in effective.
Comment 4 _ gtzabari 2004-09-06 02:32:15 UTC
Radim,

I can still reproduce this issue in dev build 20040911800 but it seems
specific to using Hibernate under Netbeans. I'll attach my build.xml
for your examination. After building 10-20 times the heap grows to
200MB and cannot be garbage collected, even forcibly.
Comment 5 _ gtzabari 2004-09-06 02:33:16 UTC
Created attachment 17380 [details]
build.xml causing problems
Comment 6 _ gtzabari 2004-09-06 02:33:37 UTC
If you need more information please let me know.
Comment 7 _ gtzabari 2004-09-07 18:43:28 UTC
This issue looks to me related to issue #43839 which was supposidly
closed a month ago. Maybe it wasn't fully-fixed after all?
Comment 8 _ rkubacki 2004-09-09 07:55:06 UTC
Petr can you check what is leaking here?
Comment 9 Petr Nejedly 2004-09-09 14:09:46 UTC
issue #43839 was fixed and fixed correctly.
I can try to simulate your environment, but that means downloading and
setting up xdoclet and so on.
Can you please do a simple test yourself regarding class leaking?

Start NB with -J-verbose:class (preferabely redirecting th eoutput to
a file) and count class loads vs. class unloads for running your
script/performing gc.
Comment 10 Petr Nejedly 2004-09-09 15:59:48 UTC
Hmm, I've reproduced the problem here using your (modified) build.xml :-(

Heap usage is OK, but perm generation goes through roof.
Investigating more.
Comment 11 Petr Nejedly 2004-09-13 15:31:10 UTC
Seems like the beans Introspector is the culprit here.
I found two classes (log4j.Layout and PatternLayout) root-referenced
from the java.beans.Introspector's methods cache.
Comment 12 Jesse Glick 2004-09-13 15:49:46 UTC
OK, if you track it down for sure please (1) file a priority JDK bug,
(2) reassign to me to implement yet another Ant module hack to clear
the cache. :-)
Comment 13 Petr Nejedly 2004-09-13 15:53:45 UTC
OK, the problem is that the author of java.beans.Introspector don't
know how to use WeakHashMap.
The workaround would be to call Introspector.flush() after finishing a
build script.

The fix would be (on the JDK side) to use also weak values, not only
weak keys.

I've just also checked the surces of the JDK1.5 and it seems that one
of the leak paths (through Introspector.declaredMethodCache) was
fixed, but the other path (through Introspector.beanInfoCache) wasn't.

Worth a JDK bug and also a workaround on our side.
Comment 14 Jesse Glick 2004-09-13 15:54:50 UTC
OK, thanks for the investigation.
Comment 15 Petr Nejedly 2004-09-13 15:56:13 UTC
> OK, if you track it down ...
Sure. I've patched the Introspector to use WeakRefs and it doesn't
leak anymore.
Comment 16 _ gtzabari 2004-09-13 16:00:55 UTC
Petr,

Great work! Can you also post a link to the BugParade issue after you
file it? I want to track this one.
Comment 17 Jesse Glick 2004-09-15 23:16:08 UTC
I filed JDK bug 5102804. Will appear on BugParade in a few days, I
presume.

Seems that the problem only occurs if the introspected class has an
explicit BeanInfo.
Comment 18 Jesse Glick 2004-09-16 19:56:52 UTC
Will just try calling Introspector.flushCaches() after every Ant run.
Unfortunate, since some caching is legitimate and helpful for performance.
Comment 19 Jesse Glick 2004-09-17 03:41:36 UTC
committed     Up-To-Date  1.23       
ant/src-bridge/org/apache/tools/ant/module/bridge/impl/BridgeImpl.java
Comment 20 _ gtzabari 2004-09-27 05:15:47 UTC
Reopening.

I can still reproduce the problem in dev build 200409201800.
Comment 21 Jesse Glick 2004-09-27 08:12:48 UTC
In which JDK?
Comment 22 _ gtzabari 2004-09-27 13:52:55 UTC
JDK 1.5 RC
Comment 23 Jesse Glick 2004-09-27 16:29:42 UTC
OK Petr could you recheck it? Perhaps the fix did not do what it is
supposed to, or perhaps there is a different leak...
Comment 24 Petr Nejedly 2004-09-27 16:37:02 UTC
> I can still reproduce the problem in dev build 200409201800

With the same build.xml or have you added some other tool in the mean
time? Please note that this class of problems is specific to used
taskdefs.
Comment 25 _ gtzabari 2004-09-27 19:26:51 UTC
Petr,

I'm going to attach my updated build.xml. I doubt much has changed
within it. I am pretty sure I have not added any new taskdefs.
Comment 26 _ gtzabari 2004-09-27 19:27:35 UTC
Created attachment 17896 [details]
Updated build.xml
Comment 27 Petr Nejedly 2004-09-29 19:14:48 UTC
Seeems that I can reproduce some leak in the newest build under JDK1.5.
It may be different, but certainly leads to OOME.
Comment 28 Petr Nejedly 2004-09-30 12:53:25 UTC
Can't reproduce with the FCS of JDK1.5.
The problem I was seeing was already fixed leak in ZipFile.

Try installing newest JDK and if the problem persists, checke whether
it is really this problem by using
jstat -class <pid>
and
jstat -gcpermcapacity
Comment 29 _ gtzabari 2004-10-19 08:17:49 UTC
Look, I'm seeing a definate leak here. I'm trying to use jstat but the
numbers mean nothing to me. How do I know if this is the same issue or
some other leak? Either way, after about 10-20 rebuilds of my project
I run out of memory and must restart Netbeans. Often I find out I am
out of memory too late and Netbeans hangs with major cpu usage and
must be terminated which sometimes leads to data loss. All very
upsetting :)

I'd like to reopen this issue if possible until this is resolved.
Comment 30 Petr Nejedly 2004-10-19 11:51:20 UTC
jvmstat -class shows the number of loaded and unloaded classes.
If you repetitively build your sources, both number should grow by
about the same amount, so the number of classes in memory should be
constant (and not much over 10000 after long usage)
Note: class unloading is asynchronous during gc, so use gc action
before checking number of unloaded classes.

If the number of classes in memory grows repeatedly during builds, it
is a class leak, otherwise it is something different, see e.g. issue
50454.
Comment 31 Marian Mirilovic 2005-12-20 15:50:09 UTC
This issue was solved long time ago. Because nobody has reopened it neither
added comments, we are verifying/closing it now. 
If you are still able to reproduce the problem, please reopen. Thanks in advance.