This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 164325 - nb 6.7 beta and nightly builds are so slow they cannot be used
Summary: nb 6.7 beta and nightly builds are so slow they cannot be used
Status: CLOSED FIXED
Alias: None
Product: ide
Classification: Unclassified
Component: Performance (show other bugs)
Version: 3.x
Hardware: Macintosh All
: P2 blocker (vote)
Assignee: issues@performance
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2009-05-02 23:30 UTC by _ ludo
Modified: 2011-05-25 11:38 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
my projects (47.56 KB, image/png)
2009-05-02 23:31 UTC, _ ludo
Details
activity after an hour the IDE is running: 3 Retrieving server data... Which Server? Which Server? (23.09 KB, image/png)
2009-05-03 00:15 UTC, _ ludo
Details
23 snapshots memory: starting at 13: they is no more progess bar and cpu is still at around 100% for a long while. (737.27 KB, application/x-compressed)
2009-05-15 17:34 UTC, _ ludo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description _ ludo 2009-05-02 23:30:49 UTC
I've been trying Nb 6.7 bits for a while (m3, beta, nightly builds including April 30th   - I change my Nb build almost
every week to trak changes).
Iam a Java developer so I have always these projects open:

1/ a few GlassFish maven project (src in glassfish.dev.java.net)
2/ a few nb projects (ajax.dev.java.net)
3/ 3 or 5 web apps to test the GlassFish plugins
4/ randomw other projects (maven grizzly)...

Every single time I started these builds with my working env, CPY time of Nb goes 100-150% (mac duo pro, 4GbRAM), for
about 6 to 12 minutes scanning, opening projects, stopping (no progress indicator) starting again with scanning projects.

I am not alone (I read netbeans feedback and nbdev or nbusers mailing lists every day). I guess there might be another
set of bugs open for this issue.

This issue is so important that I cannot use NetBeans anymore for my day to day work. I recently move to IntelliJ and I
do not have these issues with the same type of projects opened.
Not sure what to do there, but this is a serie of regressions from NetBeans 4.1, NetBeans 5.0, NetBeans 6.0, NetBeans
6.1 and NetBeans 6.5. Every time, I have a faster laptop and everytime the IDE seems slower... 
I do not think you need thread dump from my starting IDE. Just use projects (GlassFish Maven, Grizzly Mave, Web App, ans
NB Modules, and also open a few dirs from the Favorites tab, just regular Java Developer tasks I assume) and see for
yourself: NetBeans 6.7 is not a usable IDE anymore.

Attached is an image of my projects. You can all get them from open Source repositories.
Comment 1 _ ludo 2009-05-02 23:31:36 UTC
Created attachment 81419 [details]
my projects
Comment 2 _ ludo 2009-05-02 23:36:59 UTC
After 20 minutes Nb is started, the scanning project progress bar is gone, but NetBeans still consumes between 8% and
57% of my CPU.
I am doing absolute nothing...And my Mac CPU is so hot I can fry an egg....


Product Version: NetBeans IDE Dev (Build 200904300201)
Java: 1.6.0_13; Java HotSpot(TM) 64-Bit Server VM 11.3-b02-77
System: Mac OS X version 10.5.6 running on x86_64; MacRoman; en_US (nb)
Userdir: /Users/ludo/.netbeans/dev

Comment 3 _ ludo 2009-05-03 00:15:30 UTC
Created attachment 81421 [details]
activity after an hour the IDE is running: 3 Retrieving server data... Which Server?  Which Server?
Comment 4 Vitezslav Stejskal 2009-05-05 11:56:48 UTC
Yes, the scanning is slow in 6.7 and yes there are many defects related to this. One of them is issue #162706, which I'm
now investigating. The testcase I'm using is Jackrabbit opensource project, which seems to be big enough to show some
real numbers, but not too big to prevent profiling. I think I found what causes the major slowdown and I hope to have it
fixed soon. Please CC yourself to 162706.

I'm not sure about the 'Retrieving Server Data' tasks. A threaddump in this case would be useful in order to quickly
identify the offending module. Also please avoid reporting several problems in one defect. The scanning performance
problem is IMO a duplicate of 162706. I'm not sure about the rest of this report.
Comment 5 Tomas Pavek 2009-05-06 15:26:27 UTC
'Retrieving Server Data' is from the bugtracking integration, I get it when expanding Issues node of a Kenai project -
the IDE runs the defined queries. I haven't seen it otherwise.
Comment 6 John Jullion-ceccarelli 2009-05-13 10:46:26 UTC
Issue #16 was fixed. Ludo, can you check with a daily build and see if it's better?
Comment 7 Vitezslav Stejskal 2009-05-14 09:12:58 UTC
Yes, issue #162706 was fixed and so far the feedback was very positive about the fix. Please try again in a newer dev
build. Thanks
Comment 8 _ ludo 2009-05-14 11:26:47 UTC
Product Version: NetBeans IDE Dev (Build 200905140201)
Java: 1.6.0_13; Java HotSpot(TM) 64-Bit Server VM 11.3-b02-77
System: Mac OS X version 10.5.7 running on x86_64; MacRoman; en_US (nb)
Userdir: /Users/ludo/.netbeans/dev

is better (4-5 minutes scans and heavy cpu usage >120%) only (still way too much...imagine restarting the IDE at a
JavaOne session during a demo...)
Then, after the scanning is done, doing nothing in the IDE, keeps the CPU to run at 60-120% for another 5 minutes.
After 5 mins (no ui indication of work in the IDE), CPU goes to acceptable level <10%

I would not consider this to be completely fixed imo (i.e 10 minutes startup time)...Of course it is much much better.
Scanning does not seem to restart randomly.
But if you compare to other IDEs, it is still  not as good.
If you compared to previous NB versions, it is not as good as well.
Comment 9 Vitezslav Stejskal 2009-05-14 11:43:18 UTC
Could you please attach threaddumps generated during scanning and then after the scanning is done, but the IDE is still
doing something? Could you measure the scanning times in Nb6.5 on the same project so that we have some comparison? What
happens after restarting the IDE? How much time before 'Scanning projects' goes away?
Comment 10 _ ludo 2009-05-14 16:19:14 UTC
cannot: 
47096 Main -Djdk.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home -Dnetbeans.system_http_proxy=DIRECT
-Dnetbeans.system_http_non_proxy_hosts= -Dnetbeans.dirs=/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/nb6.7:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/ergonomics1:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/ide11:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/java2:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/MacOS/../Resources/NetBeans/bin/../xml2:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/apisupport1:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/webcommon1:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/websvccommon1:/Applications/NetBeans/NetBeans Dev
200905140201.app/Contents/Resources/NetBeans/enterprise5:/Applications/NetBeans/NetBean
48666 Jps -Dapplication.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home -Xms8m
dhcp-usca14-133-133:bin ludo$ jstack 47096
47096: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

dhcp-usca14-133-133:bin ludo$ jstack -F 47096
Attaching to process ID 47096, please wait...
attach: task_for_pid(47096) failed (5)
Error attaching to process: Error attaching to process, or no such process
Comment 11 _ ludo 2009-05-14 16:25:23 UTC
restart is still slow and cpu intensive for more than 4 minutes, scanning the jdk jars (why a second time?) the .m2 repo
that contains glassfish, grizlly, some kenai projects, the glassfish jars, etc etc
And after the scanning is done, (no UI indication) the CPU keeps running.

IntelliJ has a manual refresh I think...This is a much better user experience... You may be not up to date, but you have
the choice...
Comment 12 Vitezslav Stejskal 2009-05-15 08:50:41 UTC
"IntelliJ has a manual refresh" - there is an experimental (and I stress this word) module that supports manual refresh.
Supporting manual refresh is much more complicated than it may sound due to the fact how all the java support is wired
to indexing, etc. This is totally out of question for this release. We may improve the experimental module for the next
release, but don't expect it to become part of the standard distribution.

Unfortunately without threaddumps or reproducible/measurable usecase I can't do anything about this issue.
Comment 13 Milos Kleint 2009-05-15 10:10:41 UTC
there was one case when maven integration silently consumed CPU (after downloading remote index, it tried to resolve
each artifact document in the lucene index through the local repository), is fixed for a few days by upgrading to newer
version of nexus-indexer and making sure just local artifacts are resolved this way, not the ones from remote repositories..

might or might not be related to this issue..

a thread dump would tell, of course..

Comment 14 _ ludo 2009-05-15 16:37:04 UTC
other method than jstack for dumps on mac?
Comment 15 Tomas Pavek 2009-05-15 16:54:58 UTC
Can you possibly create a profiling snapshot instead of thread dumps?
http://wiki.netbeans.org/FaqProfileMeNow
Comment 16 _ ludo 2009-05-15 17:34:17 UTC
Created attachment 82213 [details]
23 snapshots memory: starting at 13: they is no more progess bar and cpu is still at around 100% for a long while.
Comment 17 _ ludo 2009-05-15 17:38:53 UTC
all in all, all these snapshots are done during 6 minutes.
The last 3 minutes, there is no progress indicator related to opening projects or scanning...Just pure cpu crunching.

After these 6 minutes, the IDE is ok (for the most part).
Comment 18 Milos Kleint 2009-05-18 06:55:09 UTC
from the few snapshots I've checked it seems only the SVN checking thread is always doing something. It will refresh
some kind of caches that eventually load maven projects and via SharabilityQuery in the project load the resolved maven
instance.. 
Comment 19 Vitezslav Stejskal 2009-05-18 10:52:58 UTC
"The last 3 minutes, there is no progress indicator related to opening projects or scanning..." - I think I know what
the problem is and will fix it. Thanks
Comment 20 Vitezslav Stejskal 2009-05-20 16:50:01 UTC
The progress bar should now be fixed - http://hg.netbeans.org/jet-main/rev/b1a16d1468e8
Comment 21 _ ludo 2009-05-21 02:51:32 UTC
So what about the svn scanning?
What about rescanning the JDK jar files?
Am I stuck with a 6 minutes startup time?
Comment 22 _ ludo 2009-05-23 17:14:14 UTC
Good news: Product Version: NetBeans IDE Dev (Build 200905230201)
Java: 1.6.0_13; Java HotSpot(TM) 64-Bit Server VM 11.3-b02-77
System: Mac OS X version 10.5.7 running on x86_64; MacRoman; en_US (nb)
Userdir: /Users/ludo/.netbeans/dev


is behaving much much much better. For me, this bug is now fixed.