This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 187071 - Huge performance regression compiling java module suite
Summary: Huge performance regression compiling java module suite
Status: VERIFIED FIXED
Alias: None
Product: apisupport
Classification: Unclassified
Component: Harness (show other bugs)
Version: 6.x
Hardware: PC Linux
: P3 normal with 3 votes (vote)
Assignee: Jaroslav Tulach
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2010-06-02 08:19 UTC by anba
Modified: 2010-06-21 13:55 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
6.8 verbose output of clean and build of FR sample (257.45 KB, application/octet-stream)
2010-06-08 13:46 UTC, Tomas Danek
Details
6.9 verbose output of clean and build of FR sample (241.93 KB, application/octet-stream)
2010-06-08 13:47 UTC, Tomas Danek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description anba 2010-06-02 08:19:05 UTC
We have a module suite with ca. 45 modules and the performance for compiling is very slow using 6.9rc1

In netbeans 6.8 the clean time for the module suite is 13 seconds.
In netbeans 6.9rc1 the clean time for the module suite is 1 minute 59 seconds . 

This means the clean operation is 9-10 times slower in netbeans 6.9rc1 compared to netbeans 6.8.

In netbeans 6.8 the compilation time for the module suite is 1 minute 0 seconds.
In netbeans 6.9rc1 the compilation time for the module suite is 3 minutes 13 seconds.

This means the compile operation is 3-4 times slower in netbeans 6.9rc1 compared to netbeans 6.8.

These times were determined using identical machines and options to the compile process. They do not differ much whether the configuration of netbeans 6.8 is imported or not.

This makes debugging very uncomfortable since the system tries to compile each time the project if debugging and a time delay of 3 minutes against 1 minute is inacceptable.
Comment 1 Antonin Nebuzelsky 2010-06-04 11:18:19 UTC
Need to investigate ASAP.

Reassigning to apisupport.

Note that performance test results show slight difference between 6.8 and 6.9 in build time for J2SE project. However, given the relatively small difference (5-10% regression) those results may not be indicating anything real.
Comment 2 converginglight 2010-06-07 17:21:57 UTC
From previous experience with Netbeans nightly trunks versus releases I can say: nighties are slower when it comes to building projects.

Can someone else confirm this?

Is it because of some debugging information, loggers, etc... ?
Comment 3 Tomas Danek 2010-06-08 10:30:40 UTC
Cannot be related to harness changes? Can reproduce, building feedreader in 6.8 ~5sec, while in 6.9 ~15 sec.
Comment 4 Tomas Danek 2010-06-08 13:46:32 UTC
Created attachment 99898 [details]
6.8 verbose output of clean and build of FR sample
Comment 5 Tomas Danek 2010-06-08 13:47:11 UTC
Created attachment 99899 [details]
6.9 verbose output of clean and build of FR sample
Comment 6 Jesse Glick 2010-06-08 20:31:29 UTC
Can reproduce running clean target on SQE suite.

-J-da improves things, which I can fix in the Ant module: core-main #425435f4b943. (No need to backport, since release builds do not use assertions.)

Even with that, performance seems cca. 2x slower when using the 6.9 harness than when using the 6.8 harness. You can select which harness to use for a given NB platform version, so there is no regression in the IDE as such and there is a simple workaround.
Comment 7 Jesse Glick 2010-06-08 21:10:25 UTC
Change from 6.8 to 6.9 harness reproducible running Ant from the command line.

There is a bit of slowdown caused by config/ModuleAutoDeps/*.xml processing, but most of the slowdown seems to come from config/Modules/*.xml parsing (incl. XPath) to support OSGi bundle clusters (jtulach's 4f0b3ab8033d). The following restores most of 6.8's speed:

diff --git a/nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java b/nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java
--- a/nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java
+++ b/nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java
@@ -584,6 +584,9 @@
                     if (!fileName.endsWith(".xml")) {
                         continue;
                     }
+                    if (entries.containsKey(fileName.substring(0, fileName.length() - 4).replace('-', '.'))) {
+                        continue;
+                    }
                     try {
                         if (expr == null) {
                             expr = XPathFactory.newInstance().newXPath().compile("/module/param[@name='jar']");

Assigning to author to review.
Comment 8 Quality Engineering 2010-06-10 06:07:26 UTC
Integrated into 'main-golden', will be available in build *201006100001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/425435f4b943
User: Jesse Glick <jglick@netbeans.org>
Log: #187071: Ant execution noticeably slower with assertions enabled.
Comment 9 Jesse Glick 2010-06-15 20:58:36 UTC
Any opinion on the proposed patch? If it seems safe and effective, I would suggest it be a 6.9.1_CANDIDATE. (Despite the existence of a workaround - use an older harness - the 6.9 performance regression might merit a P2 for this issue; not sure.)
Comment 10 Jaroslav Tulach 2010-06-15 21:58:03 UTC
Looks reasonable. Applied as core-main#2a6d68c9e941
Comment 11 rbalada 2010-06-16 08:55:00 UTC
Please verify bugfix for this bug, so it can be included in release691 repository.

Thanks,
-R
Comment 12 Quality Engineering 2010-06-17 03:19:28 UTC
Integrated into 'main-golden', will be available in build *201006170001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/2a6d68c9e941
User: Jesse Glick <jglick@netbeans.org>
Log: #187071: Skipping parsing of config/Modules/*.xml if appropriate module JAR seems to exist in regular locations
Comment 13 Petr Blaha 2010-06-17 11:51:46 UTC
verified
Comment 14 rbalada 2010-06-21 13:55:41 UTC
main #2a6d68c9e941 transplanted to release691 #2d0a8ffafc64
main #425435f4b943 transplanted to release691 #b7def3359d4d