This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Dell Precision 220, PIII 800MHz, 512MB RAM, Windows 2000, JDK 1.4.2_04 Project jEdit open and jEdit.java open in editor. Showing code completion for "System.": trunk 200405131800: 1282ms, 937ms, 687ms refactoring 040513: 1829ms, 1250ms, 1141ms Showing code completion for "View.": trunk 200405131800: 1344ms, 875ms, 703ms refactoring 040513: 2969ms, 1546ms, 1245ms
do we know why the number is so bad even for trunk builds? I remember we did incredible amount of work to make this fast in 3.5
results from optimizeIt after five invocations of completion on "new " global org.netbeans.modules.javacore.ExclusiveMutex.enter - 35.9% main slice - 12.3% from org.netbeans.modules.java.JavaEditor$2.actionPerformed ... org.netbeans.modules.java.JavaEditor.parseSource() next: 22.6% from org.netbeans.jmi.javamodel.JavaClass$Impl.getName() called from org.netbeans.modules.editor.java.NbJavaJMISyntaxSupport.getJavaDocURLs() - I will look at this, whether it is possible to reduce the getName() calls 8% org.netbeans.jmi.javamodel.JavaClass$Impl.getDeclaringClass() called from org.netbeans.modules.editor.java.NbJMIPaintComponent.formatClassName() called from org.netbeans.modules.editor.java.NbJMIResultItem$ClassResultItem.getItemText() and next 5% from org.netbeans.modules.editor.java.NbJMIPaintComponent.formatClassName() called from org.netbeans.modules.editor.java.NbJMIPaintComponent.getPreferredSize()
I have fixed several CC performance problems in editor module. I believe the results of invoking cc on "new |" could be better within 20-30% As for cc on "System.|" the current optimizeIt results are: RP: 50.8% - org.netbeans.jmi.javamodel.TypeClass$Impl.resolve() called from org.netbeans.modules.editor.java.JMIUtils.getExactClass() EQ: 8% - org.netbeans.jmi.javamodel.Field$Impl.getType() called from org.netbeans.modules.editor.java.NbJMIPaintComponent$NbFieldPaintComponent.draw() /cvs/editor/libsrc/org/netbeans/editor/ext/Completion.java,v <-- Completion.java new revision: 1.29.66.1; previous revision: 1.29 /cvs/editor/libsrc/org/netbeans/editor/ext/CompletionJavaDoc.java,v <-- CompletionJavaDoc.java new revision: 1.25.14.5; previous revision: 1.25.14.4 /cvs/editor/src/org/netbeans/modules/editor/java/Attic/JMIUtils.java,v <-- JMIUtils.java new revision: 1.1.2.14; previous revision: 1.1.2.13 /cvs/editor/src/org/netbeans/modules/editor/java/Attic/NbJMIPaintComponent.java,v <-- NbJMIPaintComponent.java new revision: 1.1.2.8; previous revision: 1.1.2.7 /cvs/editor/src/org/netbeans/modules/editor/java/Attic/NbJMIResultItem.java,v <-- NbJMIResultItem.java new revision: 1.1.2.12; previous revision: 1.1.2.11 /cvs/editor/src/org/netbeans/modules/editor/java/Attic/NbJavaJMICompletion.java,v <-- NbJavaJMICompletion.java new revision: 1.1.2.3; previous revision: 1.1.2.2 /cvs/editor/libsrc/org/netbeans/editor/ext/java/JCCellRenderer.java,v <-- JCCellRenderer.java new revision: 1.9.180.3; previous revision: 1.9.180.2
Tondo, can you measure code completion again? I did some measuring on my machine(Tecra S1, Pentium Mobile 1.6GHz, 1GB ram, Windows XP). Showing code completion for "System.": yesterday's trunk: ~700ms, 600ms, 570ms today's refactoring: ~920ms, 750ms, 690ms
Please, update your build before measuring. I've just checked in bugfix of issue #43873
I've done a measurement with today's build (MC3) and also measured the yesterday's trunk build for comparison. Project jEdit open and jEdit.java open in editor. Memory meter turned off ;o) Dell Precision 220, PIII 800MHz, 512MB RAM, W2K, JDK 1.4.2_04 Showing code completion for "System.": trunk 200405251800: 1219ms, 688ms, 703ms refactoring 040526: 1375ms, 922ms, 969ms Showing code completion for "View.": trunk 200405251800: 718ms, 657ms, 657ms refactoring 040526: 2938ms, 1188ms, 1109ms Showing code completion for "import java.": trunk 200405251800: 860ms, 656ms, 656ms refactoring 040526: 953ms, 1016ms, 1047ms So, there still is a regression, though not that bad as it seemed yesterday.
New measurement with the same setup. Showing code completion for "System.": trunk 20040528-0610: 1500ms, 828ms, 734ms refactoring 040527: 1000ms, 843ms, 813ms Showing code completion for "View." (open project's class): trunk 20040528-0610: 844ms, 688ms, 703ms refactoring 040526: 4109ms, 1016ms, 859ms Showing code completion for "import java.": trunk 20040528-0610: 1156ms, 734ms, 703ms refactoring 040526: 875ms, 750ms, 719ms Showing code completion for "new ": trunk 20040528-0610: 4000ms, 1093ms, 969ms refactoring 040526: 7453ms, 1687ms, 1812ms
dev build from Jun 20 sources, Linux, JDK1.5.0b55 code completion behaviour is terrible in latest builds. couple observations from runs with two small projects open: if you invoke CC on the same place more times o.n.a.java.classpath.ClassPath.getRoots is called at least twice. I am not sure why - the code seems to be caching results if you do it on various places it can be called >10 times. It seems to be suspicious. Many classes from JDK are parsed when CC is to be displayed. Many classes from JDK are parsed again when another CC action is called (checked with debug print added to ASTProvider.createASTree).
Moved to new subcomponent java/javacore.
Yesterday's build seems to be pretty slow, so I did another set of CC measurements with the same setup: Showing code completion for "System.": trunk 200406230100: 1312ms, 859ms, 859ms Showing code completion for "View." (open project's class): trunk 200406230100: 7328ms, 969ms, 1141ms <- very bad! Showing code completion for "import java.": trunk 200406230100: 1078ms, 750ms, 782ms Showing code completion for "new ": trunk 200406230100: 69s, 10250ms, 10234ms <- terribly bad!
If I can judge, it's better in the qbuild 200406251039, but Tonda's measures will say more.
New measurement... Showing code completion for "System.": trunk build 200406301800: 1515ms, 828ms, 781ms Showing code completion for "View." (open project's class): trunk build 200406301800: 8312ms, 1000ms, 859ms <- still bad Showing code completion for "import java.": trunk build 200406301800: 704ms, 719ms, 640ms Showing code completion for "new ": trunk build 200406301800: 8891ms, 6890ms, 6547ms <- still bad
Changing summary text.
FYI, our performace suite is up again and this test shows that Code completion doesn't meet UI performance criteria ...... [nb_dev](200407141800), [jdk1.5.0](beta3 - b58) on our lab machines: --------------------- Linux RH9 JDS2 Sol9 W2K WXP 1st usage 2144 1534 4451 3254 3515 2nd usage 148 208 365 115 120 --------------------- Test case: - run IDE - Open java file - type text "System." - and type "." and wait for completion dialog ----------------------
New measurement, toghether with comparison to Netbeans 3.6. Dell Precision 220, PIII 800MHz, 512MB RAM, Windows 2000, JDK 1.4.2_04 Project jEdit open and jEdit.java open in editor. Netbeans 3.6 versus trunk 200407181800. Showing code completion for "System.": nb36: 1328ms, 594ms, 687ms trunk: 1656ms, 829ms, 813ms Showing code completion for "View." (open project's class): nb36: 922ms, 594ms, 563ms trunk: 7718ms, 1641ms, 1016ms <- !!! Showing code completion for "import java.": nb36: 812ms, 578ms, 609ms trunk: 812ms, 640ms, 750ms Showing code completion for "new ": nb36: 610ms, 312ms, 282ms trunk: 5344ms, 7266ms, 5125ms <- !!!
New measurement after the latest improvements from Thursday and Friday. trunk build 20040725-1602 Showing code completion for "System.": trunk: 1407ms, 672ms, 656ms Showing code completion for "View." (open project's class): trunk: 6312ms, 2515ms, 969ms <- !!! Showing code completion for "import java.": trunk: 797ms, 765ms, 750ms Showing code completion for "new ": trunk: 5547ms, 4109ms, 3391ms <- !!!
Talked about CC with Pavel. I am adding one more type of measurement - also jedit's class (as well as the View class above), but this time the class is not extending any other class, just java.lang.Object. trunk build 20040729-1140 Showing code completion for "Mode." (open project's class): trunk: 1141ms, 718ms, 704ms
Correction - showing code completion for "Mode m; m." because "Mode." just appends "class" and does not show the CC popup.
Created attachment 16651 [details] A couple minor optimizations that might help
I added some logging to org.netbeans.modules.javacore.ClassIndex, and noticed that for each invocation of code completion, call to getClassesByFQNPrefix(String fqnPrefix) is called +/- 20 times; each time it rebuilds its result list. The attached patch seems to help - it caches the last requested package name, and weakly caches the result. One other optimization I'm not sure is correct (I really don't know this code): Results were being filtered based on whether their length was < the passed package name. I believe testing if the last index of '.' is a match would give fewer false positives, but as I said, I don't know this code.
One other thing - when trying the patch, I tried logging if the weak reference to the result list had been cleared. Even with forcing garbage collection, it never was; this suggests that something is holding a hard reference to the return value.
Created attachment 16672 [details] Implementation of lazy iterator for getClassesBySNPrefix
Yesterday and today I have commited changes that should make the code completion faster (all data necessary for code completion are now lazily stored in the persistent storage). This change significantly improves performance of consequent invocations of code completion, it also speeds up files opening, finding usages and other tasks. To make also the very first invocation of code completion faster, I added a warmup parsing of javax.swing.JFrame and all its super-classes/-interfaces to the initial scanning (JFrame was chosen because the biggest problem for the first invocation are classes with very long inheritance hierarchies which swing classes typically are - this reduces number of files to be parsed during the code completion invocation). I would like to kindly ask Tonda to provide new measurements.
New measurement with Martin's improvements. The only real-bad case is now listing all available classes in "new |". trunk build 20040809-1429 Showing code completion for "System.": trunk: 1250ms, 781ms, 687ms; 1063ms, 688ms, 766ms Showing code completion for "View." (open project's class): trunk: 2016ms, 1297ms, 1047ms; 1609ms, 1000ms, 937ms Showing code completion for "import java.": trunk: 687ms, 641ms, 640ms; 907ms, 625ms, 672ms Showing code completion for "new ": trunk: 6750ms, 8253ms, 4031ms; 5359ms, 7453ms, 3969ms <- !!! Showing code completion for "Mode m; m.": trunk: 875ms, 703ms, 687ms; 766ms, 657ms, 656ms
; in the lists of numbers above means restart of the IDE :o)
I don't know if you tried the patch I attached; weakly caching the result of the last call seems to make a pretty dramatic difference, in my subjective experience (eliminating the excessive calls to the method in question would probably accomplish the same thing).
Tim, thanks for your patch. I am looking at it right now. The fix of replacing "length" by "lastPeriod" was a good catch! Also thanks for finding out that there are several calls to the getClassesByFQNPrefix method during a single invocation of code completion. I am going to investigate if that can be avoided, since caching results of this method is not that simple - classes can be deleted while they are held in the cache - we would need to be able to detect that and invalidate the cache or re-check it every time we use the cached information.
In fact I just realized that in this case "lastPeriod == result.getName().lastIndexOf('.')" if and only if "length > result.getName().lastIndexOf('.')" since "prefix.equals(result.getName().substring(0, result.getName().length()))" so the code was correct since both conditions are equivalent...
Regarding several invocations of getClassesByFQNPrefix - each of these invocations is made on a different instance of ClassIndex, so the caching should not have any effect. There is one instance of ClassIndex per classpath element. So for each classpath element on the current classpath one call to ClassIndex is made, each returns a different result - the results are then merged.
I have modified classindex to return a lazy list based on Tomas' patch. Tondo, please try to measure "new |" case again. Thanks. Checking in src/org/netbeans/modules/javacore/LazyImmutableList.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/LazyImmutableList.java,v <-- LazyImmutableList.java initial revision: 1.1 done Checking in src/org/netbeans/modules/javacore/ClassIndex.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/ClassIndex.java,v <-- ClassIndex.java new revision: 1.13; previous revision: 1.12 done
I have measured the "new " case again. Showing code completion for "new " (explicitly via CTRL+SPACE): trunk: 4359ms, 2796ms, 2641ms Showing code completion for "new J" after restart: trunk: 1187ms, 921ms, 985ms Automatic display of the code-completion window now happens only after the user writes at least one letter of the class name and listing all classes must be invoked explicitly using CTRL+SPACE. Given this and given that the time for listing all classes has improved significantly and listing classes starting with J is around 1 second, I think CC is not a beta-stopper anymore.
New measurement. trunk build 200410131800 Showing code completion for "System.": trunk: 1578ms, 1125ms, 1109ms; 3360ms, 1188ms, 1109ms Showing code completion for "View." (open project's class): trunk: 2547ms, 1703ms, 1610ms; 2109ms, 1672ms, 1968ms Showing code completion for "import java.": trunk: 953ms, 890ms, 907ms; 984ms, 1015ms, 921ms Showing code completion for "new ": trunk: 9500ms, 3078ms, 2938ms; 7359ms, 2469ms, 2375ms Showing code completion for "Mode m; m.": trunk: 1062ms, 1188ms, 1203ms; 1281ms, 1344ms, 1219ms These are significantly worse numbers than with the last measurement above (trunk build 20040809-1429). I disagree with waiving this issue in the current state.
Note that NetCAT members (who were asked to tell us about the most annoying performance issues) are mostly complaining about code completion being slow.
Do we actually know what is the thing taking up so much time in displaying code completion?
AFAIK the last regression is caused by a forced reparse of the current document before each CC invocation. This fixes some InvalidObjectExc. But unfortunatelly adds a few hundreds millis to each CC. Javacore team is currently looking at some ways to improve.
The last regression should now be fixed. We have managed to avoid reparsing without introducing any InvalidObjectException. Tondo, could you please provide new numbers and let us know whether they are acceptable for waiver? Thanks. Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java,v <-- ResourceImpl.java new revision: 1.44; previous revision: 1.43 done
trunk build 20041022-1058, with 1.4.2_04 (but I verified that w/ 1.5.0 it produces very same numbers) Showing code completion for "System.": trunk: 2141ms, 875ms, 969ms; 1657ms, 922ms, 891ms (was 1578ms, 1125ms, 1109ms; 3360ms, 1188ms, 1109ms) Showing code completion for "View." (open project's class): trunk: 2906ms, 1375ms, 1656ms; 5078ms, 1579ms, 1468ms (was 2547ms, 1703ms, 1610ms; 2109ms, 1672ms, 1968ms) Showing code completion for "import java.": trunk: 813ms, 672ms, 672ms; 734ms, 656ms, 672ms (was 953ms, 890ms, 907ms; 984ms, 1015ms, 921ms) Showing code completion for "new ": (BTW, this has to be invoked explicitly by CTRL+SPACE because otherwise it waits for at least one letter as a prefix for the class name - as agreed before Beta 2 (above)) trunk: 4969ms, 3313ms, 3188ms; 7344ms, 3109ms, 3312ms (was 9500ms, 3078ms, 2938ms; 7359ms, 2469ms, 2375ms) Showing code completion for "Mode m; m.": trunk: 1078ms, 985ms, 938ms; 907ms, 860ms, 906ms (was 1062ms, 1188ms, 1203ms; 1281ms, 1344ms, 1219ms) This is noticeably better, but still worse than on 2004-08-09.
FYI: I have tried increasing AST cache size back to 16 (now it is 2). This did not influence the results.
Martine, anyone in javacore has any ideas for more improvements?
Dusan is evaluating the performance using OptimizeIt. Any other help (in terms of profiling and identifying possible improvements) would be appreciated.
Radim is trying to find out something in profiler now as well.
I've made another improvement. According to my measurements the performance should be improved at least by 25% not considering the 500ms delay of the automatic invocation (at least for the case of "View.") Checking in src/org/netbeans/modules/javacore/ExclusiveMutex.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/ExclusiveMutex.java,v <-- ExclusiveMutex.java new revision: 1.36; previous revision: 1.35 done Checking in src/org/netbeans/modules/javacore/JMManager.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/JMManager.java,v <-- JMManager.java new revision: 1.72; previous revision: 1.71 done Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/SemiPersistentElement.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/SemiPersistentElement.java,v <-- SemiPersistentElement.java new revision: 1.55; previous revision: 1.54 done Processing log script arguments... src/org/netbeans/modules/editor/java/NbJavaJMICompletionQuery.java Checking in src/org/netbeans/modules/editor/java/NbJavaJMICompletionQuery.java; /cvs/editor/src/org/netbeans/modules/editor/java/NbJavaJMICompletionQuery.java,v <-- NbJavaJMICompletionQuery.java new revision: 1.43; previous revision: 1.42 done
trunk build 200410281800, with 1.5.0 Showing code completion for "System.": trunk: 1594ms, 843ms, 719ms; 1141ms, 828ms, 750ms (was 2141ms, 875ms, 969ms; 1657ms, 922ms, 891ms) Showing code completion for "View." (open project's class): trunk: 2171ms, 1125ms, 1079ms; 1360ms, 1266ms, 1140ms (was 2906ms, 1375ms, 1656ms; 5078ms, 1579ms, 1468ms) Showing code completion for "import java.": trunk: 750ms, 735ms, 672ms; 734ms, 672ms, 672ms (was 813ms, 672ms, 672ms; 734ms, 656ms, 672ms) Showing code completion for "new ": (BTW, this has to be invoked explicitly by CTRL+SPACE because otherwise it waits for at least one letter as a prefix for the class name - as agreed before Beta 2 (above)) trunk: 5297ms, 3250ms, 3157ms; 6875ms, 3703ms, 3125ms (was 4969ms, 3313ms, 3188ms; 7344ms, 3109ms, 3312ms) Showing code completion for "Mode m; m.": trunk: 875ms, 765ms, 812ms; 875ms, 797ms, 797ms (was 1078ms, 985ms, 938ms; 907ms, 860ms, 906ms) So, now we are basically on the numbers we had in Beta 2. There is still a chance to decrease the CC timeout (currently 500ms as a default setting) or start preparing the contents of CC before this timeout from the last keystroke (300ms in advance -> after 200ms from the last keystroke). But it is questionable if this would not slow down editor with too much parsing if the user is typing too slow.
Reassigning to editor for evaluation of the idea of decreasing the auto-completion timeout/computation of the data in advance.
As we discussed on the performance meeting on Wednesday, it does not seem to be a good idea. Based on the feedback from NetCAT, where the users were asked to decrease the popup delay timeout from 500ms to 200ms. The feedback was mixed. Some users reported improvement, some users noticed the opposite. We agreed to waive this issue for NB 4.0 now. The option of decreasing the popup delay timeout will be documented as a tip.
Waiver approved for 4.0.
*** Issue 53765 has been marked as a duplicate of this issue. ***
Several teams have reported that performance of code completion is not good enough in 4.1. I have filed an issue #53765 on editor module but according to editor developers nothing can be done at the moment on editor's side to improve the performance of code completion. Will you address this issue in 4.1? Thanks.
Reassigning to Tomas, since he is working on performance improvements.
JMManager.getJavaExtent() is now a lot more faster. See issue #52866. JMManager.getJavaExtent() is also used indirectly by code-completion.
AST can be created without classpath. Creation of string representation of classpath was removed. This makes java file parsing faster. Checking in src/org/netbeans/modules/javacore/parser/ASTProvider.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/parser/ASTProvider.java,v <-- ASTProvider.java new revision: 1.39; previous revision: 1.38 done
Starting of nested transaction was optimized. Checking in src/org/netbeans/modules/javacore/ExclusiveMutex.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/ExclusiveMutex.java,v <-- ExclusiveMutex.java new revision: 1.39; previous revision: 1.38 done
toArray() is faster than iterating through all LightAttrList. Internal javacore code was changed to avoid iterating through LightAttrLists and now uses toArray() to get all elements from LightAttrList. http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/ JMManager.java.diff?r1=1.96&r2=1.97 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/AttributeValueImpl.java.diff?r1=1.11&r2=1.12 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/ClassDefinitionImpl.java.diff?r1=1.13&r2=1.14 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/CollectionMatcher.java.diff?r1=1.2&r2=1.3 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/FeaturesList.java.diff?r1=1.3&r2=1.4 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/ImportImpl.java.diff?r1=1.17&r2=1.18 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/JavaClassImpl.java.diff?r1=1.49&r2=1.50 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/LightAttrList.java.diff?r1=1.11&r2=1.13 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/MetadataElement.java.diff?r1=1.46&r2=1.47 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/ResourceImpl.java.diff?r1=1.68&r2=1.69 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/SemiPersistentElement.java.diff?r1=1.66&r2=1.67 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/SubClassesCollection.java.diff?r1=1.8&r2=1.9 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/ javamodel/ParameterizedTypeImpl.java?r1=1.20&r2=1.21 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/parser/ InfoUtil.java.diff?r1=1.4&r2=1.5 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/parser/ MDRParser.java.diff?r1=1.70&r2=1.71 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/parser/ MemberFieldScope.java.diff?r1=1.4&r2=1.5 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/parser/ MethodScopeMember.java.diff?r1=1.1&r2=1.2 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/parser/ StaticImportScope.java.diff?r1=1.4&r2=1.5
Wow, so many things going on lately with this issue, yeah! :-) Will you do some other changes? I can't wait for Tonda to execute a new performance test for comparison.
Please note that Tomas was continuously working on this for several weeks (not just lately). Now he just updated this issue with some changes that he made over time to improve the code completion performance. We expect further improvements the week after next (when he returns from his vacation).
Great, thanks for the effort, it's really appreciated.
Great to see the commits! I am on vacation next week. I will measure it right after I get back to work.
added fast check for presence of TypeParameters attached to CallableFeature - this avoids (in most cases) constructiong of LightAttrList and its iterator Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/CallableFeatureImpl.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/CallableFeatureImpl.java,v <-- CallableFeatureImpl.java new revision: 1.22; previous revision: 1.21 Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/ParameterizedTypeImpl.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ ParameterizedTypeImpl.java,v <-- ParameterizedTypeImpl.java new revision: 1.22; previous revision: 1.21 Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/SemiPersistentElement.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ SemiPersistentElement.java,v <-- SemiPersistentElement.java new revision: 1.70; previous revision: 1.69
getSourceElementIfExists() now checks if cls is source element Checking in src/org/netbeans/modules/javacore/JMManager.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/JMManager.java,v <-- ExclusiveMutex.java new revision: 1.99; previous revision: 1.98 done
remember found resource inside MDR transaction http://java.netbeans.org/source/browse/java/editor/src/org/netbeans/modules/editor/java/ JMIUtils.java?r1=1.14&r2=1.15 http://java.netbeans.org/source/browse/java/editor/src/org/netbeans/modules/editor/java/ NbJavaJMISyntaxSupport.java?r1=1.5&r2=1.6
getFeatureAtPos() improved so that it does not iterate through imports and transient elements of java class and resource Checking in src/org/netbeans/modules/editor/java/NbJavaJMISyntaxSupport.java; /cvs/java/editor/src/org/netbeans/modules/editor/java/NbJavaJMISyntaxSupport.java,v <-- NbJavaJMISyntaxSupport.java new revision: 1.7; previous revision: 1.6 done
unnecessary call to getType() removed Checking in src/org/netbeans/modules/editor/java/NbJMIResultItem.java; /cvs/java/editor/src/org/netbeans/modules/editor/java/NbJMIResultItem.java,v <-- NbJMIResultItem.java new revision: 1.10; previous revision: 1.9 done
getImportedClass() optimized to avoid getImportedNamespace() and imports list iterator Checking in src/org/netbeans/modules/editor/java/JMIUtils.java; /cvs/java/editor/src/org/netbeans/modules/editor/java/JMIUtils.java,v <-- JMIUtils.java new revision: 1.16; previous revision: 1.15 done
NaturalMemberNameComparator.compare() improved to avoid get() on List of parameters; comparison of JavaClasses fixed Checking in src/org/netbeans/modules/editor/java/JMIUtils.java; /cvs/java/editor/src/org/netbeans/modules/editor/java/JMIUtils.java,v <-- JMIUtils.java new revision: 1.19; previous revision: 1.18 done
findFeatures() speed-up - move constant expressions out of loops, use toArray() to get features of the JavaClass, match names only if necessary Checking in src/org/netbeans/modules/editor/java/JMIUtils.java; /cvs/java/editor/src/org/netbeans/modules/editor/java/JMIUtils.java,v <-- JMIUtils.java new revision: 1.20; previous revision: 1.19 done
I remeasured current trunk (200503242007) and also 4.0 FCS. On the slow W2K machine as usual, using jdk 5.0. Showing code completion for "System.": trunk: 1141ms, 781ms, 718ms (4.0: 1484ms, 844ms, 718ms) Showing code completion for "View." (open project's class): trunk: 4547ms, 985ms, 906ms (4.0: 1781ms, 1078ms, 1188ms) Showing code completion for "import java.": trunk: 906ms, 766ms, 750ms (4.0: 937ms, 656ms, 672ms) Showing code completion for "new ": (this has to be invoked explicitly by CTRL+SPACE because otherwise it waits for at least one letter as a prefix for the class name) trunk: 14140ms, 9297ms, 8250ms (4.0: 6969ms, 3484ms, 3188ms) Showing code completion for "Mode m; m.": trunk: 969ms, 812ms, 781ms (4.0: 875ms, 750ms, 765ms)
Performance of some consistency and other checks we needed to add in 4.1 significantly optimized. After this change the code completion of "new " is faster than in 4.0 (running with the same parameters in netbeans.conf). Checking in src/org/netbeans/mdr/storagemodel/StorableObject.java; /cvs/mdr/src/org/netbeans/mdr/storagemodel/StorableObject.java,v <-- StorableObject.java new revision: 1.43; previous revision: 1.42 done Checking in src/org/netbeans/modules/javacore/ClassIndex.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/ClassIndex.java,v <-- ClassIndex.java new revision: 1.33; previous revision: 1.32 done Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java,v <-- ResourceImpl.java new revision: 1.77; previous revision: 1.76 done Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/SemiPersistentElement.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/SemiPersistentElement.java,v <-- SemiPersistentElement.java new revision: 1.71; previous revision: 1.70 done
Yes, there is a significant improvement in the "new |" case. I measured almost identical times for this case in trunk now as in 4.0. Other cases have shown also a slight difference, but both increases and decreases, so these may be only measurement variations. Showing code completion for "System.": trunk: 1422ms, 672ms, 765ms Showing code completion for "View." (open project's class): trunk: 3485ms, 1109ms, 968ms Showing code completion for "import java.": trunk: 890ms, 750ms, 797ms Showing code completion for "new ": (this has to be invoked explicitly by CTRL+SPACE because otherwise it waits for at least one letter as a prefix for the class name) trunk: 6937ms, 3797ms, 3093ms Showing code completion for "Mode m; m.": trunk: 907ms, 781ms, 860ms
Regression in the first invocation of a code completion fixed. Showing code completion for "View." is faster. Checking in MergedClassPathImplementation.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/classpath/ MergedClassPathImplementation.java,v <-- MergedClassPathImplementation.java new revision: 1.23; previous revision: 1.22 done
I remeasured current trunk again (200503301800). On the slow W2K machine as usual, using jdk 5.0. We are now equal to 4.0 or even better than 4.0 in some cases! Showing code completion for "System.": trunk: 1110ms, 719ms, 703ms (4.0 was: 1484ms, 844ms, 718ms) Showing code completion for "View." (open project's class): trunk: 1672ms, 1093ms, 1016ms (4.0 was: 1781ms, 1078ms, 1188ms) Showing code completion for "import java.": trunk: 875ms, 734ms, 765ms (4.0 was: 937ms, 656ms, 672ms) Showing code completion for "new ": (this has to be invoked explicitly by CTRL+SPACE because otherwise it waits for at least one letter as a prefix for the class name) trunk: 5172ms, 2719ms, 5171ms (4.0 was: 6969ms, 3484ms, 3188ms) Showing code completion for "Mode m; m.": trunk: 907ms, 797ms, 797ms (4.0 was: 875ms, 750ms, 765ms)
Build 200503301800 Performance in general on a moderatly slow W2K machine much improved. However there still seems to be a problem with remembering items already encountered. Here is what I'm seeing: 1. First time hitting an item the CC window takes a couple of seconds to come up. 2. Next few times hitting that same item the CC window comes up very quickly (near the popup delay setting?). 3. At random(?) intervals however the editor seems to drop back to the first time scenario. 4. CC Window still fails to come up once in a while. The last two may be related to syntax checker operations. Seems to be some corrolation between the syntax checker doing a extended search down the classpath and 4 in particular. For example I have an item called "req" (HttpServletRequest type). I'll type several lines with "req." and the CC window comes up and responds normally. However, if syntax checker triggers in the middle of me typing ("req.s" as an example) the CC window doesn't come up. If I delete the period and retype it I do get the CC window after some delay (first time scenario?).
Thanks for the feedback! I think number 4 is filed as a separate bug - issue 57435.
waiver has been approved by NetCAT4.1
Adding 4.1_WAIVER_APPROVED keyword.
Code completion was partially rewritten for 5.0. There are speedups in some code-completion workflow scenarios, this is without a doubt. For example when you continue in typing and the list of suggested items has to be narrowed. This is fine. However, to be able to close this bug I remeasured the same scenarios as you can see above, with 5.0beta, 4.1 final and 3.6 final on the same old HW (the Dell Precision 220, PIII 800MHz, 512MB RAM, W2K). 1) Showing code completion for "System.": 5.0beta: 1343ms, 641ms, 610ms 4.1: 1468ms, 703ms, 688ms 3.6: 2688ms, 766ms, 578ms 2) Showing code completion for "View." (open project's class): 5.0beta: 3422ms, 844ms, 781ms 4.1: 3750ms, 1015ms, 875ms 3.6: 610ms, 562ms, 563ms 3) Showing code completion for "import java.": 5.0beta: 578ms, 562ms, 547ms 4.1: 750ms, 657ms, 672ms 3.6: 594ms, 562ms, 547ms 4) Showing code completion for "new ": (invoked explicitly by CTRL+SPACE) 5.0beta: 6641ms, 3234ms, 3109ms 4.1: 5281ms, 3688ms, 3547ms 3.6: 375ms, 140ms, 62ms 5) Showing code completion for "Mode m; m.": 5.0beta: 625ms, 578ms, 578ms 4.1: 828ms, 719ms, 687ms 3.6: 578ms, 578ms, 563ms With 5.0 we are almost at the same numbers as with 4.1. And as you can see we are still very far from 3.6 completion speed. Especially for the cases 2) and 4) we are very slow now (3.6 was pretty fast). So, for 5.0, my suggestion is to focus more on code-completion speedup now. Otherwise this issue will have to be waived again (for third time in a row!) which is bad.
Why do you think that data are the same as in 4.1? In test scenarios (from your measurements) 1) is the same or faster than 3.6, little bit faster than 4.1, 2) first invocation is still really slow, other ones are IMO acceptable, all little bit faster than 4.1, 3) is the same as in 3.6, notably faster than 4.1, 4) is still really slow, but IMO this is the most obscure usecase in test scenarios (thousands classes are offered in CC). There is regression against 4.1 in first invocation, 5) is almost the same as 3.6, faster then 4.1. We still want to investigate current state and do improvements, but we do not expect major improvements. But current state seems to me better than 4.1 and really near to 3.6 in most scenarios. Do tests run still on JDK 1.4.2?
We have investigated this again. During first invocation in 'new |' we saw one full GC in manual testing which affects result, it costs about 37% of time on my machine. Unfortunately currently we do not know how to reduce data volume during this operation. Other think is, that tests are running on JDK 1.5. The number of classes available on JDK 1.5 is more than 6000, on JDK 1.4.2 it is about 4200 classes. I do not know about number of offered classes in 3.6. Bear in mind, that current offered classes are correct, this was not always true in 3.6. We tried also to optimize JMIUtils.isTopLevelClassAccessible() and JMIUtils.isAccessible(), which was reported by OptimizeIT as time consuming. Unfortunately this does not help.
FWIW, a fix for issue 65135 might be a minor help - I found that simply pressing the down arrow in the code completion dialog caused every jar on the boot class path and many files in the current project to be stat'd, which is a blocking IO operation (File.isDirectory()/converting File -> URI -> URL is the culprit).
To Pavel / Tomas: perhaps we should try to run profiler with memory profiling on to check for allocation hot spots and cut the momory requirements by small pieces. I know that it is slow and painfull at this stage. To Tim: File.isDirectory is a good starting point. I am afraid there are many other similar method that touches I/O. Wrt case caching of files that belong to platform classpath (binary/source/javadoc) might save some conversion and probably is one of the things called here. If Pavel (or anyone) wants to try agent attached to bug you pointed to) I am ready to help. It can help us to narrow down who calls these method during this activitity.
Finally closing this issue as fixed. The following low-level and high-level improvements were done for 5.0 which significantly improved CC usability and responsivness. Separate issues will be filed for the test cases 2) and 4) above. *) sort new objects according to MOFID - this greatly improves hits/miss ratio in FileCache since there is a lots of cases where objects are read sequentially http://mdr.netbeans.org/servlets/ReadMsg?list=cvs&msgNo=1516 *) primary index of btree storage improved. Btree storage now uses special PrimaryIndex instead of SinglevaluedBtree index. This PrimaryIndex is much smaller and faster. http://mdr.netbeans.org/servlets/ReadMsg?list=cvs&msgNo=1519 *) the result list shrinking is implemented in java completion i.e. when the user types within an identifier the query is only evaluated once at the begining and after subsequent characters the non-matching items from the initial results are eliminated but there are no JMI queries performed. This speeds up the java completion considerably. *) the new API is fully optimized for asynchronous computation so there should no longer be any blocking of user's typing. *) completion delay constants were tuned for better responsiveness - issue 67216
Issue 67902 and issue 67903 were filed for the test cases 2) and 4).
Reorganization of java component