This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 122639 - go to file much to slow
Summary: go to file much to slow
Status: RESOLVED FIXED
Alias: None
Product: editor
Classification: Unclassified
Component: Parsing & Indexing (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Vitezslav Stejskal
URL:
Keywords: API, PERFORMANCE
: 135931 139834 147620 156665 160372 161528 (view as bug list)
Depends on: 147620
Blocks:
  Show dependency tree
 
Reported: 2007-11-23 14:00 UTC by cs42de
Modified: 2009-05-07 08:36 UTC (History)
13 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Thread Dump while waitung for Got To File (14.19 KB, text/plain)
2007-11-29 10:11 UTC, cs42de
Details
Thread Dump after waitung for Got To File (9.52 KB, text/plain)
2007-11-29 10:11 UTC, cs42de
Details

Note You need to log in before you can comment on or make changes to this bug.
Description cs42de 2007-11-23 14:00:40 UTC
invoking 'Go to File' takes much to long

.form files should be hidden since there is no differnt behaviour selecting the .form or the .java file

Product Version: NetBeans IDE 6.0 Beta 2 (Build 200710212201)
Java: 1.5.0_10; Java HotSpot(TM) Client VM 1.5.0_10-b03
System: Windows XP version 5.1 running on x86; Cp1252; de_DE (nb)
Comment 1 Jiri Prox 2007-11-26 13:23:58 UTC
Please capture threaddump when IDE has too slow response.
Comment 2 cs42de 2007-11-29 10:07:27 UTC
i added two thread dumps - one while i was waiting and one after

it appears that the performance differs - first time calling 'Go to File' always is slow.
if you call it again it is much faster but when you start coding and doing ohter stuff and you use it after a while
again it is slow again

to me nb 6.0 all in all appears slower and less 'smother'

maybe i just have to get me an new computer (now 2.8 GHz + 1GB)
Comment 3 cs42de 2007-11-29 10:11:00 UTC
Created attachment 53653 [details]
Thread Dump while waitung for Got To File
Comment 4 cs42de 2007-11-29 10:11:59 UTC
Created attachment 53654 [details]
Thread Dump after waitung for Got To File
Comment 5 stardog 2008-02-12 18:39:35 UTC
Please fix this issue soon!  I use this feature constantly throughout the day to locate java classes and I have to 
wait for every search.  I have a Dual Core x64 Windows XP machine with 2 gigs of RAM (and I give 500megs to netbeans), 
and I am using Java 1.6.0_04 so the problem is probably with the search algorithm.  
Comment 6 Petr Hrebejk 2008-02-16 09:00:55 UTC
Well, the performance of go to file. Should be the same as performance of the:
find -name "whatEverFileName". If its slower then there is a problem. If the same it probably wont  be fixed soon.

For "stardog". Using it for locating of classes ins not the best idea. If you want to open class please use "Go To Type"
rather than "Go To File". GTT is much faster, supports features like CamelCase etc. GTT has the shortcut Ctrl+o.



Comment 7 stardog 2008-02-18 14:01:39 UTC
Thanks for taking the time to give some advice, phrebejk.  I am using Go To Type and it is MUCH quicker, and I am very 
happy!
Comment 8 cs42de 2008-02-19 10:38:06 UTC
Using 'Go To Type' makes me happy too!
thx
Comment 9 cbulcu 2008-04-11 23:56:49 UTC
Go To File is too slow indeed. I know that NB is aimed to close the gap to IDEA. The thing is that in IDEA there is no
performance difference between go to type and go to file. So it should be possible to do it in NB too. The problem is
present in NB 6.1, as we all know.

6.1 is a big step in the right direction though. So keep up the very good work, and I barely wait for news about NB 7. :-)
Comment 10 cbulcu 2008-04-12 00:05:33 UTC
Go To File is too slow indeed. I know that NB is aimed to close the gap to IDEA. The thing is that in IDEA there is no
performance difference between go to type and go to file. So it should be possible to do it in NB too. The problem is
present in NB 6.1, as we all know.

6.1 is a big step in the right direction though. So keep up the very good work, and I barely wait for news about NB 7. :-)
Comment 11 madhurtanwani 2008-05-22 15:29:01 UTC
I have similar complaints for the "Go To File" feature. For me, the "Go To Type" does not work quite as well. I have
many files, which I frequently keep switching between. 

My observation is that the go to file feature works acceptably fast, when the number of files to be searched is small -
obviously :). Hence, the suggestion is why not index/hash the files in a project, when it is created (or first time Go
to file is used). With this approach the first time experience might be BAD but subsequent lookups should be very fast.

I also see that people are happy with the Go to type responses. I also observe that if I use "Go To File" for a class
name the lookup takes a long time. But if I use "Go To Type" for the same class name, it appears almost immediately. So
is it, that we are doing some level of caching in the "Go To Type" feature and not in the "Go To File" feature?

I would appreciate if this feature fixed ASAP - would make life much easier!

Thanks, 
Madhur Tanwani
Comment 12 deans4j 2008-07-29 02:07:36 UTC
I'm suffering from this issue in NB 6.1 too. I use this feature quite a lot, and I hate when I have to wait for a big
seconds. I love NB please fix this issue soon. Cheers!
Comment 13 Jan Becicka 2008-09-03 10:44:02 UTC
*** Issue 139834 has been marked as a duplicate of this issue. ***
Comment 14 cbulcu 2008-11-06 23:33:33 UTC
this enhancement would be very welcome for me too. It is very useful for big projects.
Comment 15 cbulcu 2008-11-26 22:44:48 UTC
It's a pain to use this feature :-( . Maybe it's possible to change things before 7. It's very often when a developer
opens non-java files while developing a web application (jsp, js, css, so on, so forth).
Comment 16 Petr Blaha 2008-12-04 13:03:14 UTC
Comment from dafreels:

Basically, since our main applications are web/spring/iBatis based, we really need to search for many different files.
Here is a list of files we use daily in order of importance:

1) *.xml (Spring/iBatis)
2) *.js, *.jsp (Javascript/JSP)
3) *.css (Style sheets)
4) *.tag, *.tld (Custom tag library files)

Basically, the first three are required for the work we do, and the 4th we could probably get by with. Maven is used to
build our projects, so 1 and 4 can be found on the class path. 2 and 3 would be in our web folder.
Comment 17 _ wadechandler 2008-12-04 21:24:41 UTC
@petr
>Well, the performance of go to file. 
>Should be the same as performance of the:
find -name "whatEverFileName"
>. If its slower then there is a problem. 
>If the same it probably wont  be fixed soon.

I wouldn't say that is a fair comparison Petr. find on the operating system level isn't something one necessarily has to
use so often. Where as when you're in your IDE working on a project it is a given that you'll be opening this or that
file possibly many times at different intervals, and you'll become more and more intimate with those files and their
names. As one who works in such an environment gets used to jumping from file to file and changing this or that value
any waiting in that process hurts the user experience versus searching a folder or a disk where one is usually trying to
find where it is. Most folks know where the file is when they really want Go To File to be fast, but they want a shorter
and faster way of getting to it than traversing the UI tree. 

I have been meaning to file something similar. Essentially for Go To Type there is an index which allows these things to
be found faster. Something like that would make Go To File much better. Think about it. You are working in a large
project such as a JSP project. In this project there are possibly hundreds of different XML, Properties, JSP, etc files.
All these things could be made to have the file names indexed and allow searching of a file name very very fast. That
would make working with those type projects just as easy as a Java class library. I have to admit that I was just
working recently, and will be again soon, on a very large web application where this feature was sorely needed. It had a
dramatic negative impact on the user experience.
Comment 18 Tomas Zezula 2008-12-05 13:11:19 UTC
Yes, the comparison is not very fair. The problem should be solved by indexing. Unfortunately currently there is no single place to put the indexing into. 
Each module knows only some roots. This problem as many others can be resolved by the upcoming parsing api which provides single indexing mechanism 
and file name index can be maintained there.
Comment 19 _ wadechandler 2008-12-05 13:40:42 UTC
That will be awesome Tomas to have a single point of entry for indexes. What is the time line for the new parsing API? I
have a couple ideas for other things which that would be handy. Is there an issue to track that API? Thanks.
Comment 20 Tomas Zezula 2008-12-05 14:05:52 UTC
The parsing API is scheduled for NB 7.0, a part of it is already integrated into M1. The question is if all languages will be migrated.


Comment 21 Petr Blaha 2008-12-08 08:38:27 UTC
alastor8472's comment
In our project we use the commercial ATG 2007.1 Suite combined with Spring/Hibernate, thus our file usage is slightly
different:
1) *.java files
2) *.properties files (which are used by ATG for instances/injection of objects)
3) *.jsp files (web content)
4) *.xml files (for Spring/Hibernate)
5) *.js / *.css for our web page developers

For our projects 1, 2, and 3 are the most important files to as cover most of work load. We use Maven for our projects,
but due to the ATG inheritance a slightly different structure, JSP/CSS/JS/XML files are located in a j2ee-app sub directory.
Comment 22 Petr Blaha 2008-12-08 08:40:10 UTC
piastucki's comment:
Our project is a big web application, however it also includes plenty of SQL scripts.
The most popular file types ordered by file count are as follows:
1) *.java
2) *.jsp
3) *.xml
4) *.sql
5) *.js, *.css
6) *.html, *tld, *.properties
Comment 23 Jesse Glick 2009-01-27 17:17:23 UTC
*** Issue 135931 has been marked as a duplicate of this issue. ***
Comment 24 David Strupl 2009-02-07 22:08:45 UTC
*** Issue 156665 has been marked as a duplicate of this issue. ***
Comment 25 Jan Jancura 2009-02-09 13:02:12 UTC
Depends on new Parsing API M2T.
Comment 26 David Strupl 2009-03-13 13:43:19 UTC
Vita promised to fix this one.
Comment 27 Tomas Pavek 2009-03-17 17:58:22 UTC
*** Issue 160372 has been marked as a duplicate of this issue. ***
Comment 28 Vitezslav Stejskal 2009-03-31 14:43:06 UTC
*** Issue 161528 has been marked as a duplicate of this issue. ***
Comment 29 Vitezslav Stejskal 2009-04-03 12:17:20 UTC
*** Issue 147620 has been marked as a duplicate of this issue. ***
Comment 30 Vitezslav Stejskal 2009-04-03 13:46:16 UTC
http://hg.netbeans.org/jet-main/rev/e9facfe91dd2

The IDE now creates file index, which is then used for goto-file dialog. There are however two differences from how
things work before:

1. The indexing infrastructure does not index hidden files (ie. those that VisibilityQuery deems hidden). Therefore the
new goto-file implementation never shows hidden files and no longer provides 'Show hidden files' checkbox. In my opinion
this is not a big deal.

2. The indexing infrastructure scans only roots registered in GlobalPath registry (eg. src, test, web directories in
ordinary java/web projects). But there may be files in the project that are not placed under any of those roots. I don't
have any good example, but I assume those would be various config files for frameworks like spring/hibernate/etc. I
don't know. These files won't show up in goto-file dialog either.

I'm not sure how big a problem #2 is. If finding these files covers more than 50% of goto-file usages then I'll somehow
fix it. But if it's not that common usecase then I'll probably just file P3/ENH and leave it for later.

Any opinions? Thanks
Comment 31 _ wadechandler 2009-04-03 16:28:52 UTC
On #2, will it find files under extra src folders? In the project properties one is able to add other source folders to
their project. Does it also work there? I haven't looked to see if those get added to global or not.
Comment 32 Tomas Zezula 2009-04-03 16:36:01 UTC
Yes, everything what is on the project sourcepaths (compilation units) should be indexed.
Comment 33 Quality Engineering 2009-04-04 07:41:47 UTC
Integrated into 'main-golden', will be available in build *200904040200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/e9facfe91dd2
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #122639 - implementing file index and using it in goto-file dialog
Comment 34 piastucki 2009-04-08 07:49:18 UTC
Go To File is really fast now. Thanks.

Could a way to include/exclude additional selected folders in/from indexing be added?  E.g. Files panel -> folder
context menu (right click) -> Properties -> Indexing (like Sorting)

Comment 35 cbulcu 2009-04-08 09:08:44 UTC
Thanks for the speed :-)

The Go To File doesn't really work when using/not using Case sensitive. Try to play with the characters when searching.
But I think this is a known issue, so I'll be waiting for the final version. :)

Anyway, the speed is what I've been expecting for so long. :)
Comment 36 Tomas Zezula 2009-04-08 09:12:47 UTC
To piastucki:
No it's not possible now, as the file index creation is a part of source root indexing it wouldn't be so easy. The infrastructure will need to run just selected 
indexers (the file names) on given folder but not the others. Otherwise it will slow down the IDE initial scan.
I will look how complicated it will be to add such a feature to RepositoryUpdater.
Comment 37 Petr Jiricka 2009-04-17 17:14:31 UTC
I think there is still a performance problem (introduced when fixing issue 162144), which I reported as a separate issue
162969.
Comment 38 piastucki 2009-05-06 08:00:49 UTC
This plugin seems to work much better than NB built-in Go To File:
http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=16495

Is there anything you can re-use from the plugin?
Comment 39 David Strupl 2009-05-06 10:27:34 UTC
From the plugin description:
"quick finding and opening any file from *main* project"
while the internal implementation searches *all* opened projects. I am not sure whether something can be reused ... did
you mean the features (UI) or the speed? If there is just one project opened is the speed of the plugin better? If you
meant features please file separate ENH requests for each feature you are missing fromt the internal implementation into
category utilities/jumpto. Thanks.
Comment 40 piastucki 2009-05-06 14:25:43 UTC
@dstrupl:
> did you mean the features (UI) or the speed?

SPEED (mainly)

>"quick finding and opening any file from *main* project" while the internal implementation searches *all* opened projects. 

The plugin is really simple (just a couple of classes) and it looks like it can be easily extended to handle as many
open projects as you want.

If there is just one project opened is the speed of the plugin better?

MUCH betetr

Cheers
Comment 41 David Strupl 2009-05-06 14:40:48 UTC
Are you talking about a recent dev build or beta? There was
http://www.netbeans.org/issues/show_bug.cgi?id=162969
which should be fixed in the dev builds (not in beta though).

BTW I have just tried on a today's build and the go to file with just one project opened is really fast with the built
in implementation ...
Comment 42 piastucki 2009-05-06 17:44:07 UTC
I am using build dated 20090502. Of course, Go to File is MUCH faster than in 6.5 or older stable builds, but it is
significantly worse than the performance of Open File Fast plugin. 
I did a simple test:
1) my project contains about 8000 files.
2) I initialize all indexes by opening Go To File and OFF dialogs and entering some pattern
3) I open Go To File dialog and enter the same pattern again - the results are found in about 1 second
4) I open OFF dialog and enter the same pattern - the results are found instantly (I would say less than 200ms)

I think the files are indexed fine by NB. However, it looks like the way the indexes are searched for the given pattern
is slow.
Comment 43 Tomas Zezula 2009-05-06 18:42:58 UTC
The Go to file has only "prefer current project option" - when more project are opened it search in all the projects.
The index is stored in the lucene index. The initialization of lucene index reader (first search) may be slower, but next
calls should be quite fast. The go to file uses generic NB lucene index which is not optimized => the performance can be improved.
Comment 44 Petr Jiricka 2009-05-07 08:28:49 UTC
Another way to speed it up would be through issue 162580 - currently we don't index all files in the project, only those
under the source roots, and for the remaining files we do scan through all files.
Comment 45 Tomas Zezula 2009-05-07 08:36:32 UTC
To Petr J.:  Yes, it's another problem, unfortunately the repository updater who does the indexing does not know anything about the non source folders. We 
talked with Vita about adding these roots to the indexing too. But it would cause more problems than benefits (nesting, running only needed indexer).