This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | [60cat] Package discovery is slow and does not get cached between invocations | ||
---|---|---|---|
Product: | java | Reporter: | wobster <wobster> |
Component: | Project | Assignee: | Milan Kubec <mkubec> |
Status: | RESOLVED FIXED | ||
Severity: | blocker | CC: | anebuzelsky, issues, jglick, jkovalsky, pjiricka, ttran |
Priority: | P2 | Keywords: | PERFORMANCE |
Version: | 6.x | ||
Hardware: | All | ||
OS: | All | ||
Issue Type: | DEFECT | Exception Reporter: |
Description
wobster
2007-09-30 20:25:35 UTC
*** Issue 117180 has been marked as a duplicate of this issue. *** I believe that Milan is maintaining the PackageView(Children). Right From the dumps attached to issue #117162 it seems that it's caused by excludes. Jesse, do you have any idea how to make includes/excludes faster? Related issues are issue #117169 and issue #117162. No, I don't know of any particular way to make things faster, unless profiling turns up some microoptimizations. Persistent caching might help but this seems hazardous. Exactly which "timestamps" you need to record is very likely dependent on operating system and perhaps even filesystem type. Even if you got it correct, it is not obvious to me that looking up the timestamps after a restart would be any faster than traversing the package structure is now. I would recommend spending effort instead on making an uncached scan faster. It is possible PackageViewChildren would run faster if specialized to use java.io.File in case the source group root were found to be a plain disk directory (not e.g. ZIP archive). This would certainly save some CPU time (the FileSystem layer adds a lot of overhead); it might or might not save I/O time, which I would guess is the blocking factor for most people. I'm not sure what the relationship to includes/excludes is supposed to be. Even without excludes you still need to show a package tree in basically the same way. Checking against the filter adds a bit of overhead but I tried to keep this minimal; normally it is a match of a subpath string against a precompiled regexp. If java.io.File-based scanning is to be used it is possible that an optional extension interface to SourceGroup would be needed which matches containership against a String relative path rather than FileObject. *** Issue 117162 has been marked as a duplicate of this issue. *** *** Issue 117169 has been marked as a duplicate of this issue. *** *** Issue 119670 has been marked as a duplicate of this issue. *** Increasing priority because of duplicate issue #119670. It seems that the delay may happen either if there is lot of files in package (thousands) or user expands large package that is on network drive. Other thread dumps are in the duplicate issue #119670. Profiling showed that the problem can be indeed in includes/excludes, more precisely in SourcesHelper.contains() method calling SharabilityQuery. Jesse could you provide some hint about the impl. and if it can be somehow optimized? I don't think we can fix this now and given the fact that it affects only very large packages (thousands of files) and also when the package is on network drive, we could probably ask for waiver for 6.0. I already disabled the nested project check in SH.SourceRoot.Group.contains for typed source roots; perhaps the call to SQ could also be done only for untyped roots, under the assumption that most unsharable subentries are going to be only in the generic source group, while unsharable subentries under a typed root is technically legal but in practice unlikely. The call to ProjectManager.isProject could probably also be done conditionally only for untyped source roots, though this might not provide significant benefit. I forgot to put the right kw. I will try those optimizations. wobster, please could you try the same use case on new builds of NetBeans 6.1? Some optimizations were done on filesystem layer that could potentially improve the performance of the operation. Thanks. Fixed by calling SharabilityQuery only for untyped source roots as suggested by Jesse. Times of node expansion are now better. http://hg.netbeans.org/main/rev/bff2c106def1 |