This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | Contention between "switch to branch" and java source model updates causes excessive delay | ||
---|---|---|---|
Product: | java | Reporter: | _ pcw <pcw> |
Component: | Unsupported | Assignee: | Tomas Zezula <tzezula> |
Status: | RESOLVED FIXED | ||
Severity: | blocker | CC: | issues, johnjullion, msandor, phrebejk, ppis, rmatous |
Priority: | P2 | Keywords: | PERFORMANCE |
Version: | 6.x | ||
Hardware: | Sun | ||
OS: | All | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: |
thread dump 1/2 way through
threaddump 3/4 way through. |
Description
_ pcw
2007-05-24 22:59:00 UTC
Created attachment 42755 [details]
thread dump 1/2 way through
Created attachment 42756 [details]
threaddump 3/4 way through.
I happened to grab a couple of thread dumps while this action was running. Maybe the will help in fixing it. Also, I would assume that many other block CVS operations might suffer from this, in particular Merge Changes, but possibly Update and others. Marking with keyword. I can see from your stack traces that in both cases CVS is updating (merging) changes to affected files and java/source and java/project modules are listening for changes in those files. In CVS there is little to improve since the files just have to be changed. Reassigning to java for evaluation. /cvs/autoupdate/ui/src/org/netbeans/modules/autoupdate/ui/Bundle.properties,v new revision: 1.17; previous revision: 1.16 /cvs/autoupdate/ui/src/org/netbeans/modules/autoupdate/ui/UnitTab.form,v <-- new revision: 1.9; previous revision: 1.8 /cvs/autoupdate/ui/src/org/netbeans/modules/autoupdate/ui/UnitTable.java,v <-- new revision: 1.7; previous revision: 1.6 /cvs/autoupdate/ui/src/org/netbeans/modules/autoupdate/ui/UnitTab.java,v <-- new revision: 1.20; previous revision: 1.19 Sorry, my commit mentioned in my last comment wasn't intended to fix this issue. So, reopening again. Maybe CVS writes file by file and Java tries to parse them one-by-one (incl. dependencies)? Would be probably better if we could process the files once after the checkout finishes. The problem is that CVS fires change file by file, java module cannot expect that CVS will change another file in some reasonable time frame. To make this issue fixable on the java side the CVS has to do the update inside the atomic action. Change the update to use an atomic action and reassign it back I will do the rest. Thanks. Citing from AtomicAction javadoc "the action should not take a significant amount of time, and should finish as soon as possible". CVS Updates can take *huge* amount of time and there may be lot of file operations involved. However, I experimented running CVS actions inside an AtomicAtion wrapper but this is causing trouble because we selectively turn filesystem event handling on and off during the update operation and this approach cannot be applied if all events should come at once. I suggest you just schedule the compiler so that it does not react to change events as they happen. The compiler has to react to events. The only one who knows about the update (producing successive events) is versioning module. The compiler cannot be started when the versioning finished since it apparently doesn't know it. Event sliding task doesn't work for versioning since there is no deterministic delay. Everyone causing events is responsible for the granularity of them. Are you saying that any module that may be modifying "more than expected" number of java files has to wrap all its code into FileSystem.AtomicAction just to make java parser usable? Responsibility is on the listener side, event producers have no idea who may be listening and what listeners do in response. If you are unable to process events on the fly then do NOT listen to them or stack them for later processing. We may continue offline if you still think CVS is to blame. This is exactly what the java parser is doing, it do update in another thread after some timeout. The problem with CVS update is unpredictability of the events. The only one who knows about the start and end of the CVS update is cvs module, right? So, it either has to delay the events and fire them after the update or it should provide API to notify about the cvs end. The general comments about the events production: It is responsibility of the event producer to produce the reasonable number of events since there may be clients which cannot be performed asynchronously. Reassigning back. What can be done in CVS is to stop using FileObject for all file operations during CVS actions - this could lead to problems if some modules rely on filesystem notifications. I support Tomas Z. It is complete nonsense that this can be hanled on the side of java parser. If there is a branch switch in a module lot other modules depend on and when the CVS operation is firing after arbitrary time one event after other the parser will have to update all the dependent projects all the time. If java gets no info from CVS then it hardly can do anything about it. And even if there would be some ugly way how to workaround this issue then it would have to be done not only in Java but in all modules which need to update something after a file change. The mechanism should obviouy rather be implemented in one place (CV, FileSystems) rather than in every listener. We agreed with Maros to track this issue on the java side. I have implemented events coalescing into the RepositoryUpdater (see issue #110171), which should ensure that if many files are changed fast, they will be compiled using one instance of the parser (per source root, of course). This should lead into much better performance in this case too. Could someone test it? Thanks. As far as I remember Jarda had promised an API to signal a start and an end of related events like VCS update. |