This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | IDE should better handle large files (was: IDE crashes when opening 27MB file) | ||
---|---|---|---|
Product: | platform | Reporter: | gaikokujin <gaikokujin> |
Component: | Text | Assignee: | Petr Nejedly <pnejedly> |
Status: | RESOLVED DUPLICATE | ||
Severity: | blocker | CC: | issues, jchalupa, jlahoda, jtulach, pzajac |
Priority: | P3 | Keywords: | PERFORMANCE |
Version: | 3.x | ||
Hardware: | All | ||
OS: | All | ||
Issue Type: | TASK | Exception Reporter: | |
Bug Depends on: | 27563, 31302, 31303 | ||
Bug Blocks: |
Description
gaikokujin
2002-11-11 20:47:36 UTC
This is a real problem. I'm afraid we have not resources to solve this problem in next release. There exists many related problem. Your workeround is not the best workeround. I am not sure if it is possible to open 300MB file in notepad :-). Did you try to open 27MB file on other IDE? How many peope want to open 27MB file in netbeans. Maybe we can made dirty workaround: Resolve DataOject as TooBigDataObject if FileObject is too big. Loader of TooBigDataObject will have bigger priority then others loaders. It won't be allowed to open the TBDO in editor, etc. Well, it may not be possible to make the IDE open 30MB files in a short term, but it should be possible to handle such cases more nicely. If the IDE informs the user it could not open the file, it should also mention the reasons (e.g. "File too big"). Also, the IDE shouldn't get into an unstable state because of this. Why is the editor tab displayed at all, if the IDE knows it can't open the file? IMO, at least this part should be fixed for the next release. There is more problems then open editor. Solve this problem only for editor case is not best idea. For example I tried to insert new class into big java file :-(... I am not sure if developers has time to fix all these use cases for 4.0 release. Hi, just some notes: The solution with data loaders won't work. Imagine situation where you have a very big executable file. You wish NB to recognize this as a executable file a to be capable of running it, not mark it is "too big". Better solution would be that the owner of the dataobject (that usually writes something like EditorSupport) would handle this situation himself. (Of course, the OpenIDE should help him, ie. provide default implementation.) This would affect only opening in the editor (potentially other operations), but not the rest of the work. This could be relatively simple and straightforward solution. But, it also rises a few question: what will be the treshold between "small" and "large" files? a static treshold (20MB) or something like: if (Runtime.getRuntime().freeMemory() + safe_pool > file_size) { open the file? } else { System.gc(); System.gc(); if (Runtime.getRuntime().freeMemory() + safe_pool > file_size) { open the file? } else { dialog.can_not_open(); } } (But, even this is quite dangerous.) I'll assign this bug to openide/editor. This bug is not only in editor. Ide can crash on any action. Honzo L. why you have reasigned this bug to the editor? I described other example. Solution with Daloader can work. you can add fileobject extension filter (java,txt,xml, ...) for BigDataObject... This solution is not best. It is only workaround. But my solution solves more use cases. Where is the mentioned NullPointerException? It is not seeing in the log. I have to try it out. reassigne to David K., new owner of editor I have changed version from 4.0 dev to S1S 4.2 (Nevada). The problem is that we are using Swing document and it loads the document completely into memory, so there is not much we can do at the NetBeans side. I'm changing type of this issue to TASK. It will be umbrella issue which will depend on list of individual DEFECT or TASK issues which we can fix. I would also like to ask PERFORMANCE team to look at this issue and file additional issues which they discover. Initially I'm adding dependency on: 27563 - memory leak in editor 31302 - save is not enabled on modified large doc 31303 - warn before opening large document Whoever else feel free to add other dependencies. Note that javax.swing.text.Document does not force you to load the whole file into memory - Tim B. has had good success with a memory-mapped document for the output window. You still need to keep a list of line start offsets, but that should be about an order of magnitude smaller than the actual file. Please note that in order to support the not-fully-in-memory documents we will likely be forced to update constraints for certain functionality in the editor. For example current word-match Ctrl-K/L scans the contents of the current doc plus all other opened docs. This needs to be performed in fractions of second. Personally I would first concentrate on being able to swap out non-active docs (including all the corresponding info - parsed stuff etc.) saving memory for the possibly big active file. In the second round I would try to build some sort of MemMappedDocumnet implementation to swap out pieces of the possibly big active doc. As I haven't found an appropriate update for 4.1 on the update center(...) (If there is, can you point me towards it?) Although my source file is only ~800kB (yet), the IDE starts to freeze (up?) quite quickly while editing (if I recall right, "code completion" (I don't think it would have been the "word match" feature) might trigger it ( ... faster?)) I think to recall something of (endless?) "rollback" exceptioning loop from looking in the IDE log (endless log writing?), freezing up (quite) everything, even IDE in "shutdown" (write (update) "classpath info" to persistent storage? never ends?). As Forte for Java 3 (based on Netbeans 3.x?) also starts to slow down after a while (sadly, even does eclipse) and even producing very nasty errors (error on file write -> once -> source file 0 Byte! Luckily for me the auto backup was still there and fortunately not yet 0) it is no real alternative (for the long term future) for me. Of course, I don't really know, if it really belongs to this (bug?) issue. If it really does, I^ll "vote" for this one. No, it's completly different issue. You can file it, performance team will certainly be interested in details. (Add PERFORMANCE keyword and cc issues@performance.netbeans.org) I have duplicated this (although older) issue to issue 98701 since that one contains more current info. *** This issue has been marked as a duplicate of 98701 *** |