This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Now, code model implementation uses a lot of memory. For example, mysql project needs about 550 Mb to be completely parsed. This is too much. The amount of memory used be code model should be decreased.
Some work was done trying to fix http://www.netbeans.org/issues/show_bug.cgi?id=83993: several prototypes were added to reduce the amount of used memory: 1) APT related: - after using token stream from APT we dispose unnecessary tokens and leave only APT structure (this gives us reducing from 51 to 46Mb - 10%) - responsible flag is "apt.dispose.token" - this approach is not safe, because by design APT supposed to be immutable, but now we parse only once => it is OK 2) String objects related: - use shared string Map for all APT tokens and CsmAST objects - this costs reducing in memory from 51 to 36Mb - 30% - responsible flag is "apt.share.text" So, as result we reduced by about 40% and I turned on "apt.share.text" (30% reducing) flag. What can be done additionally: - we can merge or even remove all non-structure responsible APT nodes from APT tree, the same note as above - APT is immutable - we can remove fake AST part, because now AST structure takes huge memory and it hanges in lazy parts of model
*** Issue 83993 has been marked as a duplicate of this issue. ***
*** Issue 84512 has been marked as a duplicate of this issue. ***
Sevral things done to achieve current memory usage of 490Mb for MySQL: 1. Usage of APTLight (Made by VladimirV) 2. Removal of unneded endOffset field from APTConstTextToken (gives ~ 5%) 3. Usage of empty Tokens for comments 4. No copy of macro maps, just creating a tree of snapshots (gives ~30%) 5. Store compound statement bodies as token list, not AST (gives ~15%)
Since this is dup of P1 bug this should be P1 too. May I ask how did you verify your improvements? Do you really think that 550MB -> 490MB improvement mean that this is resolved problem? Comment about APTStringManager: this class is broken. You use WeakHashMap where only key is weakly referenced (file:///usr/local/jdk1.5.0_docs/docs/api/java/util/WeakHashMap.html) and because you use the same object as a key and and value your key is always strongly referenced. So I guess it does not work as you expect. The synchronization also seems to be suspicious as the map content can be mutated in get() too (GC thread can work at any time though it you couldn't notice so far).
Sorry, I have forgotten to write initial metrics: when I started this issue, MySQL were eating 1 Gb of memory. So we have improved this number to 490Mb. Vladimir Kvashin - the reporter of the issue, said that number of 550 Mb was measured on the MySQL which was not fully configured and so not all files were parsed. He agreed with the current initial number of 1Gm. So I'm closing this issue. Also thank you for comments on the APTStringManager implementation, I've forwarded them to the developer responsible for this class.