It looks like there is a major problem with excessive memory usage during background project scanning.

I run Netbeans with a very large heap, 3.2Gb (-Xmx3200m). But if I do a compile of my project (which is very large [approx 1800 files] and which uses most of that heap) when the IDE decides to do a background scan of projects (which are very large), then I reliably run out of heap space. Not only that but the entire system slows down, screen don't redraw and compiles that usually take 2 min start taking 6+ minutes because the IDE is in permanent GC mode.

Here is the message from the log:

[org.netbeans.modules.java.source.indexing.JavaIndex]: Not enough memory to compile folder: /scratch/tbrussea/src/alexandria/cycorp/cyc/java/tool/subl/src.
WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread org.netbeans.modules.sampler.InternalSampler@4353357f
WARNING [org.netbeans.core.TimableEventQueue]: no snapshot taken

I think that background scanning of projects should not be started while a project is compiling (or any other large task -- profiling maybe). Also, if another large task is started, then background scanning should be halted and resumed when the large task is finished. Alternatively, project compilation should happen in it's own JVM.

Also, if I look at the heap during background scanning it allocates tons of garbage and runs up the heap size very quickly. I'm fairly certain that this is a new change since background scanning was introduced.

Right now, background scanning of projects is vastly worse for me than the old (and merely annoying) scheme of being forced to wait. I'd dearly love an options that returns the IDE to the good old days of merely annoying.