This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 161176 - [67cat] False compilation errors in editor (was: VirtualSourceProvider does not work anymore)
Summary: [67cat] False compilation errors in editor (was: VirtualSourceProvider does n...
Status: RESOLVED WORKSFORME
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 7.0
Hardware: All All
: P2 blocker with 4 votes (vote)
Assignee: Tomas Zezula
URL:
Keywords: REGRESSION
: 160900 164288 164341 165348 166851 167647 167969 168013 168031 168054 168151 168770 (view as bug list)
Depends on:
Blocks: 163156 164288
  Show dependency tree
 
Reported: 2009-03-25 17:25 UTC by Petr Hejl
Modified: 2011-07-12 13:09 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Invalid error icons - there are no compilation or parsing errors (15.79 KB, image/png)
2009-06-11 12:43 UTC, deftex
Details
cache index segment apparently of interest (73.42 KB, application/x-compressed)
2009-07-31 17:37 UTC, jessholle
Details
my VirtualSourceProvider implementation (5.19 KB, text/plain)
2009-07-31 17:45 UTC, jessholle
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Hejl 2009-03-25 17:25:59 UTC
With new Parsing API GroovyVirtualSourceProvider is not called anymore - as a consequence Groovy files are not available
in Java code completion.
Comment 1 Petr Hejl 2009-03-26 10:07:30 UTC
Actually more important thing is that java parser does not know about groovy classes so it is displaying errors in
source files.
Comment 2 Vitezslav Stejskal 2009-03-26 14:43:34 UTC
This sounds rather serious...
Comment 3 Tomas Zezula 2009-03-30 10:08:02 UTC
Not sure how serious it is. As far as I know there are two clients of this API and just one uses it in reality (groovy).
This API is used during index building to allow mixed compilation unit among java and languages compiled into class files.
The only use case for it is this:

java:
class A {
   private B b;
}

groovy:
class B {
  private A a;
}

Comment 4 Petr Hejl 2009-04-21 13:25:59 UTC
VirtualSourceProvider seems to be required even for groovy only code (in situation when there is not 1-to-1 mapping
between file and class and such class is used in other groovy file).
Comment 5 Petr Hejl 2009-05-06 14:52:42 UTC
*** Issue 164341 has been marked as a duplicate of this issue. ***
Comment 6 Petr Hejl 2009-05-11 10:10:47 UTC
Changing the description to reflect the problem user sees.

When groovy class is used in Java source this gives the user false error - this happens even for demo project bundled
with ide.
When user defines multiple classes in one groovy source file and use these in other file he will get compilation error
in the editor.
Comment 7 Petr Hejl 2009-05-11 10:11:29 UTC
*** Issue 160900 has been marked as a duplicate of this issue. ***
Comment 8 Petr Hejl 2009-05-17 19:24:36 UTC
*** Issue 165348 has been marked as a duplicate of this issue. ***
Comment 9 Petr Hejl 2009-05-19 14:10:28 UTC
*** Issue 164288 has been marked as a duplicate of this issue. ***
Comment 10 Jiri Kovalsky 2009-06-02 17:58:21 UTC
This issue was also reported by NetCAT 6.7 participant.
Comment 11 Tomas Zezula 2009-06-02 18:19:37 UTC
I will integrate VSP this week.
Comment 12 Tomas Zezula 2009-06-04 12:51:28 UTC
integrated into jet-main: 6ad317084d5d
Comment 13 Quality Engineering 2009-06-08 18:45:25 UTC
Integrated into 'main-golden', will be available in build *200906081401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/6ad317084d5d
User: Tomas Zezula <tzezula@netbeans.org>
Log: #161176:False compilation errors in editor (was: VirtualSourceProvider does not work anymore)
Comment 14 Vitezslav Stejskal 2009-06-10 21:59:13 UTC
http://hg.netbeans.org/jet-main/rev/4a23160602c7 - this is fixing some problems introduced by 6ad317084d5d. Mainly there
are indexers registered for 'all languages' (aka MimePath.EMPTY) and they should not be considered by the algorithm
determining mime types that each particular indexer is registered for. Without this fix these indexers received *all*
files for every mimetype in the resources map. However even with this fix indexers registered for multiple mimetypes
will receive the same set of files multiple times, which is not very efficient. I'll change this when I rewrite this
whole algorithm for issue #166340.
Comment 15 Petr Hejl 2009-06-11 10:16:26 UTC
*** Issue 166851 has been marked as a duplicate of this issue. ***
Comment 16 deftex 2009-06-11 12:40:20 UTC
Just downloaded & tried 200906110201. While the parser error seems to have been fixed (the is no error in the Java file that incorporates a Groovy object), 
the there are still Red exclamation marks on the project & package icons, indicating a parsing/compilation error that is non-existent.

See attached screenshot...
Comment 17 deftex 2009-06-11 12:43:09 UTC
Created attachment 83442 [details]
Invalid error icons - there are no compilation or parsing errors
Comment 18 Quality Engineering 2009-06-11 18:43:01 UTC
Integrated into 'main-golden', will be available in build *200906111401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/4a23160602c7
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #161176: following up on 6ad317084d5d; ignoring 'all languages' indexers, they are always registered to recieve all the files
Comment 19 Petr Hejl 2009-06-25 14:06:12 UTC
*** Issue 167647 has been marked as a duplicate of this issue. ***
Comment 20 Tomas Zezula 2009-06-28 19:34:15 UTC
The error badges in the editor are not related to this issue which original name was "Missing VSP support" and was renamed to the "False compilation errors in 
editor". Also the error badges on packages are unrelated to the editor.
There are two possible reasons of these error badges.
1) The groovy VSP generates wrong (non valid) java stub - can be handled by java support, I've fill separate issue on it #167756.
2) When the groovy file on which depends a java file is deleted (created) the java support doesn't know about it. There an enhancement filled by madamek on 
groovy for this.
Comment 21 jessholle 2009-07-01 15:52:20 UTC
I have this same issue with my own VirtualSourceProvider implementation plug-in.

I could have sworn this worked at some earlier point in 6.7 development, but it clearly does *not* work in the final 6.7
release.

This is a *very* serious issue for our NetBeans usage -- and I'll have to recommend that everyone here go back to 6.5
until this is fixed.

Are there plans to supply this fix in a near term module / plug-in update from the update center?  [Please.]
Comment 22 Jiri Kovalsky 2009-07-01 17:11:29 UTC
This issue has "67patch-candidate" keyword in the "Status whiteboard" which means, it will be considered for NetBeans
6.7 Update 1 [1].

[1] http://wiki.netbeans.org/NetBeans671
Comment 23 Tomas Zezula 2009-07-01 17:15:12 UTC
In the early state of the NB 6.7, only part of the java module was migrated to the parsing api but there was not the indexing api. The java module used the old 
RepositoryUpdater which had a support for the VirtualSources. Later we have merged the indexing api and migrated java to use it. The new API had no support 
for VirtualSources. I've reintroduced the VirtualSources after the feature freeze of the NB 6.7 release, it's not a part of the NB 6.7. But it will be covered by the 
first NB 6.7 patch. This patch will also contains the fix of #167756 which handles situations when the generated stub has errors.
Comment 24 Max Sauer 2009-07-02 12:20:22 UTC
*** Issue 168013 has been marked as a duplicate of this issue. ***
Comment 25 Petr Blaha 2009-07-02 13:10:40 UTC
v
Comment 26 Petr Hejl 2009-07-02 13:27:07 UTC
*** Issue 168031 has been marked as a duplicate of this issue. ***
Comment 27 pgebauer 2009-07-03 11:56:38 UTC
The fix has been ported into the release67_fixes repository.
http://hg.netbeans.org/release67_fixes/rev/492174fc2c29
Comment 28 Petr Hejl 2009-07-06 18:35:20 UTC
*** Issue 168151 has been marked as a duplicate of this issue. ***
Comment 29 Petr Hejl 2009-07-07 14:58:05 UTC
*** Issue 168054 has been marked as a duplicate of this issue. ***
Comment 30 Geertjan Wielenga 2009-07-09 11:27:06 UTC
Added a note about this issue into the Groovy tutorial: http://www.netbeans.org/kb/docs/java/groovy-quickstart.html
Comment 31 Vitezslav Stejskal 2009-07-13 15:28:41 UTC
http://hg.netbeans.org/release67_fixes/rev/492174fc2c29 does not seem to contain all the changes from 6ad317084d5d and
4a23160602c7. I'm fixing it now.
Comment 32 Vitezslav Stejskal 2009-07-13 16:42:59 UTC
Hopefully this transplants all the changes correctly - http://hg.netbeans.org/release67_fixes/rev/986e07dd5e4f
Comment 33 Jiri Prox 2009-07-16 15:18:17 UTC
v in 6.7.1
Comment 34 Petr Hejl 2009-07-19 09:36:03 UTC
*** Issue 168770 has been marked as a duplicate of this issue. ***
Comment 35 jessholle 2009-07-28 15:52:20 UTC
I am still seeing this error in 6.7.1 (installed fresh as 6.7.1 rather than obtained by updating 6.7) for my plug-in
which implements VirtualSourceProvider.

The same plug-in works flawlessly in 6.5.  Is there something that I need to change -- if not, this bug is most
certainly *not* resolved by 6.7.1.
Comment 36 jessholle 2009-07-28 15:54:30 UTC
I have a good amount of logging in my module -- but the log management module no longer appears to be available for 6.7.
 How does one go about managing NetBeans logging otherwise?
Comment 37 Vitezslav Stejskal 2009-07-29 09:46:01 UTC
"I am still seeing this error in 6.7.1 (installed fresh as 6.7.1 rather than obtained by updating 6.7) for my plug-in
which implements VirtualSourceProvider." - So, does you VirtualSourceProvider get instantiated and called? The groovy
plugin uses VirtualSourceProvider as well and it seems to work ok. 
Comment 38 jessholle 2009-07-29 12:22:42 UTC
It seems that my plug-in is not getting called.  Could this have something to do with lazier plug-in loading in 6.7?

My plug-in is activated and the IDE log does not show any errors associated with it.
Comment 39 Vitezslav Stejskal 2009-07-29 16:50:01 UTC
"It seems that my plug-in is not getting called." - Do you mean that your implementation of VirtualSourceProvider is not
getting called? Is it correctly registered (eg. is it instantiated, but not called)? It should be registered in the
default Lookup (eg. via o.o.util.lookup.ServiceProvider annotation).

Also I'm not sure if this was required prior 6.7, but you should have JavaCustomIndexer registered for your mime type in
MimeLookup. Look at how groovy.editor does it. It adds something like

<file name="JavaIndexer.shadow">
    <attr name="originalFile" stringvalue="Editors/text/x-java/JavaIndexer.instance"/>
</file>

to the Editors/text/x-groovy folder.
Comment 40 jessholle 2009-07-30 20:17:52 UTC
I added more logging to my plug-in and hard-wired the verbosity to Level.ALL (though I did find the log management
plug-in, which really should be part of NetBeans' own module development module set).

In my VirtualSourceProvider implementation, I log "Instantiated" in the constructor and "Translate called" first thing
in translate() [followed by a lot of other logging later in translate()].

The result (with only a couple of projects open and clearing the var directory prior to startup) is:

FINE [com.ptc.rbinfohandler.RbInfoHandler]: Instantiated
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 6 source roots took: 8384 ms

That's it -- note the lack of "Translate called".  My VirtualSourceProvider implementation is called out in
META-INF/services just as it was in 6.5.1.

Is there something else that needs to be done to actually get my VirtualSourceProvider to be called?!?  My
getSupportedExtensions() returns Collections.singleton( "rbInfo" ) -- my comments state that extensions do not include
the "." for this API and that certainly worked in 6.5.1.  My index() returns "true".  I've tried normal, autoload, and
eager settings -- which made no difference.

Something is screwy here as this worked great in 6.5.1.
Comment 41 Tomas Zezula 2009-07-31 10:02:51 UTC
There is one important change.
The NB 6.7.x is using a common infrastructure for parsing and indexing (the parsing API). In the 6.5 the creation of set of files to be scanned was done by 
java module. In the 6.7.x the set is created by parsing API and passed to the java. As your file is not known for the java indexer the parsing API is not 
passing it to java module. You have to tell the parsing API that your files should be handled by the java indexer.
This is done by registering the java indexer for your files:

 <folder name="Editors">
        <folder name="text">
            <folder name="your-mime-type">              
                <file name="JavaIndexer.shadow">
                    <attr name="originalFile" stringvalue="Editors/text/x-java/JavaIndexer.instance"/>
                </file>
            </folder>
        </folder>
    </folder>

Also you have to have mime type recognizer for your file type associating your mime-type.
If you have any problems feel free to reopen.
Comment 42 jessholle 2009-07-31 12:03:08 UTC
Thanks for the help.  How do I add a "mime type recognizer" for my file type?
Comment 43 jessholle 2009-07-31 12:04:24 UTC
Also are these changes compatible with 6.5?  Or do I need separate versions of my plug-in for 6.5 and 6.7 now?
Comment 44 Tomas Zezula 2009-07-31 12:22:18 UTC
You have to provide mime type recognizer. The simplest way is to use extension based mime type recognizer.
Here is an example:
1st) You have to register the recognizer in the layer.xml
<folder name="Services">
        <folder name="MIMEResolver">
            <file name="MyMimeResolver.xml" url="MyMimeResolver.xml">
                <attr name="position" intvalue="300"/>
            </file>
        </folder>
</folder>

2nd) You have to create MyMimeResolver.xml file in the same package as you have a layer.xml

<!DOCTYPE MIME-resolver PUBLIC "-//NetBeans//DTD MIME Resolver 1.0//EN" "http://
www.netbeans.org/dtds/mime-resolver-1_0.dtd">
<MIME-resolver>
    <file>
        <ext name="my_file_extension"/>
        <resolver mime="text/x-myfiletype"/>
    </file>
</MIME-resolver>


Unfortunately the changes are not compatible with 6.5. This was a reason why I didn't want to put the VirtualSourceProvider into the API, as I was not able 
to keep the compatibility.  The module with the registered java indexer (6.7) should work even in 6.5. The IDE may complain in log about broken link as the
java indexer doesn't exist in 6.5. But it should work.
Comment 45 jessholle 2009-07-31 15:27:23 UTC
Thanks.

This all now works great with one notable exception.  I can't "Go To Declaration" or "Go To Source" from usages of the
classes produced by my VirtualSourceProvider.  In 6.5, when I did this it would take me to the file from which the given
class was generated in my VirtualSourceProvider.

Any ideas here?
Comment 46 Tomas Zezula 2009-07-31 17:25:34 UTC
Does your VirtualSourceProvider.index() method return true?
If so, it seems that the java module is not able to resolve the file. Can you attach one lucene index containing such a virtual source?
I will look on the resource path.
Comment 47 jessholle 2009-07-31 17:29:48 UTC
Yes, my index() returns true (always).

How do I find such a Lucene index to attach?
Comment 48 jessholle 2009-07-31 17:37:32 UTC
Created attachment 85625 [details]
cache index segment apparently of interest
Comment 49 jessholle 2009-07-31 17:42:25 UTC
I attached the cache/index segment that is apparently of interest (as s1.zip).

The entries of interest are clientResource and launcherResource.

I'll attach my VirtualSourceProvider implementation shortly.  It is very short and should simply create a virtual class
"A" for every A.rbInfo (with the appropriate Java package given the placement of rbInfo in the source tree).
Comment 50 jessholle 2009-07-31 17:45:41 UTC
Created attachment 85626 [details]
my VirtualSourceProvider implementation
Comment 51 Tomas Zezula 2009-07-31 17:58:52 UTC
Your VSP seems good.  I've looked into the caches and it seems that the java indexer didn't store the resource name into it.
I've filled an issue #169658 to resolve it.
Comment 52 jessholle 2009-07-31 19:21:24 UTC
Thanks!
Comment 53 Petr Hejl 2009-09-25 10:23:25 UTC
*** Issue 167969 has been marked as a duplicate of this issue. ***
Comment 54 Jeffrey Rubinoff 2011-07-12 12:12:23 UTC
This appears to have regressed in 7.0. I was testing the Introduction to Groovy tutorial, http://netbeans.org/kb/docs/java/groovy-quickstart.html. This has the simple Groovy class GreetingProvider, contents 
class GreetingProvider {

    def greeting = "Hello from Groovy"

}
I'm supposed to be able to call this from a JFrame inside the same package, with the code 
    String greeting = provider.getGreeting().toString();
    jTextField1.setText(greeting);
Compilation fails utterly to know what to do with provider. I cannot reproduce the provider methods with code completion.
Comment 55 Jeffrey Rubinoff 2011-07-12 13:09:24 UTC
So sorry, guys. Exact same app built by Geertjan works on my machine, same groovy jar, everything. Gremlins on my machine, it seems.