This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 268108 - Performance issue - URL LuceneIndexFactory.getIndexFolder() takes 10% of RepositoryUpdater.worker
Summary: Performance issue - URL LuceneIndexFactory.getIndexFolder() takes 10% of Repo...
Status: NEW
Alias: None
Product: editor
Classification: Unclassified
Component: Parsing & Indexing (show other bugs)
Version: Dev
Hardware: PC Windows 7
: P3 normal (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-20 08:02 UTC by NukemBy
Modified: 2016-11-24 10:43 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
getIndexFolder - selfprofiler (129.31 KB, image/jpeg)
2016-09-20 08:02 UTC, NukemBy
Details

Note You need to log in before you can comment on or make changes to this bug.
Description NukemBy 2016-09-20 08:02:11 UTC
Created attachment 162122 [details]
getIndexFolder - selfprofiler

Take a look onto attached screenshot of self-profiler - these 10% can be avoided completely.

Let's start analysis at 

    parsing.indexing\src\org\netbeans\modules\parsing\impl\indexing\lucene\ <<LuceneIndexFactory.java>> 

    @NonNull
    private URL getIndexFolder (@NonNull final FileObject indexFolder) throws IOException {
        assert indexFolder != null;
        final String indexVersion = Integer.toString(VERSION);
        URL result = Util.resolveFile(FileUtil.toFile(indexFolder), indexVersion, Boolean.TRUE /* isDirectory*/);
        final String surl = result.toExternalForm();
        if (surl.charAt(surl.length()-1) != '/') {       //NOI18N
            result = new URL(surl+'/');  //NOI18N
        }
        return result;
    }

here - Util.resolveFile() is always called with 'isDirectory = Boolean.TRUE'

   Side note: I think it is better to make 'indexVersion' as a derived string constant - to avoid repetitive conversions 
      final String indexVersion = Integer.toString(VERSION);
      ->
      private final static String VERSION_S = Integer.toString(VERSION);

Next it goes into openide.util\src\org\openide\util\ <<BaseUtilities.java>>

    public static URI toURI(File f) {
        URI u;
        if (pathToURISupported()) {
            try {
                u = f.toPath().toUri();
            } catch (java.nio.file.InvalidPathException ex) {
                u = f.toURI();
                LOG.log(Level.FINE, "can't convert " + f + " falling back to " + u, ex);
            }

    which (1) goes into overriden 
        org.netbeans.modules.parsing.impl.indexing. <<Util$FastFile.toPath>> (Util.java:381)
    which always throws exceptions when 'isDirectory != null'  
    (throwing/catching exceptions is rather expensive operation - better be avoided when can be handled by simple 'if')

        @Override
        public Path toPath() {
            if (isDirectory != null) {
                throw IP;
            } else {
                return super.toPath();
            }
        }

    and (2) next in a 'fallback catch' it resorts to java.io.File.toURI(), which internally does following:

        public URI toURI() {
            try {
                File f = getAbsoluteFile();
                String sp = slashify(f.getPath(), f.isDirectory());
                if (sp.startsWith("//"))
                    sp = "//" + sp;
                return new URI("file", null, sp, null);
            } catch (URISyntaxException x) {
                throw new Error(x);         // Can't happen
            }
        }

        - checks 'f.isDirectory()' - very expensive operation, see
          https://netbeans.org/bugzilla/show_bug.cgi?id=65135

        - does plenty of inefficient 'string += string' operations

            private static String slashify(String path, boolean isDirectory) {
                String p = path;
                if (File.separatorChar != '/')
                    p = p.replace(File.separatorChar, '/');
                if (!p.startsWith("/"))
                    p = "/" + p;
                if (!p.endsWith("/") && isDirectory)
                    p = p + "/";
                return p;
            }

    ... and taking all that into account I would avoid using File.toURI() if possible

Next it does following:

        final String surl = result.toExternalForm();
        if (surl.charAt(surl.length()-1) != '/') {       //NOI18N
            result = new URL(surl+'/');  //NOI18N
        }

     - result.toExternalForm(); - is also rather complex because covers all general cases
     - additional check for "charAt(-1) != '/'" actually not needed because it will always be generated for folders in File.slushify();

Note: indexing.Util.resolveFile() internally calls BaseUtilities.toURI().toURL(), and URI.toURL() has following implementation

    public URL toURL()
        throws MalformedURLException {
        if (!isAbsolute())
            throw new IllegalArgumentException("URI is not absolute");
        return new URL(toString());
    }

    - this effectively causes back-parsing of 'string' to URL - plenty of additional code


*********************************** FINALLY ********************************************

... all that long way generation of simple URI string

    from    C:\NetBeans\workspace\dev-1\cache\index\s696\java\15
    to ->    "file:/C:/NetBeans/workspace/dev-1/cache/index/s696/java/15/1/"

takes 10+% of the scanner process.

Assuming that input parameter 'FileObject indexFolder'

    - is ALWAYS folder
    - is ALWAYS in normalized/canonical form

    it is safe to generate URL manually and remove that 10+% of CPU usage

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
    private final static String VERSION_S = Integer.toString(VERSION);
    
    @NonNull
    private URL getIndexFolder (@NonNull final FileObject indexFolder) throws IOException {
        assert indexFolder != null;
        
        String path = indexFolder.getPath();
        
        if( File.separatorChar != '/' ) 
            path = path.replace(File.separatorChar, '/');
        
        URL result = new URL("file", "", '/' + path + '/' + VERSION_S + '/');
        
        return result;
    }