This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 183308 - 60s in FileUtil.normalizeFile
Summary: 60s in FileUtil.normalizeFile
Status: VERIFIED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: All All
: P2 normal (vote)
Assignee: Jaroslav Tulach
URL: 69patch1
Keywords: PERFORMANCE
Depends on:
Blocks: 182064 186345 199471 205165
  Show dependency tree
 
Reported: 2010-03-31 20:20 UTC by Exceptions Reporter
Modified: 2011-12-15 11:15 UTC (History)
9 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter: 167423


Attachments
nps snapshot (256.00 KB, application/nps)
2010-03-31 20:20 UTC, Exceptions Reporter
Details
nps snapshot (85.53 KB, application/nps)
2010-04-19 09:55 UTC, jonathanramsey
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Exceptions Reporter 2010-03-31 20:20:07 UTC
Build: NetBeans IDE Dev (Build 100331-eb8d22d7ddd4)
VM: Java HotSpot(TM) Client VM, 10.0-b19, Java(TM) SE Runtime Environment, 1.6.0_04-b12
OS: Windows XP
Maximum slowness yet reported was 7359 ms, average is 7359
Comment 1 Exceptions Reporter 2010-03-31 20:20:13 UTC
Created attachment 96475 [details]
nps snapshot
Comment 2 Jaroslav Tulach 2010-04-02 18:55:09 UTC
Strange snapshot, shows just 203ms.
Comment 3 Tomas Pavek 2010-04-15 14:50:16 UTC
The other snapshots in the report are more interesting, showing long waits (~20s) on native FS calls like WinNTFileSystem.getBooleanAttributes or canonicalize0. Could possibly be worth logging such files.

e.g. http://statistics.netbeans.org/exceptions/exception.do?id=364377
Comment 4 jonathanramsey 2010-04-19 09:55:56 UTC
Created attachment 97603 [details]
nps snapshot

I had just opened NetBeans, and it was loading a multi module Maven project (1 parent, 3 child projects).
I had also opened an XML file by dragging and dropping it into NetBeans, and I did "Remove trailing spaces" and "Format" menu actions on it.
Comment 5 Jaroslav Tulach 2010-04-20 11:52:26 UTC
core-main#ec6876f08f7a
Comment 6 Quality Engineering 2010-04-21 04:38:46 UTC
Integrated into 'main-golden', will be available in build *201004210200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ec6876f08f7a
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #183308: Logging more info about files exceeding 500ms during their normalization
Comment 7 Egor Ushakov 2010-05-06 11:27:38 UTC
I have looots of such warnings, what should I do with them?
Comment 8 Egor Ushakov 2010-05-06 11:55:05 UTC
e.g.
WARNING [org.openide.filesystems.FileUtil]: FileUtil.normalizeFile(/export/work/cnd-main/cnd.modelimpl/build/classes) took 5,623 ms. Result is /export/work/cnd-main/cnd.modelimpl/build/classes
Comment 9 Egor Ushakov 2010-05-07 08:44:50 UTC
either fix it or remove this warning
Comment 10 Jaroslav Tulach 2010-05-07 12:56:39 UTC
I guess the answer to your individual case is clear: stop using NFS. The WARNING is there to identify users who are not using NetBeans according to recommended policy (http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints#Use_fast_filesystem.2Fdisk) - e.g. who have sources on slow disks. In case you can prove that your sources are on local disk, let me know and you'll be allowed to provide more information.

To get on the same wave with you, let me summarize: Either move your sources to local disk or stop complaining.
Comment 11 Egor Ushakov 2010-05-27 12:40:46 UTC
Many clients use findFileObject (even from the AWT) and as soon as it goes to physical fs there will be performance issues... see bug 182064 for example.
Comment 12 Egor Ushakov 2010-05-28 06:23:36 UTC
there is an implementation in java.net.URLStreamHandler (inside parseURL) in pure java, can't we copy it?
Comment 13 Vladimir Voskresensky 2010-06-04 20:11:23 UTC
copying java.net.URLStreamHandler can help Unix users. Cache can help other systems (i.e. on windows canonicalPath is called which is I/O operation)
Comment 14 Vladimir Voskresensky 2010-06-07 11:42:26 UTC
org.netbeans.modules.cnd.utils.cache.CndFileUtils has two methods with cache:
normalizeFile
normalizeAbsolutePath
Comment 15 Jaroslav Tulach 2010-06-16 13:21:06 UTC
OK, caching the FileUtil.normalizeFile: core-main#52bb45cda002
Comment 16 Egor Ushakov 2010-06-16 14:32:11 UTC
Great! Isn't it a good candidate for 6.9.1?
Comment 17 Vladimir Voskresensky 2010-06-16 15:25:36 UTC
Great! Let me mark it
Comment 18 Jesse Glick 2010-06-16 23:20:53 UTC
This patch caused test failures. Fixed in: core-main #8287a13dabb0
Comment 19 Petr Blaha 2010-06-17 11:51:35 UTC
verified
Comment 20 rbalada 2010-06-17 13:05:12 UTC
Transplanted core-main #52bb45cda002 into release691 #81f9a50b71eb and core-main #8287a13dabb0 into release691 #3e2f5b657883
Comment 21 Quality Engineering 2010-06-18 03:20:04 UTC
Integrated into 'main-golden', will be available in build *201006180001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/52bb45cda002
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #183308: Caching result of FileUtil.normalizeFile. Code inspired by CndFileUtils.
Comment 22 giorgio42 2010-06-22 17:09:13 UTC
One observation related to the "don't put your sources on network drives" argument.

More than 10 years ago I was working in team writing a large client server system with clients being on Windows and the server being on Unix. The client was being developed in C++ using VisualStudio.

The source files were all located on Solaris and being accessed through Samba.
VS checks files for external changes every other second, therefore this put quite a load on the file server. BUT: 40 developers were able to work concurrently in this setup.

Therefore my opinion is (with all the progress with HW and networks during the last 10 years) that this argument is simply not valid. If putting sources on a network drive is not possible with NetBeans (like it was with VS more than 10 years ago), it is very simply just broken.