This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 152968 - Use old index while downloading new one
Summary: Use old index while downloading new one
Status: RESOLVED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Maven (show other bugs)
Version: 6.x
Hardware: PC Linux
: P3 blocker (vote)
Assignee: Milos Kleint
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-11 16:12 UTC by getaceres
Modified: 2012-04-12 12:18 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description getaceres 2008-11-11 16:12:32 UTC
I have a slow internet connection and everytime that I start Netbeans, the maven plugin starts to download the Maven
Central Repository Index. The problem is that, while it is downloading (it could take even hours) all the search
functions for plugins or artifacts don't work and only show loading. It's worse even because when I try to create a new
project, it shows the same message in the archetype selection and as I can't select an archetype, I cannot continue with
the creation of the project and I have to either:

1. Switch to offline mode, create the project and then enable online mode. This means loosing the repository part
downloaded to the moment.

2. Wait for the repository index to download. That could mean waiting for long time.

So I think that when it's refreshing the repository index, it should save the old index meanwhile for functions like
searching and archetype selection and then switch to the new index once it's completely downloaded. That way, there is
no productivity loss due to a slow connection or a slow repository server.
Comment 1 Milos Kleint 2008-11-12 10:39:52 UTC
more likely to be considered a bug.
the central repository performance dowgraded recently and indeed takes a long time, not your network problem probably.
at the same time it's being updated weekly I think, therefore the zip file gets downloaded frequently.
Comment 2 Milos Kleint 2008-11-18 09:12:45 UTC
It seems close to impossible to force nexus (the 3rd party library) to download stuff to a custom location. I guess we
will need to wait for nexus to implement incremental updates (then the big file gets downloaded just once and next time
you should get a weekly/monthly diff only).

as a workaround to the issue, you can set the updating period to "never" in Tools/Options/Misc/maven dialog.
You can force a manual update from the Maven repository browser ui.
Comment 3 tprochazka 2009-01-28 07:34:55 UTC
2mklient:
I don't understand your explanation.
Netbeans store index here
c:\Users\Tomas\.netbeans\dev\var\cache\mavenindex\
New index is downloading from
http://repo1.maven.org/maven2/.index/nexus-maven-repository-index.zip and unziped to
c:\Users\Tomas\.netbeans\dev\var\cache\mavenindex\

Why it's not possible use old var\cache\mavenindex\ during downloading new zip file with new index?

Netbeans use Nexus-indexer for exclusive aceess to var\cache\mavenindex\? For search and updating?
Comment 4 Milos Kleint 2009-01-28 07:41:06 UTC
because I have no way of telling when the old index is going to be switched to the new one. I just call a method in
nexus indexer that performs the complete task of downloading and switching.
Comment 5 Milos Kleint 2009-04-06 12:17:39 UTC
I consider it rather risky to try any bypassing of the nexus indexer library. The optimal way to tackle the slow
download times is to rely on improvements in the nexus indexer library itself. AFAIK they are working on incremental
updates of the index that shall be reasonably faster than the current "grab-all" approach.

closing as later. We will upgrade to the latest nexus indexer before or after beta and see if something improves, If not
we will have to wait for the fix for the next netbeans release. Feel free to file as issue to the nexus indexer that the
download and replacement of the index shall be 2 step operation.
Comment 6 Quality Engineering 2009-11-02 11:03:11 UTC
NetBeans.org Migration: changing resolution from LATER to WONTFIX
Comment 7 _ gtzabari 2011-04-29 15:29:02 UTC
Reopening this issue. I've got a relatively fast connection (500kB/s down) and downloading the index from central takes a very long time (over 10 minutes). I see two problems:

1. Netbeans is only downloading at around 50-100kB/s (perhaps the remote end is at fault here)
2. Nexus Indexer supports incremental updates but Netbeans doesn't seem to be using it: http://www.sonatype.com/people/2009/05/nexus-indexer-20-incremental-downloading/
Comment 8 _ gtzabari 2011-04-29 15:33:30 UTC
Seems to be related to issue 190399. If Netbeans 7.0 really uses Indexer 3.0, why does it take so long to download the index the first time Netbeans is run? Aren't you are bundling a baseline version of the Maven index with Netbeans?
Comment 9 Milos Kleint 2012-04-12 07:47:40 UTC
a few things changed in the latest (7.2) sources.

1. archetypes are currently not populated from the index, but from the archetype catalog files at the root of all repositories (central has it and so do repositories based on nexus for example)
2. we respect the mirrors in settings file entirely (used to be only non-star mirrors in 7.1)
3. we respect proxy settings now.
4. all repositories used in current projects are transparently added to the list of repositories and according to mirrors merged into one or more items
5. all queries to the indexing manager are initially only done on existing indexes, the non existing are skipped and partial result is returned relatively fast.
Comment 10 Milos Kleint 2012-04-12 08:07:59 UTC
(In reply to comment #8)
> Seems to be related to issue 190399. If Netbeans 7.0 really uses Indexer 3.0,
> why does it take so long to download the index the first time Netbeans is run?
> Aren't you are bundling a baseline version of the Maven index with Netbeans?

we don't bundle the maven index with netbeans. it would seriously increment the download size of netbeans. please see issue 206752
the bottleneck is the remote central storage bandwidth for sure. You can avoid it by using a repository manager, and mirror all server requests through the manager (nexus, archiva, artifactory,...). I have a testing nexus instance at home and the actual download is quick, then unpacking the index becomes the most time consuming  work.
Comment 11 Milos Kleint 2012-04-12 12:18:08 UTC
additionally today fixed issue 211075 makes sure that incremental updates are used on subsequent index downloads, again speeding up the process. (with exception of the first download of course)

not sure if this issue is an exact duplicate of any of the mentioned issues, but I assume fixing them also fixed this one.