This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 152208 - Plugin Portal is not search-engine crawlable
Summary: Plugin Portal is not search-engine crawlable
Status: RESOLVED FIXED
Alias: None
Product: updatecenters
Classification: Unclassified
Component: Pluginportal (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Jan Pirek
URL: http://plugins.netbeans.org/
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-03 15:51 UTC by jcatchpoole
Modified: 2012-01-02 16:25 UTC (History)
1 user (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jcatchpoole 2008-11-03 15:51:03 UTC
I am not sure if this should be posted as a new issue, or if it is completely covered by issue 136922.  I see there is a
fix coming for issue 136922, but I am not sure to what extent that will address the problems here.

Google, and probably other search engines, are unable to properly crawl the Plugin Portal.  I recently added a
Sitemap.xml for the PP, and submitted it to Google.  I can now see diagnostic info about what happens when Googlebot
attempts to index the pages I listed in the Sitemap.  Basically every page I submitted (explicit links to PP module
detail pages, for eg : http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=10327), Google lists
as not indexed because of a "redirect error".  

It links to this page for more info, where it suggests that cookies, sessions, and redirects will impact crawlability :
https://www.google.com/support/webmasters/bin/answer.py?answer=35156&ctx=tltp&hl=en

wget -S for the example page above shows that cookies, sessions and redirects are *all* used :

[jc97959@viscous] : tmp $ wget -S "http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=10327"
--16:23:12--  http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=10327
           => `PluginDetailPage.jsp?pluginid=10327'
Resolving plugins.netbeans.org... 66.199.229.210
Connecting to plugins.netbeans.org|66.199.229.210|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.0 302 Found
  Connection: keep-alive
  X-Powered-By: Servlet/2.5
  Set-Cookie: JSESSIONID=62d360d1d1a639ffffffffd23d9d18f6f00fb; Path=/PluginPortal
  Location: http://plugins.netbeans.org/PluginPortal/faces/MainPage.jsp
  Content-Type: text/plain; charset=iso-8859-1
  Content-Length: 0
  Date: Mon, 03 Nov 2008 14:48:18 GMT
  Server: Sun Java System Application Server Platform Edition 9.0_01
Location: http://plugins.netbeans.org/PluginPortal/faces/MainPage.jsp [following]
--16:23:13--  http://plugins.netbeans.org/PluginPortal/faces/MainPage.jsp
           => `MainPage.jsp'
Reusing existing connection to plugins.netbeans.org:80.
HTTP request sent, awaiting response... 
  HTTP/1.0 200 OK
  Connection: keep-alive
  X-Powered-By: Servlet/2.5
  X-Powered-By: JSP/2.1
  Content-Type: text/html;charset=UTF-8
  Content-Language: en-US
  Date: Mon, 03 Nov 2008 14:48:19 GMT
  Server: Sun Java System Application Server Platform Edition 9.0_01
  Content-Length: 71231
Length: 71,231 (70K) [text/html]

100%[====================================================================================================================================>]
71,231       120.76K/s             

16:23:14 (120.41 KB/s) - `MainPage.jsp' saved [71231/71231]

In fact is looks like the *only* pages from the PP in Google's index are static pages - directory listings, xml files,
txt files.

http://www.google.com/search?q=site%3Aplugins.netbeans.org

Pls feel free to close as a dup if the fix to 136922 will address this ...  just wanted to file in case not.
Comment 1 Jiri Kovalsky 2012-01-02 14:03:03 UTC
Honzo, if I am not mistaken, this issue can be resolved as fixed since Plugin Portal 2.0, right?
Comment 2 Jan Pirek 2012-01-02 16:25:11 UTC
yes, this is now fixed