I believe this is the same problem as 30149. Sorting gets enabled, and
the treetable presents some invalid model elements in the table -
selecting these then causes some exceptions.

Yes, it's probable that by solving 30149 this defect will vanish as
well. I've entered bugs separate, because they look so from user's
point of view (I believe).

Reassigning to openide/explorer: I believe this is a treetableview
bug. See

Date: 05 Feb 2003 16:05:39 +0100
From: Davide Alberto Molin <davide.molin@galactica.it>
To: dev@openide.netbeans.org
Subject: [openide-dev] Bad behaviour of TreeTableView

for an independent report of this problem on the tree table view.

*** Issue 30149 has been marked as a duplicate of this issue. ***

we've encountered the same exception when setting the rootcontext to a
different rootnode while having something selected in the TTV. I think
that particular issue is already fixed in the trunk. 

Passing to Jiri Rechtacek, owner of tree table view.

*** Issue 31505 has been marked as a duplicate of this issue. ***

This issue keeps coming back, so I'm bumping the priority up to P2 -
the visible artifacts (blank lines, etc.) look pretty bad.

*** Issue 30699 has been marked as a duplicate of this issue. ***

It is serious bug. The tasks/nodes in view (which is sorted) might be
unsynchronized with table model. If I called Purge Completed in any
task then the completed task persisted but some of rest tasks
disappeared. This problem leads to exp if the tasks are cascaded. Of
course, only if the nodes are sorted. The implementation of our
TTV.comparator is strange. I think it will ask heavy effort. Tor,
Jesse: don't you have any idea or patch ;-)) I'll investigate tomorrow.

Tor, I cannot see the noted exception of today's build (with updated
tasklist, Purge... action was removed). Did you any workaround? Don't
you have any Test which lead to the incorrect behavior of TTV? My test
code, which plays with Java files in the folders (sorted by name),
works for me. I haven't seen any place of tree table's code leaded
into problems. Some ideas?

I haven't done anything, so the problem should still be there. Don't
look for an exception though - the first thing you should notice is a
corrupted display (typically with some empty lines). If you try
operating on one of those empty lines, THEN you may get an exception
(since the node the tasklist is getting is invalid).

This is not 100% reproducible obviously.... I'll try to see if I can
come up with a reproducible scenario. I've avoided sorting for a while
since I was running into this problem.

Tor, I catch a incorrect behavior when creating subtasks in TaskList.
I have a stucture:
Summary
 |-> aaa
      |-> bbb
1) unsorted
- select aaa
- from popup menu invoke New Task and insert "ccc"
I have in TaskList.add()
root: Summary
parent: aaa
task: ccc

- new task is added as child of aaa, beside bbb --> correct

2) sort by name
- select aaa
- from popup menu invoke New Task and insert "ccc"
I have in TaskList.add()
root: Summary
parent: bbb
task: ccc

- new task is added as child of bbb --> incorrect

3) change to unsorted and back to sort by name
- select aaa
- from popup menu invoke New Task and insert "ddd"
I have in TaskList.add()
root: Summary
parent: aaa
task: ddd

tree is:
Summary
|->aaa
    |-> bbb
         |-> ccc
    |-> <empty task>
but: children of aaa in TaskChildren are: bbb, ccc, ddd

Could you check if anything isn't corupted? Maybe it lead to any next
problems.

I don't think so. If you run TaskList.print(), you'll see that the
model is correct. So I thought perhaps the nodes themselves are
getting corrupted.  So I wrote a new metho in TaskListView -
"nodePrint", which fetches the root node in the view, and recursively
calls getChildren.getNode() to print out the full node tree (by
printing node.getDisplayName() for each node).

I then ran this until I got a "corrupted view" (see attached
screenshot).  I had a simple list 
  bbb
  aaa
     ccc

and then I click on New to add a new task, "ddd". This should
get prepended to the list (I did not make it a subtask of
anything else), so you end up with
  ddd
  bbb
  aaa
    ccc

and indeed if you look at the output (which is present in the
screenshot, on the left hand side), both the model (with the header
"Task List:") and the node hiearchy (with the header "Task List (As
Seen via Nodes)") are correct - yet the display is not (the new item
is "blank".

I've putback my debugging method (nodePrint) so you can test it
yourself if you cvs update your source tree (use the trunk, not
release35 branch). I'll also attach "patch.diff", which modifies
TaskList.java to call prints as shown in my screenshot.  (Note - it
requires -J-Dtasklist.debug=true on the command line).

Created attachment 9287 [details]
Screenshot showing problem along with diagnostic output

Created attachment 9288 [details]
Patch which enables some diagnostics about tasklist modification

I spent the evening commute home on this bug, and I've found the problem.

I believe the SortedNodeTableModel in TreeTableView is the culprit.

With some println's I narrowed my focus down to getChild(). It gets
called with the wrong index.

Here's what happens.  I have a list, containing 
   A

Then I insert B - at the front of the list, so the real list is
   B
   A

In the sorted view, this is shown as
   A
   B


When an element is inserted into the tree, the tree is smart and only
requests the value for the new cell - so getChild() gets called only once.

BUT, it gets called with index=0! Note that this is the index of the
newly inserted element in the UNSORTED list, not in the sorted list. 

So here's the problem: DefaultTreeTable model calls nodesWereInserted,
with indices pointing to elements in the UNSORTED model. The
SortedNodeTreeModel does not translate these indices, so we end up
with incorrect indices to getChild().

A very simple fix, which I used to verify that this is indeed the
issue, is to simply override nodesWereInserted() in
SortedNodeTreeTable, by inserting something like
    if (sorting) {
        nodeStructureChanged(node)
    } else {
        super.nodesWereInserted(node, childIndices);
    }
In other words, when we're sorting, we don't know exactly which child
index to use, so update all.

But (besides the wasted performance in redrawing cells that have not
changed) this has the bad side effect of collapsing all nodes.

So a correct fix needs to subclass nodesWereInserted (and perhaps some
of the other nodesWere... methods as well), and when sorting,
translate the index array from the unsorted indices to the sorted
list. (And note the javadoc for DefaultTreeModel.nodesWereInserted:
the indices must be sorted in ascending order.)    

I started trying to implement this on the train, but ran into some
difficulties and then ran out of time.

But I confident this is the problem. I guess this is only showing up
in TreeTableView since no other views are offering sorting?

By the way, I noticed some strange code in the TreeTableView.  In
getChild for example, there's a check like this:

    if ((sortedByProperty != null || sortedByName || noSorting) {
        ...
        sort(list)

Why is "noSorting" in the above check????

Ales, please look on it from author's view :) Thanks

Created attachment 9335 [details]
a bugfix (draft)

Tor, Ales. Thank you for help.

Tor, your proposed fix works, but only partly.
Fixed the problem with inserting new nodes, it doesn't fix a problem
with any change of value in sorted column and also doesn't problem
with removing nodes if are being sorted.
1. changes of value
-- it can be fixed by much same way as nodesWereInserted, pretty easy

2. removing nodes
-- it's serious worse, nodesWereRemoved is invoked ex-post, a bad
nodes is removed already
-- VisualizerNode is notified by an event and invokes childrenRemoved,
which effective removes node
-- the indices in an event are corresponding to not-sorted indexes
-- these indexes must be converted to other indexes by sorting

Pls. look on attached patch, it should be reviewed before apply
because the changes in VisualizerNode seems risky to me. On the other
hand, it works ok for me with this patch. Thanks for comments.

I applied the patch and ran with it, but after a short while I got
this exception:

*********** Exception occurred ************ at Mon Mar 10 15:11:24 PST
2003
java.lang.ArrayIndexOutOfBoundsException: -1
        at java.util.Vector.elementAt(Vector.java:434)
        at
javax.swing.tree.DefaultMutableTreeNode.getChildAt(DefaultMutableTreeNode.java:230)
        at
javax.swing.tree.VariableHeightLayoutCache.treeNodesRemoved(VariableHeightLayoutCache.java:543)
        at
javax.swing.plaf.basic.BasicTreeUI$TreeModelHandler.treeNodesRemoved(BasicTreeUI.java:2455)
        at
javax.swing.tree.DefaultTreeModel.fireTreeNodesRemoved(DefaultTreeModel.java:528)
        at
javax.swing.tree.DefaultTreeModel.nodesWereRemoved(DefaultTreeModel.java:308)
        at
org.openide.explorer.view.NodeTreeModel$Listener.removed(NodeTreeModel.java:142)
        at
org.openide.explorer.view.VisualizerChildren.removed(VisualizerChildren.java:105)
        at
org.openide.explorer.view.VisualizerEvent$Removed.run(VisualizerEvent.java:111)
        at
org.openide.explorer.view.VisualizerNode$QP.run(VisualizerNode.java:475)
        at
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:178)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:454)
[catch] at
java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:201)
        at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:145)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:137)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:100)

Created attachment 9344 [details]
a proposed patch (2nd draft)

The second patch (bugfix30150_2.diff) fixed the problems with sorting
in TaskList and a reported exception also.

fixed, a proposed patch was (with a minor change) applied in main
trunk. If won't no problem, will be committed in release35 branch
tomorrow.

I grabbed the patch on my way out this morning and tested it on the train coming in to work. And I ran into some problems.  First I thought something was wrong in the suggestions module, since as I was iterating through tasks (F12) it was skipping some of them.  But then I got this exception (see next attachment).

NOTE - no sorting was in effect!! To reproduce this, try using the suggestions view for a while. It dynamically adds and removes suggestions after document edits which is why I think it runs into some issue with the index translation.

Created attachment 9361 [details]
Exception encountered while using the latest patch

I haven't seen it yet. I committed a debug check to VisualizerNode
which should signalize a problem and correct it.

I cvs updated org.openide.explorer.view and used the result again -
and I'm still running into exceptions occasionally. I didn't see any
additional tracing or debug messages anywhere (I looked in ide.log as
well). Here's a sample exception:

*********** Exception occurred ************ at Tue Mar 11 16:38:22 PST
2003
java.util.NoSuchElementException
        at java.util.LinkedList$ListItr.next(LinkedList.java:486)
        at
org.openide.explorer.view.VisualizerChildren.removed(VisualizerChildren.java:92)
        at
org.openide.explorer.view.VisualizerEvent$Removed.run(VisualizerEvent.java:111)
        at
org.openide.explorer.view.VisualizerNode$QP.run(VisualizerNode.java:486)
        at
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:178)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:448)
[catch] at
java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:230)
        at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:183)
        at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
        at java.awt.Dialog.show(Dialog.java:538)
        at org.netbeans.core.NbPresenter.superShow(NbPresenter.java:690)
        at org.netbeans.core.NbPresenter.doShow(NbPresenter.java:733)
        at org.netbeans.core.NbPresenter.run(NbPresenter.java:721)
        at org.openide.util.Mutex.doEventAccess(Mutex.java:932)
        at org.openide.util.Mutex.readAccess(Mutex.java:157)
        at org.netbeans.core.NbPresenter.show(NbPresenter.java:706)
        at
org.netbeans.modules.java.JavaConnections.reallyShowChangesDialog(JavaConnections.java:470)
        at
org.netbeans.modules.java.JavaConnections.access$300(JavaConnections.java:46)
        at
org.netbeans.modules.java.JavaConnections$1.run(JavaConnections.java:313)
        at
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:171)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:448)
        at
java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:230)
        at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:183)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:99)

Could this at all be the cause of issue 31957?

to dnoyeB: No, these problems are delimited only on TreeTableVview.
Noted CVS trees are not TTV. I don't think that there is any shared
defect.

Created attachment 9393 [details]
jar to lib/patches

Tor (or other), could you test a attached jar? Thanks
A cause of problem with remove a node from TTV (as same as of reported
exception) is dependence VisualizerChildren on an order
Node.getChildren().getNodes(). TTV does only sorting on TreeModel's
level, not on node's level => so, VisualizerChilder couldn't depend on
node's order. Other possible solution (but not useful) is reorder the
nodes in TTV by Index cookie. The attached patch (also as previous)
try remove this dependence.

Thanks, I'll try it. I spent some time this morning trying to narrow
the problem, and I was able to get a reproducible test case - but as
soon as I took out some System.err.println's in ExplorerManager, it no
longer worked - so I think this problem is timing sensitive (note that
the Suggestions view will both remove and add nodes in very short
order - I wonder if some timers in the node handling is affected by this.)

Two other comments - I see you added some logging, but the
INFORMATIONAL ones don't seem to show up anywhere. I changed it to
unconditional System.err.println's so I could see them.

Also, I see that in VisualizerNode, you're transforming indices and
sorting - this is -only- done when sorting is in  effect on the
treetableview, right? All the problems I've been observing has been
when sorting is -not- on, and since I would see your "child not found"
error I suspect you're transforming indices in all cases. For
performance it would be good to only do this when necessary, and it
would make the bugs I've been seeing go away for the common case (no
sorting), but of course we need to get the bug fixed in all cases.

Ok, I'll get the patch now and use it in my daily work today.

(P.S. in VisualizerNode: tranform -> transform) 

Created attachment 9397 [details]
patch removes dependence to node's order

Thanks for comments.
I trying a different way how to remove the described dependence. I
don't transform the indices for VisualizerEvent.Removed no longer. I
removed the indices from VisualizerEvent's constructor all over, now I
give an array of removed nodes in place of these indices.
VisualizerChildren processes this array with O(n*m), n = count of
children, m = count of removed nodes. It's not good enough (but, in
practice size of removed nodes is significant less that all children).
If this patch will work, I'm going to investigate performance
possibilities more.

I didn't get a chance to use the patch a lot today (meetings,
meetings) but I did try to stress test it tonight and haven't seen any
problems, so the patch works better than what is in the trunk for me!

reopen, this bug has not been fixed yet, target = 3.5

I applied visualizerChange.diff patch in main trunk. This defect is
fixed in 4.0 now. If no problem appears, it will be merged in
release35 branch on Mar 17.

fix merged to release35 branch

verifying...

This issue had *1 votes* before move to platform component