This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 149315 - Unit test results should be inlined in the node display name!
Summary: Unit test results should be inlined in the node display name!
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: JUnit (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: junit-issues@java
URL:
Keywords:
Depends on:
Blocks: 172772
  Show dependency tree
 
Reported: 2008-10-07 06:10 UTC by Torbjorn Norbye
Modified: 2009-10-02 12:30 UTC (History)
1 user (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Current output (49.71 KB, image/png)
2008-10-07 06:11 UTC, Torbjorn Norbye
Details
Screenshot of test output including the test output (80.40 KB, image/png)
2008-10-07 06:12 UTC, Torbjorn Norbye
Details
Patch against current sources (October 6th 2008) to implement this (enabled on a flag, -J-Djunit.inline_result=true) (4.57 KB, patch)
2008-10-07 06:13 UTC, Torbjorn Norbye
Details | Diff
Patch for ruby.testrunner (as simple as possible rather than properly formatted) (4.21 KB, patch)
2008-10-07 06:31 UTC, Torbjorn Norbye
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Torbjorn Norbye 2008-10-07 06:10:36 UTC
When I'm running my unit tests, and I get failures, all I see is the red "Failed" message. I don't know why it failed.
To find out, I have to use my mouse to click on each failure to "open" its details area, where the message is included.

However, I noticed that we're also displaying the running time for the tests. Personally, I don't care how many seconds
a test took to run - but I -do- want to see the message that the test failed with! Sometimes I can immediately tell what
the issue is so I can continue fixing my test or the associated behavior. Other times I can tell it's a failure I can
ignore (for example that a performance test failed because it took too long, which is expected if I had the test
suspended while debugging part of it).

So here's an idea -- why don't we display the status messages from each test right in the test summary?

I liked that idea so much that I went ahead and patched my own clone to do that - and I've been using it that way for a
couple of days now. And I'm convinced it's a big improvement - at least for my workflow. And I don't see how this
wouldn't be an improvement for anyone.

I'll attach a couple of screenshots: first a screenshot which shows what the junit window looks like today - you get
some failures, with no clue as to why. You do however know how many milliseconds each test took ;-)

In the second screenshot, junit-proposed.png, you can see what happens if we include the status message right in the
display name for each node. In many cases, this is enough for me to decide what the problem is and decide which test to
focus in on. (And for example after making a change to the code, running the test again and seeing the same message
tells me more than having to click my way down to find out if this is the same or a new failure.)

Finally, the third attachment is my patch to do this to the Junit module. I suspect the patch for the Ruby test runner
is very similar.
Comment 1 Torbjorn Norbye 2008-10-07 06:11:33 UTC
Created attachment 71253 [details]
Current output
Comment 2 Torbjorn Norbye 2008-10-07 06:12:21 UTC
Created attachment 71254 [details]
Screenshot of test output including the test output
Comment 3 Torbjorn Norbye 2008-10-07 06:13:53 UTC
Created attachment 71255 [details]
Patch against current sources (October 6th 2008) to implement this (enabled on a flag, -J-Djunit.inline_result=true)
Comment 4 Torbjorn Norbye 2008-10-07 06:31:26 UTC
Created attachment 71256 [details]
Patch for ruby.testrunner (as simple as possible rather than properly formatted)
Comment 5 Erno Mononen 2008-10-10 10:32:54 UTC
Thanks for the patch Tor. I like the idea, I'll try it out and integrate then to the Ruby test runner (post 6.5 of 
course). 
Comment 6 Erno Mononen 2008-10-15 10:53:57 UTC
I integrated the patch to ruby.testrunner, #9c93366658bc. I used a different system property for ruby.testrunner though 
(perhaps we'll want to consolidate this later): ruby.testrunner.inline_result, enabled by default.
Comment 7 Torbjorn Norbye 2008-10-15 15:52:49 UTC
I don't think we need to have a system property for it at all, since I think everybody will want this. At most it should
be a property to turn it off.

I included the system property in my patch in case there was a chance this behavior could be included in 6.5 ;-)
Comment 8 Quality Engineering 2008-10-15 16:18:04 UTC
Integrated into 'main-golden', will be available in build *200810151401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/9c93366658bc
User: Erno Mononen <emononen@netbeans.org>
Log: #149315: Unit test results should be inlined in the node display name!
- applied Tor's patch for ruby.testrunner
Comment 9 Erno Mononen 2008-10-15 19:41:15 UTC
Yeah, in the ruby test runner the property is turned on by default; will remove it if there is no negative feedback 
(which I don't really expect).
Comment 10 cms 2008-12-05 11:07:27 UTC
Integrated in trunk for JUnit module. The integrated fix doesn't rely on property junit.inline_result. Looks like it's 
not useful.

BTW, Tor, thanks for the suggestion!

http://hg.netbeans.org/main/rev/1310a99b2549
Comment 11 Quality Engineering 2008-12-07 05:32:36 UTC
Integrated into 'main-golden', will be available in build *200812070201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/1310a99b2549
User: cms@netbeans.org
Log: #149315, #69040: JUnit test results UI improved (added tooltip; failure reason is shown near failed test name)