| Author |
 Topic  |
|
|
Stephen
Tomato Guru
United Kingdom
781 Posts |
 Posted - Feb 19 2004 : 04:47:24 AM
|
I was about to post this rant in the thread to which it refers, but I restrained myself and decided to raise it in a thread in the Private Beta instead.
I notice that someone rated thread 1484 — surround selection no longer works with quote marks — "5: Fatal". Of course, I don't know who did it, but I've noticed a few occasions recently where people have rated their favourite bugs 5, even if they could by no stretch of the imagination be called Fatal. I'm only picking on this one as an example of a recent trend, but I don't see how anyone could rate this higher than 3: Important, and 2: Desirable might be more like it.
"Grade inflation" is a common problem in ratings in online forums, as I'm sure people have found in other contexts. I thought that developers, being a generally smart, logical bunch, might resist it, but it seems not.
I also notice that most threads are not rated, and those that are are often only rated by one person (because my rating almost always swings the overall rating).
This is a long way of getting round to some questions. I wondered: how useful do the WT Team find these ratings? Are they useful to you? How can we make them more useful? How can we discourage people from voting all their own bugs at 5? Any other observations or thoughts?
</rant> |
Stephen Turner
ClickTracks http://www.clicktracks.com/
Winner: ClickZ's Best Web Analytics Tool 2003 & 2004
|
|
|
mrcode925
Ketchup Master
USA
54 Posts |
Posted - Feb 19 2004 : 08:46:14 AM
|
I think it is obvious that the problem with the rating system is the option names. They aren't ratings, they are categories. Something is either fatal or it isn't. It is either a request or it isn't. Rating, of course, is a degree of difference. There is no point in getting more than one opinion on what category something is. There is no reason to calculate the average opinion of what category something is. If the rating was just limited to the person who started the post then it might work.
Also, I would not disagree with the categories on the rating system because not all problems are equal. If an option in VAX is fatal it is truely a problem, but not if that option is only used once in a blue moon. It may be more important to fix a minor annoyance that is shared by 100's of users on an hourly basis. I think that this is where the rating system should be focused. Simply try to determine how important each issue is to everyone and use the average rating to determine where priorities may lay. |
If it starts to make sense, you're in a cult. |
 |
|
|
WannabeeDeveloper
Tomato Guru
Germany
775 Posts |
Posted - Feb 19 2004 : 12:27:40 PM
|
I don't see any point in rating a thread at all!
If it's a bug, it needs to be fixed, no matter what I think of it's importance!
Of course everybody will rate "his" threads with 5, having in mind "I have that error, cannot work with that bug, fix it quick!"
I never rate any threads, because I think it's up to the WT guys to rate any importance of bugs...
|
 |
|
|
|
LarryLeonard
Tomato Guru
USA
1041 Posts |
Posted - Feb 19 2004 : 1:12:29 PM
|
quote:
If it's a bug, it needs to be fixed, no matter what I think of its importance!
Hey, even I can't fix all my bugs, and I'm nearly perfect. Scarce resources being, well, scarce, they have to make choices. The countryside is littered with the corpses of small software companies that fixed every bug...
quote:
I never rate any threads, because I think it's up to the WT guys to rate any importance of bugs...
The problem is, what they think is a horrible bug that threatens our PBFs ("Precious Bodily Fluids" - google it), we might think is trivial. And vice versa. I have the same experience with my users; that's just the way it is in software.
Having said that, I must confess that I can't seem to remember to rate threads; maybe I only get motivated enough to rate a thread if it's a really obnoxious bug, and there are less and less of those as time goes by...
|
|
|
|
Stephen
Tomato Guru
United Kingdom
781 Posts |
Posted - Feb 19 2004 : 1:19:36 PM
|
| Kudos to whomever gave this thread five tomatoes. |
Stephen Turner
ClickTracks http://www.clicktracks.com/
Winner: ClickZ's Best Web Analytics Tool 2003 & 2004
|
|
|
|
Uniwares
Tomato Guru
Portugal
2322 Posts |
Posted - Feb 19 2004 : 3:03:19 PM
|
Probably the rating should not be shown publicly to solve this problem. When you rate a bug, you may consider it fatal (after all it somehow concerned you) but its really a matter of guestimate to give proper weight to a bug and it takes lot of experience to do it sensitive.
When I rate bugs (which I rarely do for bugs reported by others) then I am (hopefuly) aware of its impact on the product (VAX) and productivity (users).
Just hide the rating from the forums for everybody except moderators. This will make the ratings fairer (i think). Also add a note to the rating drop list, that it is intended to indicate how important this bug is to the reporter, not to make it look more important.
|
|
|
|
Stephen
Tomato Guru
United Kingdom
781 Posts |
Posted - Feb 19 2004 : 3:43:23 PM
|
I'm not sure how much that helps. People will still be tempted to rate their own bugs highly.
Maybe you shouldn't be able to rate your own bugs. Or maybe only a list of trusted users can rate. Or something like that.
But really, I want to know from the WTites whether they find it useful, and how we can communicate this sort of information most effectively to them. Maybe we should each list our own top bugs, for example? Or maybe you already know which bugs are most important without us having to tell you? |
Stephen Turner
ClickTracks http://www.clicktracks.com/
Winner: ClickZ's Best Web Analytics Tool 2003 & 2004
|
|
|
|
LarryLeonard
Tomato Guru
USA
1041 Posts |
Posted - Feb 19 2004 : 4:08:35 PM
|
quote:
I want to know from the WTites whether they find it useful, and how we can communicate this sort of information most effectively to them
That's the crux of the issue, right there: our goal should be to make it easier for WT to fix the bugs that are most important to us.
The problem is, it's hard for us to do that without a comprehensive list of the currently outstanding bugs. Clearly, a list like that is not something any company wants to make readily available; who wants to see newsgroup postings with the subject, "WT admits to over 200 bugs in VAX!"? Not me. (The fact that many of the bugs are of the "Some icons are too red" variety won't matter, any more than it slowed down the "8000 bugs in Win2K" nonsense a few years ago.)
So, I don't know... maybe the best thing is for each of us to come up with our "Top 10 Bugs" list, like someone suggested...
|
|
|
|
WannabeeDeveloper
Tomato Guru
Germany
775 Posts |
Posted - Feb 20 2004 : 01:54:39 AM
|
quote:
Originally posted by LarryLeonard
quote:
If it's a bug, it needs to be fixed, no matter what I think of its importance!
Hey, even I can't fix all my bugs, and I'm nearly perfect. Scarce resources being, well, scarce, they have to make choices. The countryside is littered with the corpses of small software companies that fixed every bug...
The importance in the sentence lies within the "I".
It really doesn't matter what I think about the severity of a bug...
Now if WT thinks this bug I found is not that severe, they will fix it somewhere in time, I'm pretty sure...
They should rate the threads to show us their priority-list (at least on the bugs we found).
I don't want a complete buglist from them either...
Hell, I'm just coding in ANSI-C, using no Classes etc., so you have found bugs I wouldn't even recognise because
I am using VA_X completely different than you do! So who am I to rate someone's bug that doesn't even bug me?
That's the point... I could say: "Hey, Larry is nearly perfect, I really like his writing-style here in the forum, so the bugs he found might be important" and rate all your threads with 5 Tomatoes... does this make a bug more important? I don't think so...
On the other way around, you might think my found bugs are somewhat unimportant, but for me they are (mostly, you know. Graphical Bugs like wrong coloring are way back on my priority-list, but I really think my last found bug "Autocompletion deletes parts of itself" is almost a showstopper. It renders one of the biggest features of VA_X almost useless).
|
|
|
|
|
support
Whole Tomato Software
5566 Posts |
Posted - Feb 20 2004 : 03:05:46 AM
|
With number of bug reports we get, it is difficult to determine which should get fixed first. We have limited resources. We cannot fix everything now.
The ratings help us determine which features to fix. We want authors to rate their own bugs. Most authors rate well. We can spot those who don't.
http://forum.wholetomato.com/forum/topic.asp?TOPIC_ID=1051
A lame bug report that gets five tomatoes is still lame and doesn't get top billing. On the other hand, a bug report which stirs discussion and "me too" comments gets more attention, regardless of the number of tomatoes. Reports that resurface repeatedly get more attention as well.
Honestly, we could have a few developers work full time on prioritizing and making public our bugs. The trouble is they wouldn't fix anything. Like everyone else, we make a reasonable attempt at prioritizing as we push ahead as quickly as possible.
Larry's list long ago of ten bugs to fix, (which we believe contained only seven items) was nice. For a guy that reports 100+ bugs, we knew which were the most important to him. We'd take a top ten list from anyone else as well. Top four might be nicer.  |
Whole Tomato Software, Inc. |
|
|
|
Stephen
Tomato Guru
United Kingdom
781 Posts |
Posted - Feb 20 2004 : 08:09:55 AM
|
There's an offer I can't refuse! Here's my list, in priority order.
Top three:
1352 Slash deletes highlighted block
1429 Navigating round file fails in presence of #if's
1601 Deletes its own autocomplete entries
Four runners-up, but some way behind:
0656 CMap's with pointer values: tooltips confused
1430 HCB confused about CString
1247 Hovering tooltips shuffled
1244 HCB only works with click-and-hover not just hover
(Do you think if I put them in two groups, they won't notice there were seven?) |
Stephen Turner
ClickTracks http://www.clicktracks.com/
Winner: ClickZ's Best Web Analytics Tool 2003 & 2004
|
|
|
|
WannabeeDeveloper
Tomato Guru
Germany
775 Posts |
Posted - Feb 20 2004 : 08:26:07 AM
|
No problem, Stephen, you can have 2 Slots of mine (so you just need 1 Bug-Slot to be on the safe-side) 
Here are two that bug me right now:
1147 Suggestin nothing after ), } or deleting lines.
1601 Deletes its own autocomplete entries
|
|
Edited by - WannabeeDeveloper on Feb 20 2004 08:28:18 AM |
|
|
|
Uniwares
Tomato Guru
Portugal
2322 Posts |
Posted - Feb 20 2004 : 10:29:23 AM
|
Ok, let me add some too. I promise I keep the Top 4 list under 50 items 
full support for template classes in C++ (maybe Generics in C#?)
1384 duplicates in HCB
1549 typedef structs
1591 parameter as member
1565 tooltips
1153 m and _ (related to 1601 ?)
No special order for these ones:
1567 wheel scrolling
1524 #define'd symbols
1500 CoUninitialize()
1464 CEvent & MFC
1462 virtual destructors
1384 function comments
1198 more parsing problems
and then there was the rest which might get fixed one day.
|
|
|
|
LarryLeonard
Tomato Guru
USA
1041 Posts |
Posted - Feb 20 2004 : 11:00:52 AM
|
Okay, here's my latest Top 10 (okay, 6 4) list from the home office in East Lansing, Michigan:
(1) forum.wholetomato.com/forum/topic.asp?TOPIC_ID=973 Fancy VAX colors don't work well at all on black backgrounds. C'mon, guys - all right-thinking, elderly, nearly blind programmers use a dark background.
(2) forum.wholetomato.com/forum/topic.asp?TOPIC_ID=973 and forum.wholetomato.com/forum/topic.asp?TOPIC_ID=986 The HCB should never erase its contents - even slightly stale information is better than no information! And it certainly shouldn't respond to hovering over int by showing me what an int is. I know what an int is - what I really want to see is the class the int is in, which was there a second ago, until you decided I needed to know all about ints.
(3) forum.wholetomato.com/forum/topic.asp?TOPIC_ID=1077 An "anathema" file would allow me to tell VAX things like, "Stop suggesting ASSERT - I only ever use _ASSERTE, so let's just pretend ASSERT doesn't even exist." This would eliminate a lot of annoying, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive, repetitive choices I have to make every single, single, single, single, single, single, single, single, single, single, baked beans, single, single, single, single time: can you tell I'm tired of having to specifically choose return over return_temporary_buffer?
(4) forum.wholetomato.com/forum/topic.asp?TOPIC_ID=1527 My AutoText ||| suddenly no longer erases itself when expanding.
Thanks for listening! And when's 1217 coming out?
|
Edited by - LarryLeonard on Feb 20 2004 11:28:43 AM |
|
|
|
Stephen
Tomato Guru
United Kingdom
781 Posts |
Posted - Feb 20 2004 : 11:09:53 AM
|
Larry: Is 1150 the same as 1508? In which case it was fixed in build 1216.
I can't reproduce 988. It looks as if it's changing the colours, which is a bit weird, but the colour is reset if you leave the colour-picker without picking one.
And I'm not sure you mean 1525. That thread seems to be about something different. |
Stephen Turner
ClickTracks http://www.clicktracks.com/
Winner: ClickZ's Best Web Analytics Tool 2003 & 2004
|
Edited by - Stephen on Feb 20 2004 11:13:27 AM |
|
|
|
LarryLeonard
Tomato Guru
USA
1041 Posts |
Posted - Feb 20 2004 : 11:30:40 AM
|
Sigh... "it's been such a long exposition"...
|
|
|
| |
Topic |
|
|
|
Topic Locked
