PDA

View Full Version : .0002 point difference


Pickin' Porkers
11-27-2012, 07:50 AM
I have not done the math yet to make this determination BUT, does anyone know what would make a .0002 point difference between two different scores in KCBS? I see a lot of that including .0004 point difference and curious to know which differences in score would create such a close division in scores! Is it appearance, taste or tenderness or a combination of these or WHAT!

Uncle Buds BBQ
11-27-2012, 08:03 AM
It's a combination of all 5 judges scores. The low score (6th) gets dropped.
Take the appearance score and multiple it by .5714, taste by 2.2858 and texture by 1.1428 to get the actual weighted number.

DawgPhan
11-27-2012, 08:06 AM
what really drives you crazy is that difference is often the difference of $1000's becuase of the outdated way of paying out places. These days the difference between 1st and 3rd might be less than 1/2 a point. but that might mean thousands of dollars to a team.

82's BBQ
11-27-2012, 08:42 AM
I have missed money by .0001 pt and knew when I turned in my box that the presentation wasn't great. I messed with it 5 times to even it out but couldn't get it were I wanted it. I suppose 6 times may have been the charm.

holysmokes
11-27-2012, 09:05 AM
We lost the national BBQ duo by .0004 points. 2500 difference.

dmprantz
11-27-2012, 09:16 AM
Presuming that you are talking about KCBS and forgetting about the low scores, if Team A gets one 8 in taste and team B gets four 8s in appearance, all other scores being 9s, when everything is weighted and summed, Team B will be ahead by 0.0002. Similarly if Team B gets two 8s in tenderness, they will still be ahead by 0.0002. Basically, taste counts for a lot, and you have to really tank it in either of the other categories to catch up to some one who gets hosed by one judge in taste.

My understanding is that the scoring will change somewhat in 2013, and this will not apply as is. The way the scoring works now ties are possible in that tenderness is twice of appearance as in the above examples, so four 8s in appearance is the same as two 8s in tenderness, but when you go to tie breaking, a higher score in tenderness always wins. The 2013 scoring is supposed to theoretically prevent the tie from occuring in the first place while not affecting the final results at all. I'd hope that some one did a Spearman rank correlation to verify....

dmp

Big Poppa
11-27-2012, 09:32 AM
if there is a tie in the spearman correlationthey can always fall back on the litmus configuration

dmprantz
11-27-2012, 09:50 AM
It's a shame that my use of big words leads you to make fun of me. Since you don't appear to be able to read my posts without doing so, maybe you should use that ignore feature you've asked about in the past....

dmp

Uncle Buds BBQ
11-27-2012, 10:07 AM
but when you go to tie breaking, a higher score in tenderness always wins.

dmp
Daniel,
You sure about that? I thought rule #7 states that the low score (that was thrown out) breaks the tie.

dmprantz
11-27-2012, 10:11 AM
http://www.kcbs.us/pdf/2012_Rules.pdf

If there is a tie in one of the categories, it will be broken
by the computer, as follows: The scores will be compared
(counting all five judges) for the highest cumulative
scores in taste, then tenderness, then appearance. If still
tied, then the low score, which was thrown out, will be
compared and the higher of the low scores will break the
tie. If still tied, then a computer generated coin toss will
be used.

Uncle Buds BBQ
11-27-2012, 10:14 AM
Yep, that's what I was quoting so when you said "but when you go to tie breaking, a higher score in tenderness always wins" I don't understand.

Untraceable
11-27-2012, 10:19 AM
Id rather lose by .0002pts than through some series of tiebreakers

dmprantz
11-27-2012, 10:21 AM
The way I read that is that if there is a weighted score tie, first you compare the two teams' taste scores from the top 5 judges, then tenderness, then appearance. Only if all three of those tie do you bring in the dropped score. Following that process and the current weightings, appearance of 98888 is the same numerical score as a tenderness of 99988, but since the tenderness is compared first in the above process, the higher tenderness score will break the tie. The intent with the weight change is to have the same final results, but not have it look like a tie. Some one on the board referred to it as a "false tie" I think. The other part of the goal from what I understand is that the only effect of the scoring change is that these situations won't look like ties.

dmp

dmprantz
11-27-2012, 10:25 AM
Id rather lose by .0002pts than through some series of tiebreakers

I'm not arguing for or against the change, but supposedly the two systems are logically equivalent. That is, the outcome isn't ever changed, it's just how it looks on paper. I'd call it an algorithmic euphamism. :) If that makes you feel better about it, great. It doesn't really matter to me because taste is still more imporant than tenderness which is still more important than appearance.

dmp

Uncle Buds BBQ
11-27-2012, 10:28 AM
The way I read that is that if there is a weighted score tie, first you compare the two teams' taste scores from the top 5 judges, then tenderness, then appearance. Only if all three of those tie do you bring in the dropped score. Following that process and the current weightings, appearance of 98888 is the same numerical score as a tenderness of 99988, but since the tenderness is compared first in the above process, the higher tenderness score will break the tie. The intent with the weight change is to have the same final results, but not have it look like a tie. Some one on the board referred to it as a "false tie" I think. The other part of the goal from what I understand is that the only effect of the scoring change is that these situations won't look like ties.

dmp
OK...I see what you are saying but I didn't interpret the rule that way.

dmprantz
11-27-2012, 10:32 AM
Without further documentation, that's how I'd write the software based on that rule, but I could be wrong. It's been known to happen:) Any one with more detailed information on the tie-breaking algorithm for KCBS is welcomed to correct me.

dmp

Pickin' Porkers
11-27-2012, 10:40 AM
. I'd hope that some one did a Spearman rank correlation to verify....

dmp

That would apply in this case but wouldn't switching to a 1/2 point scoring system alleviate some of these tie and "close call" issues as well?

dmprantz
11-27-2012, 10:49 AM
That would apply in this case but wouldn't switching to a 1/2 point scoring system alleviate some of these tie and "close call" issues as well?

I'll answer the question with another question: Would changing from a 10 point scale (0-9) to a 20 point scale (0-19) prevent this? My answer is mathematically no, and the reason why I asked it is that it's the same thing put into different words. BBQ scoring is a discrete system that doesn't have a normal distribution of values, so in the end you have a finite number of scores. Whether those scores involve fractions, whole numbers, or are assigned by Greek letters, it doesn't really matter. All you really need to do is rank those scores in order from 1 to whenever and decide which ones beat which ones. Increasing the the quantity of discrete scores gives you the flexibillity to have more granular results, but only if you specifically choose to. It all depends on the ranks you give the combinations and how you want to enforce it algorithmically.

There is the psycological effect though, and I don't account for it. If scoring stays in the 6-9 range currently, half scores could keep it in that same range and still have the same granularity, but again, mathematically it makes no difference as far as I am concerned.

Sorry to be such a geek about it, but this is what I do, or at least part of it.

dmp

EDIT: Increasing the number of discrete values would actually have the opposite effect. The distribution of scores becomes closer so scores look closer than they are. You also increase the odds of having tie scores, or require that weightings be carried out to more decimal places to avoid ties. If you want to avoid ties and "close scores" you should reduce the number of scores available. In the end, keep in mind that whatever the mathematics show on paper, your score sheet currently has 1,000,000,000,000,000,000 possible values on it per category. Your goal is to get the best combination regardless of how close it is to the next highest.

Pickin' Porkers
11-27-2012, 12:04 PM
There is the psycological effect though, and I don't account for it. If scoring stays in the 6-9 range currently, half scores could keep it in that same range and still have the same granularity, but again, mathematically it makes no difference as far as I am concerned.

.

I understand this well and understand the distribution as well too. It is the the psychological issue I am more concerned with, in having the scoring system changed! As a judge there have been numerous times I have teetered on the fence between an 8 & 9...but give the 9 as instructed to give the cook the benefit of the doubt. Consequently, other judges may as well which accounts for the groups of ties which abound on score sheets. Not being the math whiz you obviously are (nothing mean intended...LOL), I am not sure the granularity would be the same if granularity is affected by explicitness or under specification. The explicitness or under specification in this case is a limit to the scores available to be given by the judges.

In short, forgetting the statistical aspects of a decision to go to a half point system....it just makes common sense!!!!

dmprantz
11-27-2012, 12:24 PM
I feel your pain. The counter-argument, which I've heard made here on The Brethren multiple times, is to use the entire range of scores available. A judge can give an entry anywhere from a 2 to 9 on any criterion.

Personally, from a mathematical standpoint, I'd rather see the raw scores stick with whole numbers and encourage judges to use them rather than to use fractional scores. Let 2 be the new six, and 8 be the 8.5 you want:) That's a training issue though, and I don't think that the KCBS can force judges to do it. I could see moving to half scores as an atlernative to get the same effect through judges' heads, but I would prefer them say only scores 5 or 6 through 9 are valid if they do that. That way you still have about the same number of valid scores. You might even be able to change the weighting so that it's the same anyway. Like I said though, as a cook who understands math, it doesn't really matter to me. Just turn in the best product you can and hope for the right tables:)

dmp

dmprantz
11-27-2012, 12:29 PM
The question though, and the nature of the beast with discrete systems, is what happens if they change the scores, 8.5 becomes the new 8, and now you can't decide between 8.5 or 9, so what do you do? Similarly, what's to prevent judges from saying all entries should get an 8, 8.5, or 9, just as many do now with 7, 8, and 9? At the end of the day, there's a finite number of discrete scores that a judge can give, and judges should give them consistantly. Anything else, and you're going down the rabit hole.

dmp

Pickin' Porkers
11-27-2012, 01:39 PM
I feel your pain. The counter-argument, which I've heard made here on The Brethren multiple times, is to use the entire range of scores available. A judge can give an entry anywhere from a 2 to 9 on any criterion.

Personally, from a mathematical standpoint, I'd rather see the raw scores stick with whole numbers and encourage judges to use them rather than to use fractional scores. Let 2 be the new six, and 8 be the 8.5 you want:) That's a training issue though, and I don't think that the KCBS can force judges to do it. I could see moving to half scores as an atlernative to get the same effect through judges' heads, but I would prefer them say only scores 5 or 6 through 9 are valid if they do that. That way you still have about the same number of valid scores. You might even be able to change the weighting so that it's the same anyway. Like I said though, as a cook who understands math, it doesn't really matter to me. Just turn in the best product you can and hope for the right tables:)

dmp

I personally would LOVE to see this but the first thing I get in my mind when reading this is the Andy Williams song "The Impossible Dream" (never did like to hear him sing however).

Judges simply won't give a score of 5...4 ...or 6 because of the notion that they are judging a pro category and also, the fear of being called down by a TC or Rep because the score is out of line with the other judges. Add to that fear, rumors float of tracking judge scores. Not too sure how that will fly...

Bottom line is yes..... using the entire range CORRECTLY is the better solution, no doubt. In the meantime.... switch to a 1/2 point system.....LOLOLOL

bbq.tom
11-27-2012, 03:54 PM
Bottom line is yes..... using the entire range CORRECTLY is the better solution, no doubt.


I personally believe the MOST judges DO use the "entire range" CORRECTLY, as it is extremely rare to find an entry that is anything less than a "6" IMHO. I don't know what or if you judge, but if you think that the entries are that bad ("below average" or worse), then you haven't been at the contests that I've judged (or at least not at the same table)!!! Comparing scores with the other judges at the table after a category usually finds all of us pretty much on the same level.

Adding .5 scores can help with the breaking up of the 7,8,9 fixation, as giving the benefit to the cook will only grant them .5 instead of a whole score higher. Just my $0.02.

Pickin' Porkers
11-27-2012, 04:38 PM
I personally believe the MOST judges DO use the "entire range" CORRECTLY, as it is extremely rare to find an entry that is anything less than a "6" IMHO. I don't know what or if you judge, but if you think that the entries are that bad ("below average" or worse), then you haven't been at the contests that I've judged (or at least not at the same table)!!! Comparing scores with the other judges at the table after a category usually finds all of us pretty much on the same level.

Adding .5 scores can help with the breaking up of the 7,8,9 fixation, as giving the benefit to the cook will only grant them .5 instead of a whole score higher. Just my $0.02.

I am confident I have not judged as much as you! The problem with the differences between comparative judging and our current system of judging on its own merits is...... all too often a judge may score something a 9 because everything worked together to give that impression THEN, the next sample comes across and the same judge says WOW...now THAT is a 9! But wait, the second sample was MUCH better than the first one and I also gave THAT a 9.....something just is not right here....

Also, and I am sure everyone says this....I also use the range...albeit I cannot ever remember giving anything less than a 6 in taste....but there are simply too many instances where a judge sits on the fence between 8 & 9 or 7 & 8 which gives little maneuvering room for the undecided. To add to that in OUR case, we cook a heck of a lot and it takes one heck of a sample to get a 9 out of me in taste......in tenderness and appearance it is not quite so hard.

These ties really MUST go....too much cash is at stake for such a subjective application of the current judging process. KCBS started as a small club for good ol' boys but has grown way too fast and gotten much more serious than a backyard gathering of drinking buddies.

dmprantz
11-27-2012, 04:48 PM
all too often a judge may score something a 9 because everything worked together to give that impression THEN, the next sample comes across and the same judge says WOW...not THAT is a 9! But wait, the second sample was MUCH better than the first one and I also gave THAT a 9.....something just is not right here....

For what it's worth, my CBJ instructor addressed this topic, almost word for word. He said something to the effect of "If you taste something and say 'that's the best I ever had' and give it a 9, and then you taste the next thing and say 'wow, that's even better' you just give it a 9 and move on." I don't really judge, but as a cook who's won a few awards, I feel I've gotten to the point where I know what good competition BBQ is. I believe in the KCBS scoring theory where there is something that equals a 9, and when you taste it, you know, even if it isn't the best thing you've ever had. The issue with it, I guess, is that as cooks get better and better (a lot of cooks take classes to do that), there are a lot more entries that meet the same bar. Maybe that's where a "final table" with comparitive judging of other bodies is useful, or maybe what KCBS does works just fine. The real quest to me is how often do ties happen in the top 5 at a competition or top 2 overall? And by tie, I mean where it comes down to a coin toss.

dmp