PDA

View Full Version : Dropping the low score??


The_Kapn
11-29-2010, 02:01 PM
I have long been a fan of "dropping" the low score from judging.
I am re-thinking my position on that.

It just dawned on me that for every place(s) a team goes up, other team(s) must go down an equal number of places.
Maybe everyone else knew this and I am just a slow learner :redface:

Using KCBS as an example:
We have two teams with similar scores.
Both teams has solid 8's for the high 4 scores.
Team A has a 6-8-8 for the 5th score and a 5-8-8 for the low score.
Team B has a 7-8-8 for the 5th score and a 6-6-6 for the low score.
(Actually, any score for Team B below approsimately 5-8-8 would have the same result).

At this point, before dropping the low score, Team A is placed higher than Team B (without even figuring the "weighting" thing)

Now, drop the low score and Team B is placed higher than Team A.

That is wonderful for Team B, but not so "peachy keen" for Team A.

Put another (perverted but true) way, Team A was unfortunate to land on a table with consistant judging and Team B actually benefited by getting the low judge.

As I said, this happens because for every place a team moves up, another team must move down.
One "is helped" and one "gets hurt".

I think I am like most teams...I look at a dramatically low score and get p**sed because I got hurt by that Bozo. :evil: :evil:
I fail to notice when the low score is real close to the 5th score and has little effect on my placing or may have moved me up one or more places.

I did a spreadsheet on an FBA comp from a couple of years ago.
This is the chicken catagory only--the other catagories and overall were similar.
FBA uses a scale from 5 to 10 in 1/2 point increments.
200 is a perfect score.
On our score sheets, we get all the judges numbers to inclode a "weighted" total from each judge.

Column 1 is the origonal placing.
Column 3 is the origonal score.
Column 4 is the "weighted" score from the lowest judge.
That makes it easy to "drop" it, as I did to get Column 5 which is the new score after dropping the low one.
Column 6 is the ranking when useing the new score,
Column 7 is the change in places by that team from using all 6 scores to using only the top 5.
NOTE: If you add all the "+" and "-" moves, the sum is ZERO, as it should be.

Several folks would have moved "in and out of the Money" if the low scores were dropped.
"Big Poppa" would have lost $200 by moving from 2nd to 4th.
"Mt. Dora" would have picked up $300 by moving from 7th to 3rd.
Lots of other place changes that had a real impact on $ and TOTY points(to include the overall).

I am not advocating anything here. Just passing on what I learned.
I certainly have a new perspective on the topic.

There is still the emotional thing that a judge "having a bad day" and giving low scores should not have that much influence on the overall score.
I still have that emotional response, but the reality is that there is a price to be paid by dropping the low score.

I have been recommending that FBA drop the low score for a long time---but that is a "two edged sword" and I think (know) I will leave it alone now.


http://i217.photobucket.com/albums/cc143/the_kapn/WinterHavenscores.jpg
Just some thoughts.

TIM

Lake Dogs
11-29-2010, 02:15 PM
It's tough, for certain. No scoring system can be perfect. I'm not a fan of KCBS's
1-9 system because of variance. However, the throwing out the lowest takes care
of most of that problem. It's sort of a "darned-if-you-do", "darned-if-you-dont" thing.
It's ok to like it. :-)

In your example above, someone must've absolutely slammed Habeus Porkus. I dont
know why, perhaps they got a bad piece of chicken, or they got the 1 rubbery piece
of skin in the bunch... Whatever the reason, obviously one judge HATED their chicken,
while the other's apparently loved it. Yes, they benefited while the others suffered
as a result. It's the system....

rksylves
11-29-2010, 02:47 PM
I'm a big fan of dropping the low score. However, I also think that the highest score should also get dropped for the very same reason. I truly believe we are ALL looking for some type of consistency in the judging.

If the table has 6 judges at it and two of the 'edge' scores are dropped then the final tally will more fully reflect how the 'Q' really was.

More than a few times in discussions after the scoring is complete, invariably someone will say that #3 brisket was too spicy, sweet, etc. and score it way down, while the rest of the table completely disagrees. It also works the other way too. Somebody will say that rib #2 was the best that they have EVER had but the rest of the table totally disagrees.

Give me a true average of the table my stuff is at and the same for the other tables. Then it truly becomes a head to head competition. Not lopsided because one judge got a hair stuck somewhere.

Russ

roksmith
11-29-2010, 03:29 PM
I've got to dissagree.
Dropping the lowest score makes sense and protects team members from some dipstick judge who isn't judging on the same scale as everyone else.
But, dropping the top score takes you further from the average score and more toward the "middle score"
The more scores you count, the closer you are the the real average.

From my personal score sheets, I see many more odd lowball scores than high oddball scores. I very seldom see just a single 9 from the judges, but have seen several single 5s and 6s.

Ford
11-29-2010, 03:43 PM
I'm a big fan of dropping the low score. However, I also think that the highest score should also get dropped for the very same reason. I truly believe we are ALL looking for some type of consistency in the judging. Russ
With only 4 scores counting you'll have a whole lot of ties and we all know people hate that.

Jorge
11-29-2010, 04:22 PM
It was subtle, but I see your point. That being said, I still favor dropping the low score at this point. Show me the #'s from multiple comps, and all the categories and I might buy in. Chicken is too much of a crapshoot to form any sort of preliminary opinion on.

HarleyEarl
11-29-2010, 07:14 PM
My vote would be to keep it as is (dropping the low score). I read in a recent Bullsheet that KCBS is looking into inconsistency with the judges. Should be an interesting year.

Munchee
11-30-2010, 05:44 AM
After reading this thread I see one one thing that stands out. The Kapn has too much time on his hands!

Fatback Joe
12-01-2010, 10:01 AM
Thanks for using the results from a contest where I got 26th in chicken. :icon_blush::boxing::becky:

carlyle
12-01-2010, 11:46 AM
Interesting use of statistics. Need a bigger sample ( more categories, more contests)
before you can draw meaningful conclusions. I admire the time and effort it took to
do the chart.

From reading KCBS BOD minutes, several aspects of judging are being looked at and likely will be changed. Who knows what we will wind up with for next year.

This topic will surely be discussed at the annual banquet in KC in January. It will be
a lively and hot discussion.

Only time will tell if proposed changes will result in an improvement in the system of scoring.

The_Kapn
12-02-2010, 04:55 PM
Thanks for using the results from a contest where I got 26th in chicken. :icon_blush::boxing::becky:

Joe, I know what you mean.
I was right there with you at 23rd at this one. :redface:
But, we were in good company--see Myron at #31 that day :-D

I need to clarify something here.
There are no statistics involved whatsoever.
The only math is subtraction (Jethro Bodine called it "take-aways") and that is all.
There are no averages, means, modes, medians, standard deviations, etc--just subtract the low score from the total and then re-rank the teams.
Sample size, meat selected, size of event, or FBA vs KCBS means nothing. Absolutely nothing.

The process here applies to any scoring system that uses an array of numbers and then deletes the low one.
I can only speak to KCBS and FBA because those are the two systems I cook and judge in.

I know the KCBS cooks have a hard time visualizing this because KCBS only provides detailed scoring (by judge) for the teams score, not everyones.

Here is a FBA score sheet from Jesup, GA in 2009.
It might help visualize the facts a little better.

http://i217.photobucket.com/albums/cc143/the_kapn/jesupPork2009rawscores.jpg

This is the first page only for the pork catagory.
For Jesup, we recieved a packet with 13 pages of detail plus a nice cover sheet.
Thank You FBA :clap2: :clap2:

With this detail, we can see if we were on a table that scored low.
Maybe they got all the mediocre entries, or maybe they were just stingy with scores.
We can also see if a judge hit us really hard and whether they did that to several teams, or just us.

Here is the same scores dropped into a spreadsheet and resorted with the low score dropped.

http://i217.photobucket.com/albums/cc143/the_kapn/JesupPork2009.jpg
I picked this one at random, but I would have been the big loser with 5 places gone. Would probably have lost my RGC also, so a loss of almost $2000 and precious TOTY points that day :redface:

A couple of interesting things here.
Git R Smoked stayed in 1st because he had a huge lead over DW.
Mr Cook moved up a lot because he had one judge that just killed him with that 26.XXX compared to the rest of his scores.
We moved down so far because all 6 of our scores were pretty much in agreement. So, when the low score is dropped, it hurt us because all of the teams near us in scoring had a really low score to drop. So, they moved up and we moved down.

Like I said above, Dropping the low score is a 2 edged sword.

Finally, let me try a visual. That might help understand the process.

Lets say the organizer calls all 23 teams up on stage and then arranges them in the order of finish prior to dropping the low score.
Then they move the teams up the line based on the new placing with the low score dropped.
You are me (Flirtin' with Disaster) that day and start 4th in line.
Git R Smoked stays in 1st place. Good for him!
Munchee moves up from 3rd to 2nd and we all clap for him. Except DW that is cause he just lost a place.
DW is right behind Munchee now (instead on in front of him) and stays there.
He goes back and brings Prime Time and Saltine up to 4th place and you move back to 5th to make room. Still clapping some, but WTF :evil:
Then he calls Forrest up from 8th to 5th. Good for him and you are happy for him, but you are now back to 6th and wondering when the bleeding will stop :evil:
Next Mr Cook moves from 10th to 6th and you slide back one more spot. DAMN :evil:
Florida Skin and Bones has been right behind you all this time, but they are now called forward to 7th place and their slide is over. But not yours!
Bethel Smokers was right behind Florida Skin and Bones and you, but now they are called forward to the 8th place. You are wondering if they have forgotten you!!
Finally, you are in 9th place and don't have to move back anymore. WHEW, what a ride!!! :evil:

I could keep on, and put you to sleep even more, but it is all the same drill from there on back to 23rd.

Folks, I am not advocating anything here.
It is just that I used to think that dropping the low score was a good thing and "leveled the field".
Seems to make sense until you evaluate it with facts.
But, It is obvious that that is not the panacea or "fix" that I thought it was.

Simply put, for every place a team moves up, some other team has to move down one.
Great for you if you move up.
Not so great if you are moved down.

Nothings perfect, for sure.

Looks like getting more consistancy with the judging results would be a productive project, not necessarily juggling the numbers.

TIM

bbq.tom
12-03-2010, 09:33 AM
I like the idea of dropping not only the low score, but the high score as well. I realize that this will make more "ties" than current situation, but it is fairer in the long run.

The last comp that I judged (KCBS) had one individual at my table who thought that one rib was the best he had ever eaten, even though the other five of us agreed that it was NOT GOOD (5s to 7s). Keeping his '9' skewed the "average" and resulted in the entry receiving higher than deserved ranking.

A total revamp of the scoring system using a 3-15 scale rather than a 3-9 scale (KCBS) would help differentiate between entries, but that is a lot of bother. I truly don't see any "good" change without a major revamp of the entire system. Meanwhile, keep dropping the low score.

DawgPhan
12-03-2010, 02:02 PM
Dropping the low score makes sure that 1 bad judge doesnt ruin your weekend. You could have great food @ an FBA contest, but if Judge 5 on Chicken hates your chicken he can wreck your chicken and your overall. If you happen to run into 2 bad judges on 1 weekend you are really sunk in the overall. If you hit 2 bad judges on one table, you need to cook better food.

Lake Dogs
12-03-2010, 03:02 PM
Tim, You said it great, it's a double edged sword for certain. It does take that one
judge out (the misbehaving one), but at a cost to others... I loved what you
said regarding:
> Nothings perfect, for sure.
>
> Looks like getting more consistency with the judging results would be a
> productive project, not necessarily juggling the numbers.

That's the key. Training.

For that matter, who's to say that the 1 rogue judge wasn't rogue afterall. I'm
serious when I say we had a contest (although it was MBN) where coming off the
table 4 judges gave this one Q 9's and 10's, and the one "rogue" judge gave it
7's and 8's. So, the rogue judge was pulled aside and asked why. The answer:
The BBQ tastes like lighterfluid. Sure enough they sampled the remaining Q and
it indeed tasted like lighterfluid... So, coming away from this we learned that 4 out
of 5 judges prefer lighterfluid.

:-)