The BBQ BRETHREN FORUMS.


Forum Portal Recipes Smoke Signals Magazine Welocme Merchandise Associations Purchase Subscription Brethren Banners
Go Back   The BBQ BRETHREN FORUMS. > Discussion Area > Competition BBQ

Notices

Competition BBQ *On Topic Only* Discussion regarding all aspects of Competition BBQ. Experiences competing or visiting, questions, getting started, Equipment, announcements of events, Results, Reviews, Planning, etc. Questions here will be responded to with competition BBQ in mind.


Reply
Thread Tools
Unread 11-29-2010, 02:01 PM   #1
The_Kapn
Moderator Emeritus

 
The_Kapn's Avatar
 
Join Date: 04-08-04
Location: Marianna, FL
Downloads: 0
Uploads: 0
Default Dropping the low score??

I have long been a fan of "dropping" the low score from judging.
I am re-thinking my position on that.

It just dawned on me that for every place(s) a team goes up, other team(s) must go down an equal number of places.
Maybe everyone else knew this and I am just a slow learner

Using KCBS as an example:
We have two teams with similar scores.
Both teams has solid 8's for the high 4 scores.
Team A has a 6-8-8 for the 5th score and a 5-8-8 for the low score.
Team B has a 7-8-8 for the 5th score and a 6-6-6 for the low score.
(Actually, any score for Team B below approsimately 5-8-8 would have the same result).

At this point, before dropping the low score, Team A is placed higher than Team B (without even figuring the "weighting" thing)

Now, drop the low score and Team B is placed higher than Team A.

That is wonderful for Team B, but not so "peachy keen" for Team A.

Put another (perverted but true) way, Team A was unfortunate to land on a table with consistant judging and Team B actually benefited by getting the low judge.

As I said, this happens because for every place a team moves up, another team must move down.
One "is helped" and one "gets hurt".

I think I am like most teams...I look at a dramatically low score and get p**sed because I got hurt by that Bozo.
I fail to notice when the low score is real close to the 5th score and has little effect on my placing or may have moved me up one or more places.

I did a spreadsheet on an FBA comp from a couple of years ago.
This is the chicken catagory only--the other catagories and overall were similar.
FBA uses a scale from 5 to 10 in 1/2 point increments.
200 is a perfect score.
On our score sheets, we get all the judges numbers to inclode a "weighted" total from each judge.

Column 1 is the origonal placing.
Column 3 is the origonal score.
Column 4 is the "weighted" score from the lowest judge.
That makes it easy to "drop" it, as I did to get Column 5 which is the new score after dropping the low one.
Column 6 is the ranking when useing the new score,
Column 7 is the change in places by that team from using all 6 scores to using only the top 5.
NOTE: If you add all the "+" and "-" moves, the sum is ZERO, as it should be.

Several folks would have moved "in and out of the Money" if the low scores were dropped.
"Big Poppa" would have lost $200 by moving from 2nd to 4th.
"Mt. Dora" would have picked up $300 by moving from 7th to 3rd.
Lots of other place changes that had a real impact on $ and TOTY points(to include the overall).

I am not advocating anything here. Just passing on what I learned.
I certainly have a new perspective on the topic.

There is still the emotional thing that a judge "having a bad day" and giving low scores should not have that much influence on the overall score.
I still have that emotional response, but the reality is that there is a price to be paid by dropping the low score.

I have been recommending that FBA drop the low score for a long time---but that is a "two edged sword" and I think (know) I will leave it alone now.



Just some thoughts.

TIM
__________________
"Flirtin' with Disaster" BBQ Team (RETIRED)
FBA and KCBS Cook and Judge.
Former owner of a WSM, a Smokey Joe, a Charbroil Commercial gasser (junk), the legendary "StudeDera", a Fast Eddy PG500, and Sherman.
Now cooking with a Yoder YS640
Proud Pellet guy cooking on real wood.
The_Kapn is offline   Reply With Quote


Unread 11-29-2010, 02:15 PM   #2
Lake Dogs
Quintessential Chatty Farker

 
Lake Dogs's Avatar
 
Join Date: 07-14-09
Location: Lake Sinclair, GA
Downloads: 0
Uploads: 0
Default

It's tough, for certain. No scoring system can be perfect. I'm not a fan of KCBS's
1-9 system because of variance. However, the throwing out the lowest takes care
of most of that problem. It's sort of a "darned-if-you-do", "darned-if-you-dont" thing.
It's ok to like it. :-)

In your example above, someone must've absolutely slammed Habeus Porkus. I dont
know why, perhaps they got a bad piece of chicken, or they got the 1 rubbery piece
of skin in the bunch... Whatever the reason, obviously one judge HATED their chicken,
while the other's apparently loved it. Yes, they benefited while the others suffered
as a result. It's the system....
__________________
Hance - Lake Dogs Cooking Team - MiM/MBN/GBA CBJ and comp cook
Lake Sinclair, GA (strategically about an hour from everywhere)
Lake Dogs is offline   Reply With Quote


Unread 11-29-2010, 02:47 PM   #3
rksylves
Full Fledged Farker
 
Join Date: 07-28-09
Location: Melbourne, Florida
Downloads: 0
Uploads: 0
Default

I'm a big fan of dropping the low score. However, I also think that the highest score should also get dropped for the very same reason. I truly believe we are ALL looking for some type of consistency in the judging.

If the table has 6 judges at it and two of the 'edge' scores are dropped then the final tally will more fully reflect how the 'Q' really was.

More than a few times in discussions after the scoring is complete, invariably someone will say that #3 brisket was too spicy, sweet, etc. and score it way down, while the rest of the table completely disagrees. It also works the other way too. Somebody will say that rib #2 was the best that they have EVER had but the rest of the table totally disagrees.

Give me a true average of the table my stuff is at and the same for the other tables. Then it truly becomes a head to head competition. Not lopsided because one judge got a hair stuck somewhere.

Russ
__________________
Lead Cook 'Second Star BBQ' using a Big Orange 'Prince', 22 OTG, SJS FBA & KCBS CBJ
rksylves is offline   Reply With Quote


Unread 11-29-2010, 03:29 PM   #4
roksmith
is one Smokin' Farker
 
Join Date: 04-21-09
Location: Marietta, Ohio
Downloads: 0
Uploads: 0
Default

I've got to dissagree.
Dropping the lowest score makes sense and protects team members from some dipstick judge who isn't judging on the same scale as everyone else.
But, dropping the top score takes you further from the average score and more toward the "middle score"
The more scores you count, the closer you are the the real average.

From my personal score sheets, I see many more odd lowball scores than high oddball scores. I very seldom see just a single 9 from the judges, but have seen several single 5s and 6s.
__________________
4000 sq inch Wood Burner named Brutus - KCSB member/judge# 32129
roksmith is offline   Reply With Quote


Thanks from:--->
Unread 11-29-2010, 03:43 PM   #5
Ford
is Blowin Smoke!
 
Ford's Avatar
 
Join Date: 04-14-07
Location: Lakeland Florida
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by rksylves View Post
I'm a big fan of dropping the low score. However, I also think that the highest score should also get dropped for the very same reason. I truly believe we are ALL looking for some type of consistency in the judging. Russ
With only 4 scores counting you'll have a whole lot of ties and we all know people hate that.
__________________
Ford
Retired competition cook. BBQ mentor.
Ford is offline   Reply With Quote


Unread 11-29-2010, 04:22 PM   #6
Jorge
somebody shut me the fark up.
 
Jorge's Avatar
 
Join Date: 01-23-04
Location: DFW, San AntonioTx
Downloads: 0
Uploads: 0
Default

It was subtle, but I see your point. That being said, I still favor dropping the low score at this point. Show me the #'s from multiple comps, and all the categories and I might buy in. Chicken is too much of a crapshoot to form any sort of preliminary opinion on.
__________________
You can't be a real country unless you have a beer and an airline - it helps if you have some kind of a football team, or some nuclear weapons, but at the very least you need a beer. --Frank Zappa

BOOGITY, BOOGITY, BOOGITY!!!

Recipient of a Huggies box!

Shut up, and cook!!!!
Jorge is online now   Reply With Quote


Unread 11-29-2010, 07:14 PM   #7
HarleyEarl
Full Fledged Farker
 
Join Date: 10-29-06
Location: Paola, Kansas
Downloads: 0
Uploads: 0
Default

My vote would be to keep it as is (dropping the low score). I read in a recent Bullsheet that KCBS is looking into inconsistency with the judges. Should be an interesting year.
__________________
Marc
Operation BBQ Relief Vice President
Operation BBQ Relief Founding Member
CBJ
Six UDS's, Traeger 125, Traeger Jr, two HD Off-sets, and a home-made Trailer mounted smoker
Funny Farm BBQ
[URL="http://www.FunnyFarmBBQ.us"]www.FunnyFarmBBQ.us[/URL]
[COLOR=darkred][B]***2010 Great American Barbecue Baste of the Year***[/B][/COLOR]
HarleyEarl is offline   Reply With Quote


Unread 11-30-2010, 05:44 AM   #8
Munchee
Got Wood.
 
Join Date: 05-25-09
Location: Lake City Fl
Downloads: 0
Uploads: 0
Default Hobbies

After reading this thread I see one one thing that stands out. The Kapn has too much time on his hands!
Munchee is offline   Reply With Quote


Unread 12-01-2010, 10:01 AM   #9
Fatback Joe
is Blowin Smoke!

 
Fatback Joe's Avatar
 
Join Date: 01-07-08
Location: Memphis, TN
Downloads: 1
Uploads: 0
Default

Thanks for using the results from a contest where I got 26th in chicken.
__________________
FBJ

WSM
Medium Spicewine
Stumps GF223
KCBS, NEBS Member
Fatback Joe is offline   Reply With Quote


Unread 12-01-2010, 11:46 AM   #10
carlyle
Full Fledged Farker

 
carlyle's Avatar
 
Join Date: 10-06-10
Location: Owatonna, MN
Downloads: 0
Uploads: 0
Default

Interesting use of statistics. Need a bigger sample ( more categories, more contests)
before you can draw meaningful conclusions. I admire the time and effort it took to
do the chart.

From reading KCBS BOD minutes, several aspects of judging are being looked at and likely will be changed. Who knows what we will wind up with for next year.

This topic will surely be discussed at the annual banquet in KC in January. It will be
a lively and hot discussion.

Only time will tell if proposed changes will result in an improvement in the system of scoring.
__________________
Carlyle, KCBS, CBJ, Organizer ( Judge Chair)
carlyle is offline   Reply With Quote


Unread 12-02-2010, 04:55 PM   #11
The_Kapn
Moderator Emeritus

 
The_Kapn's Avatar
 
Join Date: 04-08-04
Location: Marianna, FL
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by Fatback Joe View Post
Thanks for using the results from a contest where I got 26th in chicken.
Joe, I know what you mean.
I was right there with you at 23rd at this one.
But, we were in good company--see Myron at #31 that day

I need to clarify something here.
There are no statistics involved whatsoever.
The only math is subtraction (Jethro Bodine called it "take-aways") and that is all.
There are no averages, means, modes, medians, standard deviations, etc--just subtract the low score from the total and then re-rank the teams.
Sample size, meat selected, size of event, or FBA vs KCBS means nothing. Absolutely nothing.

The process here applies to any scoring system that uses an array of numbers and then deletes the low one.
I can only speak to KCBS and FBA because those are the two systems I cook and judge in.

I know the KCBS cooks have a hard time visualizing this because KCBS only provides detailed scoring (by judge) for the teams score, not everyones.

Here is a FBA score sheet from Jesup, GA in 2009.
It might help visualize the facts a little better.



This is the first page only for the pork catagory.
For Jesup, we recieved a packet with 13 pages of detail plus a nice cover sheet.
Thank You FBA

With this detail, we can see if we were on a table that scored low.
Maybe they got all the mediocre entries, or maybe they were just stingy with scores.
We can also see if a judge hit us really hard and whether they did that to several teams, or just us.

Here is the same scores dropped into a spreadsheet and resorted with the low score dropped.


I picked this one at random, but I would have been the big loser with 5 places gone. Would probably have lost my RGC also, so a loss of almost $2000 and precious TOTY points that day

A couple of interesting things here.
Git R Smoked stayed in 1st because he had a huge lead over DW.
Mr Cook moved up a lot because he had one judge that just killed him with that 26.XXX compared to the rest of his scores.
We moved down so far because all 6 of our scores were pretty much in agreement. So, when the low score is dropped, it hurt us because all of the teams near us in scoring had a really low score to drop. So, they moved up and we moved down.

Like I said above, Dropping the low score is a 2 edged sword.

Finally, let me try a visual. That might help understand the process.

Lets say the organizer calls all 23 teams up on stage and then arranges them in the order of finish prior to dropping the low score.
Then they move the teams up the line based on the new placing with the low score dropped.
You are me (Flirtin' with Disaster) that day and start 4th in line.
Git R Smoked stays in 1st place. Good for him!
Munchee moves up from 3rd to 2nd and we all clap for him. Except DW that is cause he just lost a place.
DW is right behind Munchee now (instead on in front of him) and stays there.
He goes back and brings Prime Time and Saltine up to 4th place and you move back to 5th to make room. Still clapping some, but WTF
Then he calls Forrest up from 8th to 5th. Good for him and you are happy for him, but you are now back to 6th and wondering when the bleeding will stop
Next Mr Cook moves from 10th to 6th and you slide back one more spot. DAMN
Florida Skin and Bones has been right behind you all this time, but they are now called forward to 7th place and their slide is over. But not yours!
Bethel Smokers was right behind Florida Skin and Bones and you, but now they are called forward to the 8th place. You are wondering if they have forgotten you!!
Finally, you are in 9th place and don't have to move back anymore. WHEW, what a ride!!!

I could keep on, and put you to sleep even more, but it is all the same drill from there on back to 23rd.

Folks, I am not advocating anything here.
It is just that I used to think that dropping the low score was a good thing and "leveled the field".
Seems to make sense until you evaluate it with facts.
But, It is obvious that that is not the panacea or "fix" that I thought it was.

Simply put, for every place a team moves up, some other team has to move down one.
Great for you if you move up.
Not so great if you are moved down.

Nothings perfect, for sure.

Looks like getting more consistancy with the judging results would be a productive project, not necessarily juggling the numbers.

TIM
__________________
"Flirtin' with Disaster" BBQ Team (RETIRED)
FBA and KCBS Cook and Judge.
Former owner of a WSM, a Smokey Joe, a Charbroil Commercial gasser (junk), the legendary "StudeDera", a Fast Eddy PG500, and Sherman.
Now cooking with a Yoder YS640
Proud Pellet guy cooking on real wood.
The_Kapn is offline   Reply With Quote


Thanks from: --->
Unread 12-03-2010, 09:33 AM   #12
bbq.tom
is one Smokin' Farker

 
bbq.tom's Avatar
 
Join Date: 11-23-10
Location: Hillsborough, NC (UNC TarHeel country!)
Downloads: 1
Uploads: 0
Default

I like the idea of dropping not only the low score, but the high score as well. I realize that this will make more "ties" than current situation, but it is fairer in the long run.

The last comp that I judged (KCBS) had one individual at my table who thought that one rib was the best he had ever eaten, even though the other five of us agreed that it was NOT GOOD (5s to 7s). Keeping his '9' skewed the "average" and resulted in the entry receiving higher than deserved ranking.

A total revamp of the scoring system using a 3-15 scale rather than a 3-9 scale (KCBS) would help differentiate between entries, but that is a lot of bother. I truly don't see any "good" change without a major revamp of the entire system. Meanwhile, keep dropping the low score.
bbq.tom is offline   Reply With Quote


Unread 12-03-2010, 02:02 PM   #13
DawgPhan
is one Smokin' Farker
 
Join Date: 08-09-07
Location: Atlanta, Ga
Downloads: 0
Uploads: 0
Default

Dropping the low score makes sure that 1 bad judge doesnt ruin your weekend. You could have great food @ an FBA contest, but if Judge 5 on Chicken hates your chicken he can wreck your chicken and your overall. If you happen to run into 2 bad judges on 1 weekend you are really sunk in the overall. If you hit 2 bad judges on one table, you need to cook better food.
DawgPhan is offline   Reply With Quote


Thanks from:--->
Unread 12-03-2010, 03:02 PM   #14
Lake Dogs
Quintessential Chatty Farker

 
Lake Dogs's Avatar
 
Join Date: 07-14-09
Location: Lake Sinclair, GA
Downloads: 0
Uploads: 0
Default

Tim, You said it great, it's a double edged sword for certain. It does take that one
judge out (the misbehaving one), but at a cost to others... I loved what you
said regarding:
> Nothings perfect, for sure.
>
> Looks like getting more consistency with the judging results would be a
> productive project, not necessarily juggling the numbers.

That's the key. Training.

For that matter, who's to say that the 1 rogue judge wasn't rogue afterall. I'm
serious when I say we had a contest (although it was MBN) where coming off the
table 4 judges gave this one Q 9's and 10's, and the one "rogue" judge gave it
7's and 8's. So, the rogue judge was pulled aside and asked why. The answer:
The BBQ tastes like lighterfluid. Sure enough they sampled the remaining Q and
it indeed tasted like lighterfluid... So, coming away from this we learned that 4 out
of 5 judges prefer lighterfluid.

:-)
__________________
Hance - Lake Dogs Cooking Team - MiM/MBN/GBA CBJ and comp cook
Lake Sinclair, GA (strategically about an hour from everywhere)
Lake Dogs is offline   Reply With Quote


Reply

Similar Threads
Thread Thread Starter Forum Replies Last Post
HELP! My WSM temp is dropping FAST! GreasePig Q-talk 23 12-07-2009 05:54 PM
How could meat temp be dropping? kvothe Q-talk 8 11-08-2009 04:03 PM
Backwoods temps keep dropping butts a fire Q-talk 16 06-18-2008 03:57 PM
Low Sodium/Low fat Q? Jorge Q-talk 21 04-29-2004 03:23 PM

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Loading



All times are GMT -5. The time now is 07:48 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Lite) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
2003 -2012 © BBQ-Brethren Inc. All rights reserved. All Content and Flaming Pig Logo are registered and protected under U.S and International Copyright and Trademarks. Content Within this Website Is Property of BBQ Brethren Inc. Reproduction or alteration is strictly prohibited.
no new posts