ブタ
The BBQ BRETHREN FORUMS.


Forum Portal Recipes Smoke Signals Magazine Welocme Merchandise Associations Purchase Subscription Brethren Banners
Go Back   The BBQ BRETHREN FORUMS. > Discussion Area > Competition BBQ

Notices

Competition BBQ *On Topic Only* Discussion regarding all aspects of Competition BBQ. Experiences competing or visiting, questions, getting started, Equipment, announcements of events, Results, Reviews, Planning, etc. Questions here will be responded to with competition BBQ in mind.


Reply
Thread Tools
Unread 08-13-2004, 10:19 PM   #1
The_Kapn
Moderator Emeritus

 
The_Kapn's Avatar
 
Join Date: 04-08-04
Location: Marianna, FL
Downloads: 0
Uploads: 0
Default Question "Split" from the "Judges Tent"

I split this off.
Great basic thread and I am learning a lot!

But, here is a more detailed and specific question for our Judges and Experienced Competetors.

Please bear with me- this is complicated.

Background-
I was a Ladies Gymnastic Judge (USGF) in the 80's for 3 years.
Sounds like a similar system for training and certifying Judges.
Classes, video, study, pracitce, and a "killer" written test.
Start at the lower levels and, with "experience, a track record, and more testing", move up in the classes and authority.
The exact details are not important- the concept is. There are rules, training, and evaluation before you become a "Judge".

We (the USGF and FIA) had a final scoring system based on the judges being "In Range".

Basically, all judges must be "in range" with each other and the "head judge". The range depended on the score. For example, for "head judge" score of 9.5 to 10- we had to be within .1 or a "conference" was called.
This acceptable range went up to .5 points at a 7 score from the head judge (or something like that). The numbers are not important.

If a conference was required, the judges discussed (in private but face to face) just what they had seen to generate the score.
Normally a matter of "viewing angle" or rules interpretation.
I will save you the details.

The bottom line was--if the Judges observed or scored things outside of an acceptable range--It had to be resolved before the score for the girl was posted.

Lots of subjectivity--seems just like judging good Q

But, I learned that Judges almost always (but they are not perfect) tried to do the right thing, within the rules, to render a valid and accurate score!
Human evaluation is subjective by nature and can never be reduced to totally objective terms.
If so- hire a computer!

I have seen remarks (in some BBQ rules) about throwing out the lowest scores to resolve ties, etc. I saw some comments in the origonal thread about spices, texture, presentation, etc. that raised my curiosity!
Other than that, I have no practical experience. Help me out here.

So- finally- here is/are the questions for the guys who have been there!

From meet to meet, are the judges normally in agreement on scores?
How about within a meet if they are way off- is anything done?
If a judge is "way low" or "way high", is the score resolved?
If a judge seems to "favor" or "dis-favor"some item, i.e. presentaion or spice leve, is action taken on the spot to equalize or normalise the scoring.


I am not advocating adoptance of rule changes in any way shape or form!
I am just curious about how the current system really works.

DF and I am entering that world as comtestants and appreciate any feedback on how the system works.

Thanks,
TIM

Just wondering
__________________
"Flirtin' with Disaster" BBQ Team (RETIRED)
FBA and KCBS Cook and Judge.
Former owner of a WSM, a Smokey Joe, a Charbroil Commercial gasser (junk), the legendary "StudeDera", a Fast Eddy PG500, and Sherman.
Now cooking with a Yoder YS640
Proud Pellet guy cooking on real wood.
The_Kapn is offline   Reply With Quote


Unread 08-13-2004, 10:32 PM   #2
willkat98
somebody shut me the fark up.
 
willkat98's Avatar
 
Join Date: 08-11-03
Location: Chicago Western Burbs, but always south of Madison Ave.
Downloads: 0
Uploads: 0
Default

Never judged official Tim, but I am certified.

So I'll try to answer from a "novice" perspectiive.

the answer: "no"

Judging in KCBS is an individual thing, with no interaction with others. Its not allowed. You have 6 entries in front of you, you bite, you think, you judge. Talking is not permitted until after the round (is this right Brooklyn)

Way low or Way high, that is the luck of the draw. You (cook) use a salty rub. Judge A likes satly, so scores an 8, judge B, dislikes salt and scores a 4 (on overall taste).

Judgeing, to me (certified but virgin) seems to be the luck of the draw
__________________
.
Bill-Chicago

"Bitch..." - Jessie Pinkman

Bandera, M-BGE, WSM, Gold Kettle, Kenmore Gasser, 36Qt Turkey Fryer, ECB, Dutch Oven
willkat98 is offline   Reply With Quote


Unread 08-13-2004, 10:37 PM   #3
BBQchef33
Grand Poobah and Site Admin
 
BBQchef33's Avatar
 
Join Date: 08-11-03
Location: Long Island, NY
Downloads: 13
Uploads: 4
Default

only thing i can comment here is that after evaluating the details of the scores from the CT competition, i learned that there is no normalization. My brisket scores ranged from a 4-6-6 to an 9-9-8. It came in second, but one judge rated it very low, while the other scores were 6-8-8, 7-9-8, 7-8-9, 8-9-9. the ranges were pretty extreme.. Same on the ribs. Seesm its every judge for themselves. They are also not allowed to discuss the entries during judging.

Also, if I understood an explanation properly. All entries start at a 5 which is average and judges rate up or down. This is a new way of doing it and alot of judges still do it the old way where everything starts as a 9 and they rate downwards from there.
__________________
Site Administrator and
Grand PooBah

CBJ with a Fuzzy Blue Hat, 18 Foot Competition Trailer, Customized Klose BYC, Custom Klose built MoAB (Mother of All Banderas) passed on to Big Dog, 1 Double Barreled Lang 84, 1 Heavily Modified Bionic Bandera, 1 Custom Super Medium Stickburning Spicewine w/stoker, 1 XL BGE, 1 Mini BGE, 1 Pit Barrel Cooker, 3 WSMs, 3 Weber Kettles, an NB Hondo, A Modified Brinkman Horizontal, DCS 48" Grill, a Broilmaster P3, a covered, pellet pooping FEC100, and our duck died. :(

News Flash:
"A mans worth is judged by the weight of his integrity "

You know your getting older when you choose your cereal for the fiber, not the toy


Smoke on KC.
WWW.BBQ-BRETHREN.COM
BBQchef33 is offline   Reply With Quote


Unread 08-13-2004, 11:22 PM   #4
mook
Full Fledged Farker
 
Join Date: 04-01-04
Location: Los Angeles, CA
Downloads: 0
Uploads: 0
Default

I agree with Phil. In our experience, our scores varied widley as well. One of the other competitors, with tons more experience and more than a few Grand Championships under his belt, said that with the old system if the scores were 5 and under, or something to that effect, the low scoring judge had to justify his score. In the current system there is no explanation required for low or widely varying scores.

That's how I remember it. maybe horn or bigmofo can correct me or add their recolections.
__________________
CB Bandera w/nameplate mod, 20 gallon barrel pit Weber Gold Kettle, and WSM's.
mook is offline   Reply With Quote


Unread 08-14-2004, 08:08 AM   #5
BrooklynQ
Banned
 
BrooklynQ's Avatar
 
Join Date: 07-18-04
Downloads: 0
Uploads: 0
Default

There is no talking allowed when you are judging. No hand gestures, no face gestures and almost no eye contact. We are allowed to discuss only AFTER the score cards go in. In CT, we did not discuss scores, we did talk about which we thought best, worst or odd.

We were told in CT to start at 6. "6 being average, 9 better than mom's apple pie and 2 being rot gut" were the exact words used. The old scoring system started at 9 and worked down.

I know that i gave one entry a 6,3,3 and no one questioned me about it. It is subjective and somewhat the luck of the draw.
BrooklynQ is offline   Reply With Quote


Unread 08-14-2004, 09:31 AM   #6
jminion
is one Smokin' Farker
 
jminion's Avatar
 
Join Date: 07-07-04
Location: Federal Way WA
Downloads: 0
Uploads: 0
Default

The correct instruction to the judges is, "9 is excellent, 5 is average and 2 is just bad, we will start at 6 and judge up or down based on how you, the judge, feel about the entry". A 1 or DQ can only be given by the KCBS Rep and they will give you the correct way to score the entry.

This being the first season under the new guidelines it is not surprising that the scores are all over the board. It use to be that a total score of mid 700s would be needed to win a lot of competitions, now a score in the 630 to 650 range can get the job done.
The old system you would start at nine and take a point off if you found a problem with the entry, the range was 9-8-7 this leads to a large number of ties and that's the reason for the change to the new starting point and instructions.
__________________
Jim
www.twoloosescrewsbbq.com
Ole Hickory EL/EDx
WSM
Primos
Klose
couple of Ranch kettles
jminion is offline   Reply With Quote


Unread 08-14-2004, 09:54 AM   #7
BrooklynQ
Banned
 
BrooklynQ's Avatar
 
Join Date: 07-18-04
Downloads: 0
Uploads: 0
Default

According to this month's Bullsheet - (August 2004 -page 10):
"Contest reps have been instructed to repeat the following after the CD meeting is completed: 'The entry starts as a 6, and is judged up or down according to the quality of the sample'." It also says that "5 is considered average."

One thing I didn't know that was also in the article: "Cooks are welcome to listen to the Judge's meeting and Table Captain's meeting." That's something I didn't see any of the cooks take advantage of. I would if possible.
BrooklynQ is offline   Reply With Quote


Unread 08-14-2004, 04:08 PM   #8
willkat98
somebody shut me the fark up.
 
willkat98's Avatar
 
Join Date: 08-11-03
Location: Chicago Western Burbs, but always south of Madison Ave.
Downloads: 0
Uploads: 0
Default

It was good that Greg and I started the judge thing this year.

I'm not sure if other judges/competitors (like Jim M) like the new system, but since it is the only one Greg and I know, it seems fair. Start at a mid point, and award better, and detract from lesser to score, rather than everything be 9 and detract only the bad points. This new system seems to award a superior product, where as if you started at 9 with a good product, but couldnt find alot wrong with it, you would end up with an 8. In the new system, a good product would be a 6, maybe a 7, but the "great" product would be an 8 or 9 being excellent.

So I like the new system just from a theoretical standpoint, even though I only have applied it in a "mock" situation at class.

Just as info, Greg and I were instructed by Mike and Theresa Lake who run the Illinois Championship (and something in Iowa) and the Butt to Butt competition you read in this months BullSheet.

Second sidebar: any Brethren who are or are Not considering competing, I would strongly suggest taking a judging class.

Not only can you see how its done if you chose to compete to see what they are looking for, you can also be a judge for future contests, or even just play around at home and rate your own Q's. Many a time have I found myself deducting points on a butt for tenderness or awarding myself for taste. Only thing I skip is appearance, since I eat it right off the grate :)
__________________
.
Bill-Chicago

"Bitch..." - Jessie Pinkman

Bandera, M-BGE, WSM, Gold Kettle, Kenmore Gasser, 36Qt Turkey Fryer, ECB, Dutch Oven
willkat98 is offline   Reply With Quote


Unread 08-14-2004, 05:39 PM   #9
midnight
Full Fledged Farker
 
midnight's Avatar
 
Join Date: 10-17-03
Location: small town South Dakota
Downloads: 0
Uploads: 0
Default

Having judged with the "old" system (start at 9 and go down) and the "new" system ( start at 6) I have to say that I liked the old system better. It was easy to look/taste a piece of meat and say to yourself ,"the meat looks burnt, thats a point off." It is much harder to label something as "average" and then decide how much above or below average it is. I also think this system makes you want to compare the entrys more, and that is a big no no.

I do understand the need for a system that has less ties in the final scores, and this might be the best way to do it.

From what I have heard around the judges tent after competitions, this new system still gives a fairly accurate final score among all entrys.

With any system there is bound to be some fluctuation in scores from one judge to another. My understanding is that if one judge is continualy scoring low on ALL entrys at more than one contest, he or she may be asked not to judge, or asked to explain thier judging method.

case in point: I had one judge at my table that didn't like coconut. When the entrys for the deserts (not an official KCBS catagory) came in, one persons entry was custard served in coconut shells. This guy said he marked lower scores just beacuse he didn't like coconut!
__________________
Some people are like Slinkys, they are good for nothing but can still bring a smile to your face when you push them down the stairs.


Down to one lonely ol' Bandera.
midnight is offline   Reply With Quote


Unread 08-16-2004, 03:53 PM   #10
jmcgrath
Is lookin for wood to cook with.
 
Join Date: 07-26-04
Location: Concord, MA
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by willkat98
It was good that Greg and I started the judge thing this year.

I'm not sure if other judges/competitors (like Jim M) like the new system, but since it is the only one Greg and I know, it seems fair.
There are two Jim Ms, McGrath and Minion. I think the new system will be much better in the long run. Right now a lot of more experienced judges are going through a relearning process.

On the whole, teams that did well under the old system continue to do well. Average and medocore teams used to get clumped together. Now they are more spread out.

Jim
jmcgrath is offline   Reply With Quote


Unread 08-16-2004, 04:06 PM   #11
willkat98
somebody shut me the fark up.
 
willkat98's Avatar
 
Join Date: 08-11-03
Location: Chicago Western Burbs, but always south of Madison Ave.
Downloads: 0
Uploads: 0
Default

Oops
Both of you have some great knowledge though.

Thanks Jim McG!
__________________
.
Bill-Chicago

"Bitch..." - Jessie Pinkman

Bandera, M-BGE, WSM, Gold Kettle, Kenmore Gasser, 36Qt Turkey Fryer, ECB, Dutch Oven
willkat98 is offline   Reply With Quote


Unread 08-16-2004, 04:45 PM   #12
The_Kapn
Moderator Emeritus

 
The_Kapn's Avatar
 
Join Date: 04-08-04
Location: Marianna, FL
Downloads: 0
Uploads: 0
Default

Well, I "asked" and so now "I learn".
And, everything I learn helps.
Will put it to good use and make the Brothers proud!

Thanks to all.
TIM
__________________
"Flirtin' with Disaster" BBQ Team (RETIRED)
FBA and KCBS Cook and Judge.
Former owner of a WSM, a Smokey Joe, a Charbroil Commercial gasser (junk), the legendary "StudeDera", a Fast Eddy PG500, and Sherman.
Now cooking with a Yoder YS640
Proud Pellet guy cooking on real wood.
The_Kapn is offline   Reply With Quote


Unread 08-16-2004, 07:22 PM   #13
2Fat
is one Smokin' Farker
 
2Fat's Avatar
 
Join Date: 08-13-03
Location: Nekoosa, Wi
Downloads: 0
Uploads: 0
Default

not sure if I like the new system---yet! It is really hard to tell what ya are doing when the scores are all over the board.. difficult to get a handle on things when you get thre 6's and three9's for appearance on the same entry. Is it 3 old judges and 3 new judges dod the different folk reall see it that much differently??

all that said will cook again this weekend

Buzz
2fat Bikers
__________________
Buzz(2FAT)
KCBS #9269 and #4605
Blazen Gridiron
BWS Competitor--sold
BWS Pro Jr--sold
BWS party(all SS)
BDS--on loan
BGE
2Fat is offline   Reply With Quote


Unread 08-16-2004, 07:36 PM   #14
BrooklynQ
Banned
 
BrooklynQ's Avatar
 
Join Date: 07-18-04
Downloads: 0
Uploads: 0
Default

As a newbie, I think the new system makes more sense. Starting at 9 and looking for ways to knock down the score seems a lot harder than starting at 6 and moving either up or down depending on the entry.

I don't know if this helps Buzz, but when we compared scores in the judge's class people were pretty consistant. What I mean by that is, let's say there were 10 judges in the class, the instructor would ask for a show of hands about the scoreof an entry. How many of you gave it a 6 - let's say- and 5 hands would go up, 5 - one hand, 7 - two hands, 8 - one hand and maybe one hand for 9 or 4. The majority of the group seemed to always be predominately one number. So it seems like the new system SHOULD work just fine.
BrooklynQ is offline   Reply With Quote


Unread 08-16-2004, 07:53 PM   #15
2Fat
is one Smokin' Farker
 
2Fat's Avatar
 
Join Date: 08-13-03
Location: Nekoosa, Wi
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by BrooklynQ
. So it seems like the new system SHOULD work just fine.
I guessit works in the sense that the smae good cooks are still winning and there are very few(maybe one in the four meats) 180's this year.. Folks were po'd after the contest in Nebraska last year!!

Last year you never would have seen three 6' and 3 nine's given to the same box--but thats the way it is--today

Buzz
2Fat is offline   Reply With Quote


Reply

Similar Threads
Thread Thread Starter Forum Replies Last Post
"Porky", Gamey", "Disgusting" Flavor Boshizzle Q-talk 24 02-21-2010 10:56 AM
Friday Night Steaks..."Bistro Style"...(featuring "SizzleVision") JD McGee Q-talk 5 10-31-2009 02:06 PM
Food Network's "Next Big Star" winner. Another "Suzy Home Maker" persona. Bamabuzzard Q-talk 78 08-04-2009 11:42 AM

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Loading



All times are GMT -5. The time now is 04:50 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Lite) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
2003 -2012 © BBQ-Brethren Inc. All rights reserved. All Content and Flaming Pig Logo are registered and protected under U.S and International Copyright and Trademarks. Content Within this Website Is Property of BBQ Brethren Inc. Reproduction or alteration is strictly prohibited.
no new posts