The BBQ BRETHREN FORUMS.


Forum Portal Recipes Smoke Signals Magazine Welocme Merchandise Associations Purchase Subscription Brethren Banners
Go Back   The BBQ BRETHREN FORUMS. > Discussion Area > Competition BBQ

Notices

Competition BBQ *On Topic Only* Discussion regarding all aspects of Competition BBQ. Experiences competing or visiting, questions, getting started, Equipment, announcements of events, Results, Reviews, Planning, etc. Questions here will be responded to with competition BBQ in mind.


Reply
Thread Tools
Unread 10-08-2007, 12:16 PM   #1
jvreeland
Got Wood.
 
Join Date: 09-24-07
Location: Pelham, Alabama
Downloads: 0
Uploads: 0
Default KCBS Juding and Ways to Improve It

Quote:
DISCLAIMER: For those that read this on the KCBS Board please do not take these comments as we hate the system as many of us do not. We just want to make the KCBS the best BBQ circuit out there. So as members we would like to voice some of our concerns and provide feedback on improvements.
After reading Gordon's post over here about the issues he experienced at the American Royal - I thought I would start this thread on ways in which the KCBS can improve judging (this was originally posted over at The Smoke Ring, but I thought I would get everyone's comments here as well).

First off - I think at a bare minimum we need to start tracking the scoring of judges. This is without a doubt the biggest step we can make to working with judges to make their scores more uniform across the chart.

Second - Going one step further I think a simple review after the table has finished judging the box on why everyone scored they way they did would be great. I am sure we can come up with a simple piece of paper that has the standardized issues with boxes/meats. For example, on a brisket box a judge could not like the brisket because it was too tough, dry, bad seasoning, cut too big, etc...
If we had this simple sheet we could better understand from the cooks side why it was scored the way it was.

Third - (and this could be a radical move, so do not crucify me) nobody is a judge if they are not a CBJ. Stop bring people off the street. Is it going to result in judges having multiple rounds of boxes, yes - but at least we have certified judges judging the competitions.

I know others have some ideas on things that can be improved so please post them.
__________________
~Jeff Vreeland
Alabama GrillBillies
[URL="http://www.socialjeff.com"]Blog[/URL] | [URL="http://www.flickr.com/photos/jvreeland"]Photos[/URL] | [URL="http://www.algrillbillies.com"]Team & Catering Page[/URL]
jvreeland is offline   Reply With Quote


Unread 10-08-2007, 01:20 PM   #2
Slamdunkpro
is Blowin Smoke!

 
Slamdunkpro's Avatar
 
Join Date: 02-27-07
Location: Northern VA
Downloads: 0
Uploads: 0
Default

Let me start by stating that I have never judged a comp. I did sit through the CBJ class and I’ve been cooking for a long time so, from that perspective:

Becoming a CBJ should mean something more that “I have $40, a pulse, and 4 hours to kill.” I sat through the class and was stunned at the lack of any standard in regard to how to judge. It was like sitting in a trendy shrink’s session. Every question was answered with “but how does it taste to you?” This is not the path to consistency.

What to do? KCBS could/should:

Standardize the turn in requirements - particularly in Chicken. Skin off/skin on, it doesn’t matter, but pick one so all judges see a level playing field. The same applies to sauce - on/off/on the side - PICK ONE and mandate it. Brisket - sliced or chopped, and so on. Stop trying to make the judges into mind readers with things like “judge it as the cook intended it”. What????

Allow judges to talk about the entries with the table captain as moderator. This would go a long way in stopping the wild disparity in scoring. The judges still score individually but they can communicated the pluses/minuses of entries prior.

Give the cooks meaningful feedback - instruct judges to use terms that people can relate to. “Terrible” and “Wonderful” are descriptive terms but hardly quantifiable or helpful. On the other hand “Tough” “Dry” “Salty” “Sweet” “Tender” “Moist” and so on not only give feedback cooks can understand they can also help judges when adjudicating a score.

My 2 cents.
__________________
Aporkalypse Now Competition BBQ team
Thanks to our sponsors: Wusthof Trident USA & Creekstone Farms
Spicewine Tandem - "The Beast"

Brinkman modified upright, Weber Kettle(s); Meadow Creek Pr-24, Super Stealth Mode black & Night Vision Red Thermopens
-----------------
MABA - (Mid Atlantic BBQ Association) President
KCBS Certified Judge - #23289
Slamdunkpro is online now   Reply With Quote


Unread 10-08-2007, 01:31 PM   #3
jvreeland
Got Wood.
 
Join Date: 09-24-07
Location: Pelham, Alabama
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by Slamdunkpro View Post
Give the cooks meaningful feedback - instruct judges to use terms that people can relate to. “Terrible” and “Wonderful” are descriptive terms but hardly quantifiable or helpful. On the other hand “Tough” “Dry” “Salty” “Sweet” “Tender” “Moist” and so on not only give feedback cooks can understand they can also help judges when adjudicating a score.

My 2 cents.
Perfectly said - it helps cooks understand what was wrong with it as well as judges "recap" why they gave it it a 6 instead of 8 or 9.
__________________
~Jeff Vreeland
Alabama GrillBillies
[URL="http://www.socialjeff.com"]Blog[/URL] | [URL="http://www.flickr.com/photos/jvreeland"]Photos[/URL] | [URL="http://www.algrillbillies.com"]Team & Catering Page[/URL]
jvreeland is offline   Reply With Quote


Unread 10-08-2007, 01:32 PM   #4
ModelMaker
is One Chatty Farker
 
ModelMaker's Avatar
 
Join Date: 04-21-06
Location: Lake Ponderosa-Montezuma, IA
Downloads: 0
Uploads: 0
Default

Just speaking for myself I would love to see a printout of judges scores. You cooks get a printout and can see how everything compares. Why not judges?
I wonder all the time how my scores compared to the other 5 at my table. We talk about the turn ins but mostly in generalities.
Would other judges care to justify their scores among their table mates?
ModelMaker
__________________
Designer of a custom drum smoker
Customized Aussie gas passer
Turkey fryer
Extremely seasoned hand hammered Wok
Pit Boss-HART BURN Competition BBQ Team
KCBS certified judge


And Your not the boss of me either!!
ModelMaker is offline   Reply With Quote


Unread 10-08-2007, 01:47 PM   #5
acorette
Full Fledged Farker
 
Join Date: 07-01-07
Location: Hamlin, NY
Downloads: 0
Uploads: 0
Default

As I took the KCBS Judging class a few weeks back, I was thinking to myself "wouldn't it be great if there was some consistency in scoring that made it less subjective and more descriptive to the cook?" Since KCBS taught me to "start at a 6 and go up or down from there," how about quantifiable adders/subtractors so that you'd arrive at a score (based upon your tastes) with more feedback toward the team? Something like a list of plusses or minuses with checkboxes.... Start at 6 then subtract one for "too salty" or subtract 2 for "way too salty." Or add one for "just the right amount of heat" and add one for "just the right amount of sweet." As mentioned, things like "tastes great," and "wonderful" don't do anything. Let's start measuring the dimensions of taste and texture, and provide quantifiable reasons for going up or down the scale.

My $0.02.

Aaron
8 Ball and the 'Que Sticks
acorette is offline   Reply With Quote


Unread 10-08-2007, 02:05 PM   #6
MilitantSquatter
Moderator
 
MilitantSquatter's Avatar
 
Join Date: 09-17-05
Location: Mooresville, NC
Downloads: 0
Uploads: 0
Default

This topic has been and will continue to be debated to death. I'm sure each of us have some great ideas & views as to how it can be improved. Unfortunately, unless all the teams are willing to voice their concerns and suggestions in an organized manner to the BOD and judging committees, or run for a BOD position themselves, not much can be accomplished online.

To use an experience at the AR as indicitive of across the board problems seems unfair. What can be expected in a contest with 500 teams and many judges not familiar with the system ?

Seems like KCBS is certainly trying to make improvements. In a subjective event such as BBQ contests, this it will never be perfect or close to it. KCBS could end up spending a lot of money and resources to address certain areas that will ultimately yield little return. For not, I'm fairly satisfied with the current system. The top teams generally stand out more often than not, the lowest scores get dropped etc. Everyone thinks there BBQ is great, no one wants to believe their BBQ is average.
__________________
Lonestar 24" Vertical offset w/ Insulated Firebox & Performer Platinum
Former owner: Jambo Backyard, Klose BYC, Med. Spicewine, Pitts & Spitts, XL & (2) Med. BGE's, 22" WSM & (2) 18" WSM's, 18" & 22" Weber Kettles
MilitantSquatter is online now   Reply With Quote


Unread 10-08-2007, 02:07 PM   #7
KC_Bobby
Babbling Farker
 
KC_Bobby's Avatar
 
Join Date: 05-03-07
Location: Gardner KS
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by ModelMaker View Post
I wonder all the time how my scores compared to the other 5 at my table. We talk about the turn ins but mostly in generalities.
Ed, I agree. The time I judged we did discuss what we gave and why with the rest of the table after everyone was done with each catagory. I think that's one way we as judges can become better and get more consistant scoring from judge to judge.
__________________
Boondoggle BBQ
Country Smokers 450
Weber Smokey Mountain 18"
Proud KCBS Member & Certified Judge

KC_Bobby is offline   Reply With Quote


Unread 10-08-2007, 02:11 PM   #8
Roo-B-Q'N
is One Chatty Farker

 
Roo-B-Q'N's Avatar
 
Join Date: 08-29-06
Location: Arlington, Nebraska
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by jvreeland View Post
After reading Gordon's post over here about the issues he experienced at the American Royal - I thought I would start this thread on ways in which the KCBS can improve judging (this was originally posted over at The Smoke Ring, but I thought I would get everyone's comments here as well).

First off - I think at a bare minimum we need to start tracking the scoring of judges. This is without a doubt the biggest step we can make to working with judges to make their scores more uniform across the chart.

Second - Going one step further I think a simple review after the table has finished judging the box on why everyone scored they way they did would be great. I am sure we can come up with a simple piece of paper that has the standardized issues with boxes/meats. For example, on a brisket box a judge could not like the brisket because it was too tough, dry, bad seasoning, cut too big, etc...
If we had this simple sheet we could better understand from the cooks side why it was scored the way it was.

Third - (and this could be a radical move, so do not crucify me) nobody is a judge if they are not a CBJ. Stop bring people off the street. Is it going to result in judges having multiple rounds of boxes, yes - but at least we have certified judges judging the competitions.

I know others have some ideas on things that can be improved so please post them.
As this is the Brethren Forum I will let Gordon do his talking over on that forum and address your comments here. I am a cook, an organizer and a judge. I judged both the AR Invitational, Open sides (all three categories), open desserts, and the all of the Open meats.

If you have ever stopped and watch what the Reps and most table captains are doing while judges are stuffing food in their mouths your questions would be answered. Every time I have judged, the table captain has gone over each judges slip before handing them to the Rep. At that point every Rep I have had has gone over the slips again before handing them to be tallied. So judges are being tracked and if needed they are being called out if a Rep needs to do so. This was explained at both the invitaional and the Open. It happens.

I have not judged a comp where the judges did not go over the scoring and the reasons why they are judged. There is no way a standardized form would be able to be made let alone the time to go over things and mark them as needed. I will use your brisket example. I was talking with two guys and one said he prefferd his brisket cut thinner than the standard number 2 pencil as laid out by KCBS the other guy said he liked his much thicker than the # 2 Pencil. These two guys were not judges however they were 2 guys from the SAME TEAM cooking the open. So a standardization in the judges tent would be tough. Turn the card over and write why if you want, it is encouraged and I do it If I have to give something 5 or below.

All CBJ's would be wonderful. But often time a non CBJ is not as harsh as a CBJ so better scoring without all CBJ's. Not right, but there it is. Teams go out of their way to see how the judges tent will be stacked so they can adjust their cooking styles.

When I cook I know if my product sucks or not and when I get a 5 or 6 I know exactly why. To many cooks are to quick to look at the judges tent and not at themselves.

Now how to improve? How about reinstating the cooking regulation for the judge that is going to be given his Master Badge. I think this was a very poor decision on the part of KCBS. I actually think a judge should cook with a team LONG before 30 judging events. It is sad how clueless many judges are about what it takes and what goes on in the pit. So as a team, if you are not already a CBJ get to a class. Once there listen and ask questions. If you are team have a CBJ cook with you one weekend and show him the ropes, I guarantee he or she will be a better judge.

Sorry rant over.
__________________
________________
www.operationbbqmw.org
KCBS CBJ 9831
Large Spicewine, Backwoods Smoker (Party), WSM, Oklahoma Joe's Offset, Wood Fired Oven
Roo-B-Q'N is offline   Reply With Quote


Unread 10-08-2007, 02:12 PM   #9
pigmaker23
is one Smokin' Farker

 
pigmaker23's Avatar
 
Join Date: 08-17-06
Location: Albany, NY
Downloads: 0
Uploads: 0
Default

well said Vinny, this topic has been beat to death over the years


Quote:
Originally Posted by MilitantSquatter View Post
This topic can certainly be debated to death and I'm sure each of us have some great ideas & views as to how it can be improved. Unfortunately, unless all the teams are willing to voice their concerns and suggestions in an organized manner to the BOD and judging committees, not much can be accomplished.

Seems like KCBS is certainly trying to make improvements. In a subjective event such as this it will never be perfect. KCBS could end up spending a lot of money and resources to address certain areas that will ultimately yield little return. For not, I'm fairly satisfied with the current system. The top teams generally stand out more often than not, the lowest scores get dropped etc.
__________________
Eric Johnson / "The Sultan of Smoke"
Mr. Bobo's Traveling BBQ Allstars !
2009 Jack Daniels World BBQ BrisketChampion
_______________________________________
Backwoods Fatboy (2) & a Chubby
Jedmaster " BabyJed" Ring of Fire
OK Joe 84" Offset
Meadowcreek PR -72 Trailer
pigmaker23 is offline   Reply With Quote


Unread 10-08-2007, 03:17 PM   #10
smoke-n-my-i's
is One Chatty Farker
 
smoke-n-my-i's's Avatar
 
Join Date: 08-27-07
Location: Hillsborough, NC
Downloads: 0
Uploads: 0
Default

I think the reason there is so much controversy is the way some of the scores we get. For example, on one of my scores, I had 4,4,6,8,8, and a 9. So why such a difference in the scores for the same piece of meat???? Why two 4's with two 8's and a 9?????? That tells me that something is astray....

I wonder where the judges got there training, and what were they expecting. If there is such a difference in the score on the same piece of meat, tell me what is going on....

There is no standard in my opinion when there is such a wide spread on the points.

It does need some help, especially when when the scores are like that, and no explanation.

Bill
__________________
KCBS Member
Smoke-N-My-I's Competition BBQ Team
2 Chargriller SuperPros with SFB's(retired)
2 UGLY drum smokers
Weber 22 1/2" & 18 1/2" Kettle
Super fast RED Thermapen
Screaming fast ORANGE Thermapen
KCBS Certified Table Captain
KCBS Certified Judge
smoke-n-my-i's is offline   Reply With Quote


Unread 10-08-2007, 03:57 PM   #11
Lakeside Smoker
is one Smokin' Farker
 
Lakeside Smoker's Avatar
 
Join Date: 01-17-07
Location: Methuen, MA
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by smoke-n-my-i's View Post
I think the reason there is so much controversy is the way some of the scores we get. For example, on one of my scores, I had 4,4,6,8,8, and a 9. So why such a difference in the scores for the same piece of meat???? Why two 4's with two 8's and a 9?????? That tells me that something is astray.... Bill
You can never tell someone what they like or dislike. People have their own tastes...that can never change.
I judged a comp. a few years ago and one rib had (IMHO) way too much cumin. Its all I could taste for the rest of the rib entry's. When rib judging was done we were all talking about the ribs and one guy said 'that was the best rib I ever had!" He was saying that about the rib I hated! How can that ever change? I think as cooks we need to know where the lines are. It took me just over a year to find out whats 'too hot'

Mike
Lakeside Smokers
NEBS
KCBS
CBJ
__________________
Blog

In polite society, we have an unspoken agreement to call our obsessions hobbies. - Stephen King
Lakeside Smoker is offline   Reply With Quote


Unread 10-08-2007, 04:13 PM   #12
slat
is Blowin Smoke!
 
slat's Avatar
 
Join Date: 10-10-03
Location: Marble Hill, Missouri
Downloads: 0
Uploads: 0
Default

I have noticed the differences in scoring this year as both a judge and team. I think that a lot of judges don't understand that what they are judging isn't supposed to be like commercial BBQ. I believe that some think that it should be and score accordingly when it's not.
I also think that maybe before a judge can judge more than 2 or 3 contests that they should have to cook with a team. That way they understand what goes on on the other side.
What about having a mentor or two at the judging to help new judges in scoring. This would be helpful where a contest is a first time contest or where there was a recent judging class. I know in my hometown they had a judging class that had over 100 people attend and they were given first choice to judge this contest. That is just too many new judges. The scores were crazy.
I think for the most part judges do a good job, but I think that a little support after the class wouldn't hurt.
__________________
Cell 573-979-2426

Proud member of the MOFO chapter.

Bandera
Turkey Fryer
Rusting and corroding Gas Grill
slat is offline   Reply With Quote


Unread 10-08-2007, 04:24 PM   #13
The Pickled Pig
is One Chatty Farker
 
The Pickled Pig's Avatar
 
Join Date: 06-21-07
Location: Overland Park, KS
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by smoke-n-my-i's View Post
I think the reason there is so much controversy is the way some of the scores we get. For example, on one of my scores, I had 4,4,6,8,8, and a 9. So why such a difference in the scores for the same piece of meat???? Why two 4's with two 8's and a 9?????? That tells me that something is astray....
Bill
The KCBS scoring is amazingly consistent from a big picture perspective. For the most part, the same teams generally win and the same teams generally do poorly. If the scoring model didn't work, we wouldn't have teams that get and stay hot.

If your entries get inconsistent scores in more than 1 contest it's time for a change. I'd recommend a change after a single poor showing. The top teams don't get 4s or 6s on any entry at any contest so something has to change if you're trying to win. Somehow you'll have to find out how to broaden the scope of the audience that likes your BBQ. To help determine what needs to change, post a pic of your entry along with your recipe and preparation techniques on this site and I'll bet many Bretheren will give you pointers without needing to taste your food.
__________________
Paul Ostrom
The Pickled Pig

Twitter - @ThePickledPig

CBJ, WSM, UDS Cookers, Weber Gasser, Weber One Touch Gold Kettle,
The Pickled Pig is offline   Reply With Quote


Unread 10-08-2007, 04:26 PM   #14
G$
is One Chatty Farker
 
G$'s Avatar
 
Join Date: 01-01-05
Location: Southern Arizona
Downloads: 0
Uploads: 0
Default

Quote:
Originally Posted by Slamdunkpro View Post
....

Standardize the turn in requirements - particularly in Chicken. Skin off/skin on, it doesn’t matter, but pick one so all judges see a level playing field. The same applies to sauce - on/off/on the side - PICK ONE and mandate it. Brisket - sliced or chopped, and so on. Stop trying to make the judges into mind readers with things like “judge it as the cook intended it”. What????
.....
IMO ...

Requiring everyone to turn in the same cut is not leveling the playing field, nor is requiring (or not) sauce, etc. We need to maintain some individuality and, dare I say "soul" in what we cook and present. It is a cooking contest not a judging contest, so let the cooks introcuce the variables and the judges interpret the results.

As to the "Judge it as the cook intended it" comment, I agree, I have never liked that statement. I wish it was not repeated so oftent in those terms.

Some good points, this thread could get long!
G$ is offline   Reply With Quote


Unread 10-08-2007, 04:34 PM   #15
lwest99
On the road to being a farker
 
Join Date: 06-11-07
Location: Lenexa, KS
Downloads: 0
Uploads: 0
Default

G$ comments got me to thinking. I do not think this would happen but what about having the cook present their box to the judges? It would no longer be a blind judging but the cook could get the correct feedback from the judges. I agree with Paul that the same teams win all the time so something must be on track but I would like better feedback. What would you think if you had to present your food to the judges at the judging tent?
__________________
Lee West
[B][URL="http://www.thepickledpig.com"]The Pickled Pig[/URL][/B]
Lenexa, KS
:icon_cool
lwest99 is offline   Reply With Quote


Reply

Similar Threads
Thread Thread Starter Forum Replies Last Post
Ways to improve consistency ZILLA Competition BBQ 17 07-12-2010 11:59 PM
Juding Class In Norwalk CT gb7488 Competition BBQ 0 03-20-2009 08:50 PM
Trying to improve my brisket, your help please. AlabamaGrillBillies Q-talk 37 03-15-2009 07:31 PM
Hermann Juding School jbrink01 Competition BBQ 1 08-24-2007 09:43 AM

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Loading



All times are GMT -5. The time now is 05:01 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Lite) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
2003 -2012 © BBQ-Brethren Inc. All rights reserved. All Content and Flaming Pig Logo are registered and protected under U.S and International Copyright and Trademarks. Content Within this Website Is Property of BBQ Brethren Inc. Reproduction or alteration is strictly prohibited.
no new posts