Judging Question

Yet another example of things that the KCBS BoD could define a bit better. You're absolutely right that 'average' is a moving target and is going to vary from judge to judge. Is it average for BBQ in general, or average for competition-style BBQ? Since there is no official explanation for this, or any of the other scores really, the marker I've heard a couple of other judges reference is that the food you'd get in a chain BBQ joint (Jack Stack, Famous Dave's, etc.) or any other venue producing food for quantity vs. quality would be categorized as a 6. I'm not saying that's right, but it's the only guideline I've heard so far that makes some sort of sense.

My understanding of the "average" is for ALL barbecue wherever it is offered, NOT just competition entries. Therefore, MOST of the "average" entries that I see at competitions are at least a "7", as they definitely qualify as better than "average" everwhere else!

This is just one more point that needs additional clarification by KCBS, IMHO.

I have asked that very question of 3 BoD members, 1 former BoD member and at least a dozen Reps. Their answers are always the same: "It's up to each judge to determine what's average for them."

I agree that something needs to be done, but what and how?

To me, average BBQ is something that I would pay for in a restaurant and not beotch about it too much. Then again, I've judged about 60 contests in the last 5 years and have cooked with a couple of very good teams besides having the chance to sample food from some of the top-notch cooks in the country. To me "average" is way higher than a McRib or Famous Dave's although both of them are making more money than I'll ever see .
If we were to use the "average" of all competition BBQ that might lead to comparing entries and would defiantly lead to a lot of lower scores.
If we use Famous Dave as "average" that's one starting point. If we use McRibs then that's another one. If we consider all of the BBQ that we've ever eaten then things really get crazy :shocked:.
I'm not sure that there is a "for sure" answer here, but I'd really like to hear what YOU as a cook or a judge would consider "average".

I believe the score table is a little shifted toward the higher end. I agree with bbq.tom that 7 is the average in competition. I think if you get a 5 you did something wrong. i can see getting a 2 if your food is raw, but how often do you see 3's or 4's on a score sheet? 5 is low enough that if you get a few, you'll be in the bottom half. I've heard more than one judge say that they don't like giving less than a 6 because then they have to use a comment card.

Just my thoughts and an observation, not saying it's right.

I'm a judge and if your getting 6's your BBQ ain't that good. When judging my range is normally 7-9. These teams work their butts off cooking BBQ for these competitions. Even though the statement 6 is average I would consider that a low score in competition BBQ. You may find a low scoring judge but from my experience most judges a pretty liberal with their judging scores.

Rookie'48 said:

I agree that something needs to be done, but what and how?

Click to expand...

That's a really tough one. My overly-developed sense of justice knee-jerk response would be to say "Scrap the whole system, put in a new one that scores 1-10 or 1-5, and re-educate all reps, cooks, and judges on the new standard". Then I consider the fall-out from something like that...complaints from cooks, complaints from judges, the costs associated with modifying all of the front and backend software used for scoring, etc. and realize that a rebuild is really a pretty ridiculous notion. Scores do tend to be skewed towards the high end, but is that really a bad thing? We're told that if you're hovering between two numbers for a score, give the cook the benefit of the doubt and give them the higher number. That mentality is going to lend itself to having scores skewed towards the high end, and as long as that skew stays consistent it's not going to have an overall impact on team placement/rankings.

In my opinion, this is yet one more example of something that could be effectively taken care of through the CBJ continuing education initiatives but only if the BoD provides the necessary clarifications. Without the latter, the former really isn't all that useful.

In any given contest, among pro teams (not backyard) it is a game of expectations pure and simple. Is any team out there trying to hit "average" -- no. The majority will be above average (7) and the ones that really shine will merit 8's or 9's. It isn't that hard and it isn't a broken system. It's just very easy to criticize.

Scores of 6 and below can still be earned, but they'll be non-modal in the data distribution and that's why the rules allow for dropping the lowest score.

Folks who think the judging system is broken need to realize that until there is only one recipe for barbecue and a computer can be programmed to score how close a bunch of cooks can get to it, there will be (a) subjectivity in the judging process and (b) occasional and random events like the famous "judge #6" story.

Continuing education and rule/standard reinforcement for judges? Amen! It's overdue. Change the scoring system. No need. It is working.

Rookie'48 said:

I have asked that very question of 3 BoD members, 1 former BoD member and at least a dozen Reps. Their answers are always the same: "It's up to each judge to determine what's average for them."

Click to expand...

I think that may the best answer that can be given for taste. The only way to get everyone trained on what is a 9,8,7 etc would be to have some giant machine prepare the samples, ship them to CBJ classes and then give electric shock treatment to any judge that did not get it correct.

I think the second best that can be done is try to educate new judges as quickly as possible at the judging table and it seems to me that immediate feedback to all judges as to how you are comparing to fellow judges and then have discussion about. Show us the score cards before they are turned into the reps. Give us print outs of how our table scored and where I was in the mix.

That would go far in my opinion for both the high scorers and the low scores. keith

I don't really have a problem with the current system, I wish there was a little less luck involved though. I don't like the fact that entries can do better or worse by landing on different tables, but that's how it is. It's subjective and it's always going to be unless they come up with BBQ eating robots to judge.

What about a different system that completely eliminates numbers and and just has boxes to check, like this.

- This is some of the best barbecue I've ever tasted
- This is really really good barbecue.
- This barbecue is good but needs a little work
- This barbecue is better than Famous Daves
- This barbecue is better than McDonalds McRib
- One bite is all I want of this barbecue
- I wouldn't feed this my dog
- I wouldn't feed this to a BBQ Judge
- Disqualified

Just Kidding Judges:icon_smile_tongue:

On a more serious note...

It is the verbiage, not the system that needs changing because what IS "average"?

However, everyone knows what is "delicious", "satisfying", or "unappetizing".

Maybe those should be the three poles by which to judge, rather than "comparing" to other Q with terms like "average".

When I took my judging class a couple of years ago, I really liked how we were taught but was bothered or confused by one thing.

When they brought the brisket around and we judged the boxes appearance then took our samples then scored the samples, too me it was the best brisket I had eaten in a very long time so I scored it a 9. My wife who sat next to me for the class score it an 8 while people at the other end of the table score it a 6 on taste. When discussing why we scored the way we did the instructor said that the brisket was average and probably should have been scored a 7. He had asked entire room what scores they gave and only one other person besides me gave it a 9 on taste, so I had to explain why I scored it the way I did. He never told me I was wrong but that comment that the brisket was only average so should probably be a 7 has always stuck with me.

I'm a cooker, not a judge, but I took the judges class and recently judged my first.
It was a small 1st year KCBS comp so there were some rookies and the food showed this.
Having that wonderful training to go by "It's up to each judge to determine what's good BBQ for them." I found myself scoring between 7 and 8. Some 6's for some really poor (comp quality) and some 9's for appearance. But none had any outstanding taste and texture. There was one really poorly cooked entry that I did give a mercy score of 6,6,6.
So to answer Eric's question, I think I did score average at a 6, but I would not expect to see much average que at a comp.
But I did get one brisket, that appearance was a 9, that after tasting, I asked myself could it be better? I answered, self, no.
it got 9,9,9.

I have judged over 35 contests and my scores have changed with experience. I used to think that if I could duplicate the entry, then considering it was a competition the score would be a 6. Now after judging over 40 contests and cooking with several teams, a 6 is really a substandard score. I score a lot of 7's and 8's and I give my share of 9's. I judged in Garnett KS recently and there were several straight nines on my scorecard. If I score a 5, I will write a comment card. If I score a 6, it would deserve a comment card but I believe the score reflects the results....just average.

I know its just what they choose to define "6" as, but doesn't average mean by itself comparing to a group? Isn't the judging samples supposed to be judged on their own merit?

If so, they should probably change the verbage.

I haven't judged a ton of comps, but I would have to say that Davidh is probably right: the more you judge, the better your sense of what "average" really is.

I helped out at an unsanctioned backyard event last week. Since I was the only one who was a CBJ, the organizer asked me to give the other judges a quick crash course on judging standards.

I also gave them the "Six is 'average'." speach and defined it for them like this:
If I taste it and think, "Yep... that's BBQ...." I give it a six. If I taste it and think, "Hmmm, that's pretty good...." I give it a seven. If I taste it and think, "Oh yeah... now THAT is good..." it's an eight. If I've taken two or three bites before I even realize it, or think "OMG... how in the HELL did they DO that??" it's a nine.

That may be a little too harsh and as I judge more comps that may mellow out, but I'm a firm believer that the best score should be reserved for the best performer. That being said, though, I'm pretty liberal with appearance and texture scores.

There's a fairly new sanctioning body that uses this criteria for their judges. Personally, I think it holds merit and aids judges in deciding on a proper score.

1 - Disqualified Only the Head judge can authorize this score.

2 - Awful, raw or inedible

3 - Had significant faults

4 - Faulted

5 - Somewhat below average

6 - Average

7 - A bit above average

8 - Very Good

9 - Great!

10 - Outstanding!!!

Perhaps giving judges (for any sanctioning body) a guideline of what a numerical score should mean is a good start.

Southern Home Boy said:

I haven't judged a ton of comps, but I would have to say that Davidh is probably right: the more you judge, the better your sense of what "average" really is.

I helped out at an unsanctioned backyard event last week. Since I was the only one who was a CBJ, the organizer asked me to give the other judges a quick crash course on judging standards.

I also gave them the "Six is 'average'." speach and defined it for them like this:
If I taste it and think, "Yep... that's BBQ...." I give it a six. If I taste it and think, "Hmmm, that's pretty good...." I give it a seven. If I taste it and think, "Oh yeah... now THAT is good..." it's an eight. If I've taken two or three bites before I even realize it, or think "OMG... how in the HELL did they DO that??" it's a nine.

That may be a little too harsh and as I judge more comps that may mellow out, but I'm a firm believer that the best score should be reserved for the best performer. That being said, though, I'm pretty liberal with appearance and texture scores.

Click to expand...

I think that is a great way to verbalize the thought process!!! An entry really has to knock my socks off to get a 9 in taste! I give comment cards for anything 5 and below and I agree giving a 6 already implies average. As a cook and a judge the majority of the time our scores echo our own thoughts on the entry. You always get one or two scores across all entries that are significantly skewed from the average but that is why the low score is dropped. Pretty darn good and fair system in my book!