KCBS Judging Update

I'd like a definition of the word "average." Is it meant to be "your average Comp turn in," or "what you'd find at an average BBQ joint," or "the average of all BBQ that you have eaten?" Those points can vary quite a bit.... It will be interesting to see what the judging class brings this year. Thanks Sled for the update!

I like the fact that they added the definitions AND did anyway with the "starting point". As Aaron said, average is still open to interpretation but this helps put a little structure around the point values. I don't think you can teach what is average, or good, or bad; especially when it comes to taste. Its kind of like pornography - its hard to describe but you know it when you see it.

Whatever happened with the comment cards?

MoKanMeathead said:

I don't think you can teach what is average, or good, or bad; especially when it comes to taste.

Click to expand...

I agree 100%. This will still be subjective - not everyone will give an entry the same score because their version of "average" will vary from others.

My point is that I'd like to understand what they expect us to judge "average" to be. If I understand "average" to mean "the average of all BBQ I've eaten," I'd have to include things like a McRib (once, never again) and some pretty crappy 'Q that people try to pass off as the real thing. Think crock-pot and boiled ribs here. Those kinds of things bring my "average" way down. If I were to take an "average" of all 'Q I've tasted as a judge, well then, that's likely a different average to start with, isn't it? It wouldn't be pulled down with much of the crap I've been forced to try outside the circuit. Judging still comes back to what you are comparing it to in your own mind.

Anyway, I just like to be anal about definitions I guess. I want to be as clear as possible - to make judging as good as possible. Don't get me wrong, I like the changes, I just want to clarify as much as possible. Plus it gives me something to ramble about.

The Board of Directors have approved changes in the judging procedures for 2008. After, a year of testing and development, a new judging slip will be used beginning March 1st, 2008. Each number on the judging slip will have a definition associated with it. The starting point of six has been abolished. There will be no starting point. The definitions are as follows.

9 – Excellent
8 – Very Good
7 – Above Average
6 – Average
5 – Below Average
4 – Poor
3 – Bad
2 – Inedible
1 – Disqualified

I think (and hope) this is a step in the right direction. The descriptive word at least gives the judge a reference to what they should score an entry based on what they see and taste. With a 4 being described as poor and 8 as very good, etc., I expect the randomness of the scoring from judge to judge to become less of an issue. I really hope this helps with the random scores from non certified judges, it should at least keep appearance and tenderness scores uniform for the most part.

I agree the descriptives are subjective, but then so is taste.

My question would be, why wait 'till March to implement the change, Why not start in January so that all events held in 2008 will be on equal judging standards.

The Judging Slip with definitions is a program the Board began as a test program for 07. Jim Minion, Linda Mullane, Don Harwell, Mike Lake and myself used these cards at all of our contest. We found the definitions reduced inconsistent scoring from a range of 7-12 percent to a range of 5 - 7 percent.

The average GC score remained in line with other contest. The board after reviewing the data, voted to adopt the score cards with definitions for 08.

However, the board latter voted to remove the 6 as a beginning point. I urged this to be delayed for a year, in order that we implement only one change at a time. My motion was voted down, and the 6 has been removed. I am concerned this will add to inconsistent scores rather than our goal in reducing this.

The Board despite an overwhelming response in favor of the comment cards, voted to end comment cards. I cannot explain the reasons for I did not understand the debate.

The March 1 date has been used over the past years for the date to implement rule changes. There are only 3 or 4 KCBS contest prior to this date. We must change the Judges CD for the removal of 6 and any changes in the rules.

We are not perfect. No change is perfect. But the changes we are making are in an effort to improve KCBS. They have been tested and the result warranted the change.

I appreciate the comments and thought I have received from the membership. Good luck in 08.
Yours in Que
Merl Whitebook

We appreciate you being so accesible Merl.

I just can't figure why the board wouldn't vote for the comemnt cards. We use them at our practice backyard contests here in California and the cooks love them. It helps to know why a judge gave you a particular score so you can working on improving your product.

I understand there would be some hurt feelings (there always are) but I can't think of a logical erason not to give constructive feedback to the cooks. Unless there aren't enough judges out there who are able to articulate their feeling in a sentence or two. I don't think that's the case, though. I would really be interested in why the BOD thinks this is a bad idea.

i don't get it. why is the midpoint below average? isn't poor and bad kinda the same thing? with it set up this way, even if you added sublime as #10 there would still be five grades below average and only four above it?

phil

Merl said:

We found the definitions reduced inconsistent scoring from a range of 7-12 percent to a range of 5 - 7 percent.

Click to expand...

I realize that sometimes you can't get all (or even most) judges to be CBJs and that point in itself can lead to some wild scoring.
I have judged a grand total of three comps thus far, so I am no expert. I do think that the table captain could / should keep an eye open for the judge(s) whose scores are way out of line with the other ones.
I don't mean that each & every judge should mark the same way because there are differences in taste. I am talking about the one guy who gives a 4 or 5 when everyone else gives a 8 or 9. That person should be asked by the T.C. (or the Rep) why their score was way off.
I know that the low score is thrown out, but what about the next comp where there might be that guy and another one or two whose scores are out of line?
Maybe some of the judges should be judged on their judging. By the way, I don't compete on my own, but I have been a "dish-bitch" a couple of times.

The removal of the 6 starting point makes sense but I suspect it will make it a bit harder for new judges to figure out what an average entry should be.

I really liked the comment cards as a judge, we used them all of last under, and am really confused as to why they were not approved. I assume that there are some out there that were worried about too much negative feedback but one of the biggest issues I have with judgeing was that there was no feedback other than a number to the cooks.

The problem (which can not be fixed in my humble opinion) is the word average. What I consider average is going to be totaly different than what others might think is average. The Human factor can not be avoided.

I loved the idea of comments on the judging card. I also thought that they should also state if the judge was CBJ or not. However, a good friend of mine came back from a contest that used the comment cards and showed it to me. The comment was "best rib I ever tasted" However, under taste his scores were 7,6,7,7,5,7. Now if you heard that it was the best ever, wouldn't you think there would be a 9 on that card. It kind of sends a mix message, which for me as a cook would make me even madder.

I think we know going in that judging is a perception. Will I stop cooking because of it (like the many others I know who have), no. I just keep having a good time.

I want to thank the people on BOD, this discussion board, cooks, judges, contest reps, organizers and all volunteers at contest. If it was not for them, I would not have met such wonderful people.

Another reason to stop training judges. Either you know good BBQ or you don't. I like that IBCA will use different judges at most every event. The judges are provided by the sponsoring party such as KofC, VFW, Elks etc. The head judge is only there to explain the simple rules. Here is the website for those rules: http://www.ibcabbq.org/topbar/rul.html

IMHO, I think any certified judge whose scores differ radically and regularly from the other people at his table should be judged, and perhaps "re-educated".

I've seen several instances where most judges at a table will agree on which entry was best, but often one particular jusdge seems to completly disagree.

Not sure how practical it would be to ty that, but I still think the KCBS judging rules are pretty close to what they need to be. I would like to see the comment cards become a reality though. If you're going to rate someone really low, or really high, they ought to know why.
db

One definition of average implies a result obtained by comparison. This is a direct contradiction with the judging guideline that states (paraphrasing), that each entry is to be judged solely on its own merits. Perhaps, something like ordinary should replace all references to "average?" Otherwise, I think the KCBS may inadvertently be inviting a whole new level of controversy.

Being outside the decision-making process, I cannot know what they were thinking, however, this is not a good move, in my opinion.

StLouQue said:

One definition of average implies a result obtained by comparison.

Click to expand...

That's where I was going too. "Average" means that you are comparing against something else. Judges don't have to compare against the other entries at the table (as is discouraged in the rules) but in order to compare against an average, you have to understand what the population you are comparing against is made up of.

You can't just say "average" without defining the population. For test scores, it's the average within a class, or within a school, or within a state. For a any set of data, the average is calculated across the specified set of data.

So my question - If KCBS wants judges to compare each entry to some "average" that they have in their head, what is that average composed of? All BBQ they've ever tasted, or some subset of all BBQ they've ever tasted, or some imaginary ideal 'average'?" Please define 'average,' and what it should contain.

Merl said the new system was tested last year and that it worked. To me, that sounds like a good enough basis to make a change. Why not try the new system before being critical?

This new scoring definition thing is already being way over thought. If you really like it, you score it high. If its just ok or does not have that Wow factor then you judge it average or less. Whats so hard about that? If I put something in my mouth and it taste good, even if it taste different than what I am used to I would give it high marks and the reverse would be true if I did not fine it appealing.
Dave