PDA

View Full Version : KCBS Judging Update


Sledneck
12-26-2007, 11:50 AM
http://www.kcbs.us/news.php?id=27

acorette
12-26-2007, 01:15 PM
I'd like a definition of the word "average." Is it meant to be "your average Comp turn in," or "what you'd find at an average BBQ joint," or "the average of all BBQ that you have eaten?" Those points can vary quite a bit.... It will be interesting to see what the judging class brings this year. Thanks Sled for the update!

MoKanMeathead
12-26-2007, 01:37 PM
I like the fact that they added the definitions AND did anyway with the "starting point". As Aaron said, average is still open to interpretation but this helps put a little structure around the point values. I don't think you can teach what is average, or good, or bad; especially when it comes to taste. Its kind of like pornography - its hard to describe but you know it when you see it.

Bigmista
12-26-2007, 01:44 PM
Whatever happened with the comment cards?

acorette
12-26-2007, 02:47 PM
I don't think you can teach what is average, or good, or bad; especially when it comes to taste.

I agree 100%. This will still be subjective - not everyone will give an entry the same score because their version of "average" will vary from others.

My point is that I'd like to understand what they expect us to judge "average" to be. If I understand "average" to mean "the average of all BBQ I've eaten," I'd have to include things like a McRib (once, never again) and some pretty crappy 'Q that people try to pass off as the real thing. Think crock-pot and boiled ribs here. Those kinds of things bring my "average" way down. If I were to take an "average" of all 'Q I've tasted as a judge, well then, that's likely a different average to start with, isn't it? It wouldn't be pulled down with much of the crap I've been forced to try outside the circuit. Judging still comes back to what you are comparing it to in your own mind.

Anyway, I just like to be anal about definitions I guess. I want to be as clear as possible - to make judging as good as possible. Don't get me wrong, I like the changes, I just want to clarify as much as possible. Plus it gives me something to ramble about.

KC_Bobby
12-26-2007, 03:17 PM
The Board of Directors have approved changes in the judging procedures for 2008. After, a year of testing and development, a new judging slip will be used beginning March 1st, 2008. Each number on the judging slip will have a definition associated with it. The starting point of six has been abolished. There will be no starting point. The definitions are as follows.

9 – Excellent
8 – Very Good
7 – Above Average
6 – Average
5 – Below Average
4 – Poor
3 – Bad
2 – Inedible
1 – Disqualified

I think (and hope) this is a step in the right direction. The descriptive word at least gives the judge a reference to what they should score an entry based on what they see and taste. With a 4 being described as poor and 8 as very good, etc., I expect the randomness of the scoring from judge to judge to become less of an issue. I really hope this helps with the random scores from non certified judges, it should at least keep appearance and tenderness scores uniform for the most part.

I agree the descriptives are subjective, but then so is taste.

Pig Headed
12-26-2007, 04:37 PM
My question would be, why wait 'till March to implement the change, Why not start in January so that all events held in 2008 will be on equal judging standards.

Merl
12-26-2007, 07:14 PM
The Judging Slip with definitions is a program the Board began as a test program for 07. Jim Minion, Linda Mullane, Don Harwell, Mike Lake and myself used these cards at all of our contest. We found the definitions reduced inconsistent scoring from a range of 7-12 percent to a range of 5 - 7 percent.

The average GC score remained in line with other contest. The board after reviewing the data, voted to adopt the score cards with definitions for 08.

However, the board latter voted to remove the 6 as a beginning point. I urged this to be delayed for a year, in order that we implement only one change at a time. My motion was voted down, and the 6 has been removed. I am concerned this will add to inconsistent scores rather than our goal in reducing this.

The Board despite an overwhelming response in favor of the comment cards, voted to end comment cards. I cannot explain the reasons for I did not understand the debate.

The March 1 date has been used over the past years for the date to implement rule changes. There are only 3 or 4 KCBS contest prior to this date. We must change the Judges CD for the removal of 6 and any changes in the rules.

We are not perfect. No change is perfect. But the changes we are making are in an effort to improve KCBS. They have been tested and the result warranted the change.

I appreciate the comments and thought I have received from the membership. Good luck in 08.
Yours in Que
Merl Whitebook

Bigmista
12-26-2007, 07:25 PM
We appreciate you being so accesible Merl.

Bigmista
12-26-2007, 07:30 PM
I just can't figure why the board wouldn't vote for the comemnt cards. We use them at our practice backyard contests here in California and the cooks love them. It helps to know why a judge gave you a particular score so you can working on improving your product.

I understand there would be some hurt feelings (there always are) but I can't think of a logical erason not to give constructive feedback to the cooks. Unless there aren't enough judges out there who are able to articulate their feeling in a sentence or two. I don't think that's the case, though. I would really be interested in why the BOD thinks this is a bad idea.

rookiedad
12-26-2007, 10:36 PM
i don't get it. why is the midpoint below average? isn't poor and bad kinda the same thing? with it set up this way, even if you added sublime as #10 there would still be five grades below average and only four above it?

phil

Rookie'48
12-26-2007, 11:45 PM
We found the definitions reduced inconsistent scoring from a range of 7-12 percent to a range of 5 - 7 percent.

I realize that sometimes you can't get all (or even most) judges to be CBJs and that point in itself can lead to some wild scoring.
I have judged a grand total of three comps thus far, so I am no expert. I do think that the table captain could / should keep an eye open for the judge(s) whose scores are way out of line with the other ones.
I don't mean that each & every judge should mark the same way because there are differences in taste. I am talking about the one guy who gives a 4 or 5 when everyone else gives a 8 or 9. That person should be asked by the T.C. (or the Rep) why their score was way off.
I know that the low score is thrown out, but what about the next comp where there might be that guy and another one or two whose scores are out of line?
Maybe some of the judges should be judged on their judging. By the way, I don't compete on my own, but I have been a "dish-bitch" a couple of times.

Leeper
12-26-2007, 11:53 PM
The removal of the 6 starting point makes sense but I suspect it will make it a bit harder for new judges to figure out what an average entry should be.

I really liked the comment cards as a judge, we used them all of last under, and am really confused as to why they were not approved. I assume that there are some out there that were worried about too much negative feedback but one of the biggest issues I have with judgeing was that there was no feedback other than a number to the cooks.

paydabill
12-27-2007, 11:59 AM
The problem (which can not be fixed in my humble opinion) is the word average. What I consider average is going to be totaly different than what others might think is average. The Human factor can not be avoided.

I loved the idea of comments on the judging card. I also thought that they should also state if the judge was CBJ or not. However, a good friend of mine came back from a contest that used the comment cards and showed it to me. The comment was "best rib I ever tasted" However, under taste his scores were 7,6,7,7,5,7. Now if you heard that it was the best ever, wouldn't you think there would be a 9 on that card. It kind of sends a mix message, which for me as a cook would make me even madder.

I think we know going in that judging is a perception. Will I stop cooking because of it (like the many others I know who have), no. I just keep having a good time.

I want to thank the people on BOD, this discussion board, cooks, judges, contest reps, organizers and all volunteers at contest. If it was not for them, I would not have met such wonderful people.

tony76248
12-27-2007, 01:55 PM
Another reason to stop training judges. Either you know good BBQ or you don't. I like that IBCA will use different judges at most every event. The judges are provided by the sponsoring party such as KofC, VFW, Elks etc. The head judge is only there to explain the simple rules. Here is the website for those rules: http://www.ibcabbq.org/topbar/rul.html

Stoke&Smoke
12-27-2007, 05:27 PM
IMHO, I think any certified judge whose scores differ radically and regularly from the other people at his table should be judged, and perhaps "re-educated".

I've seen several instances where most judges at a table will agree on which entry was best, but often one particular jusdge seems to completly disagree.

Not sure how practical it would be to ty that, but I still think the KCBS judging rules are pretty close to what they need to be. I would like to see the comment cards become a reality though. If you're going to rate someone really low, or really high, they ought to know why.
db

StLouQue
12-27-2007, 06:26 PM
One definition of average implies a result obtained by comparison. This is a direct contradiction with the judging guideline that states (paraphrasing), that each entry is to be judged solely on its own merits. Perhaps, something like ordinary should replace all references to "average?" Otherwise, I think the KCBS may inadvertently be inviting a whole new level of controversy.

Being outside the decision-making process, I cannot know what they were thinking, however, this is not a good move, in my opinion.

acorette
12-28-2007, 07:56 AM
One definition of average implies a result obtained by comparison.

That's where I was going too. "Average" means that you are comparing against something else. Judges don't have to compare against the other entries at the table (as is discouraged in the rules) but in order to compare against an average, you have to understand what the population you are comparing against is made up of.

You can't just say "average" without defining the population. For test scores, it's the average within a class, or within a school, or within a state. For a any set of data, the average is calculated across the specified set of data.

So my question - If KCBS wants judges to compare each entry to some "average" that they have in their head, what is that average composed of? All BBQ they've ever tasted, or some subset of all BBQ they've ever tasted, or some imaginary ideal 'average'?" Please define 'average,' and what it should contain.

The Pickled Pig
12-28-2007, 10:52 AM
Merl said the new system was tested last year and that it worked. To me, that sounds like a good enough basis to make a change. Why not try the new system before being critical?

Mo-Dave
12-28-2007, 11:24 AM
This new scoring definition thing is already being way over thought. If you really like it, you score it high. If its just ok or does not have that Wow factor then you judge it average or less. Whats so hard about that? If I put something in my mouth and it taste good, even if it taste different than what I am used to I would give it high marks and the reverse would be true if I did not fine it appealing.
Dave

DawgPhan
12-28-2007, 11:27 AM
Not to be critical of the new judging sheet, but it seems like they could ease everyone's mind by providing some of the research or who evaluated the results from the test. That score sheet is basically the heart of the whole kcbs competition. Making changes to it shouldnt be done because it seemed to work or because it looked good. It seems like the whole sheet should have been created by a professional company who creates survey sheets and then it should have been tested by people who test these types of products.

Also the testing for it should have taken place somewhere besides a contests where you cant really control all of the variables and you are testing something on someone elses time. Certainly a test was going to have to be made in a real situation, but that should have been after countless tests under controled situations. You shouldnt be getting answers when you test it at comps but just confirming what you already knew.

While the intentions were honest the impression that I got was that KCBS decided to change the scoring sheet and the only support for that was that it seemed to work better when we tried it a couple of times...and that isnt and shouldnt cut it with the competition folks...

Pig Headed
12-28-2007, 11:51 AM
I personally think that the old scoring way worked fine. I did like the idea of comment sheets and wished the BOD had ok'd them. I think the reason for the change was the overwhelming complaints about the inconsistant judging. While I'm pretty new at competitions, I think that the consistancy of the top teams, bears witness that the old system worked.

Sir Smoke A Lot
12-28-2007, 12:37 PM
I would have liked to see comment cards approved. I did not get any at the few contests I did last year, but I think it would have been useful. I don't really have an opinion on the new scoresheets. I guess having a little more "definition" applied to each number may be helpful. Time will tell, I guess!

eurycea
12-28-2007, 02:26 PM
You can't just say "average" without defining the population. For test scores, it's the average within a class, or within a school, or within a state. For a any set of data, the average is calculated across the specified set of data.




100% correct. And poor means lacking money - how are judges suppose to know if the entry lacks money??? And bad is a song by Michael Jackson - what are the odds of that showing up in one of the boxes.

Do you think it is possible that the board doesn't mean average in the mathmatical sense in the same way they don't mean poor in the lacking money sense?

StLouQue
12-28-2007, 04:30 PM
I agree with you agreeing with me. However,

You can't just say "average" without defining the population. For test scores, it's the average within a class, or within a school, or within a state. For a any set of data, the average is calculated across the specified set of data.

I understand where you're going... But as a judge, what's wrong with assigning my own criteria for appearance, taste, and flavor and you, yours? That which I deem average (or ordinary) is, at least on my scoresheet.

So my question - If KCBS wants judges to compare each entry to some "average" that they have in their head, what is that average composed of? All BBQ they've ever tasted, or some subset of all BBQ they've ever tasted, or some imaginary ideal 'average'?"

I think what you are describing is more of a "standard."

Then again, I may be over-thinking.