New KCBS Scoring Program Test

[stl-rich]

At well run comps judges are not allowed to just sit as they please - good reps try to balance the master judges, the experienced, the semi-experienced and the noobs.

Good reps do a good job

[Grizmt]

Originally Posted by stl-rich

At well run comps judges are not allowed to just sit as they please - good reps try to balance the master judges, the experienced, the semi-experienced and the noobs.

Good reps do a good job

That's been my experience at almost every comp I've been at. Hey, that's it! Take the judges newly discerned rankings and mix 'em up at each table, tough,easy and undecided.

[Funtimebbq]

That was suggested many pages ago but will only work after judges have had several contests for the system to calculate their scoring pattern. Unfortunately, by the time that happens, all judges will be scoring so high, it will not matter. Once judges see themselves being sent to re-education camps, those who are on the bubble will get the message and start scoring higher, defeating the system.

[smoke-n-my-i's]

Originally Posted by Jorge

I think his point was that many teams seem to have bought into that theory, not that it was accurate.

Originally Posted by pahutchens

Yeah that's what I thought I said too

sorry abt that one... I totally read it too fast and missed the point.... I seem to do that some times, and regret it later....

[CBQ]

Originally Posted by Carnivorous Endeavors BBQ

I wonder how this would work out if all certified judges scores were kept and averaged over a period of time or several contests. Judges could then be assigned to tables so that all tables have near the same cumulative average for scoring. If all tables had a close to the same average in scoring, it would seemingly(on the surface) start a contest with a more level playing field. This way you shouldn't get 1 table who likes to score all 9's for practically everything. Just a thought....

That might be an interesting idea. Talking to reps, they sometimes KNOW there are certain judges that always score low or score high. Some reps have told me they dread certain judges showing up at a contest because they know a lot of 5s and 6s are going to be handed out for some good bbq. The reps, however, don't have any power to DO anything about that. The above suggestion might have to mitigate some of the risk - but I wouldn't try to group like judges together.

Winning the table...when it's a low scoring table...doesn't help you. Spreading out low scoring judges evenly across all the tables would be more fair to the teams.

[Grizmt]

Originally Posted by CBQ

That might be an interesting idea. Talking to reps, they sometimes KNOW there are certain judges that always score low or score high. Some reps have told me they dread certain judges showing up at a contest because they know a lot of 5s and 6s are going to be handed out for some good bbq. The reps, however, don't have any power to DO anything about that. The above suggestion might have to mitigate some of the risk - but I wouldn't try to group like judges together.

Winning the table...when it's a low scoring table...doesn't help you. Spreading out low scoring judges evenly across all the tables would be more fair to the teams.

This brings up some food for thought. Why is it that someone scoring 5-6 is considered "low scoring"? Doesn't the rating scale go from 2-9 or has this truly turned into 6-9 scoring? Is there now an attitude like in T-Ball where there's a trophy for just being there?
The reps said "they know a lot of 5-6's are going to be handed out for some good BBQ", in whose opinion (theirs?) and why is their opinion any more valid that the person judging it?
Then you have the question of who's palate is correct, could the people who're "low scoring" actually have a better palate than the "high scoring"?
Would y'all prefer false high scoring? If so then maybe the scoring and judges instructions should be changed to reflect that.

I hope you don't take this as me jumping on you or what you posted, it's not meant that way, I think you bring up some good points for discussion.

I keep hearing how this judge or that judge scored low so the judge "must" be the issue, why isn't the same said for judges that score crazy high when the rest of the table doesn't? I've never,ever seen a judge taken aside for scoring too high, just too low.
A good example of this whole argument is Hells Kitchen. You have all the chefs make a dish and invariably each one thinks it tastes fantastic. When it's tasted by Gordon or a selected group of celebrity chefs they have a way different opinion. Same thing on Chopped,Cupcake wars,wine tasting etc.

Until such a time where an absolute set of iron clad taste criteria can be laid out so every person tastes the same way/thing AND the judges are all instructed the absolute same way then you're going to have personal taste variations/rule interpretations that are not explainable nor changeable.

I guess one thing could be done which is having to pass a palate test before you're allowed to be a judge.

[smoke-n-my-i's]

Here is a section from my TC pamphlet that was given to us at our class. Dated 4-14-2011

And as a cook, I have seen this posted in areas used by judges before, so it is not top secret.

Judges are told the scoring system is from 9 to 2.
9 is excellent
8 very good
7 above average
6 average
5 below average
4 poor
3 bad
2 inedible

So, there seems to be some judges that think that most meat coming across their score sheet is average or below. So, what do they deem would be a 7, 8, or even a 9? That is a good question.

Is the judges given a different scoring sequence in their classes? I would hope not.

[New Pal Frank]

In the CBJ class my wife and I took 2 1/2 years ago, we were told that (average) 6, was what every backyard cook (meaning home cook) would produce. Not what the average team would produce. Therefore, most of the Q that has come accross my tables in the 25 contests we have judged has been 7 or higher.
Also if i score below a 7, I will give a coment card so the team isn't left scratching their head as I have since we started competing this year.
As a couple of Reps have told us in the judges meeting before hand, ther is NO limit to how many 9's you can give out in a contest. If everything presented to you is top shelf product, it is a 9 and I agree. We are judging on criteria, not comparison.

[Grizmt]

Originally Posted by New Pal Frank

In the CBJ class my wife and I took 2 1/2 years ago, we were told that (average) 6, was what every backyard cook (meaning home cook) would produce. Not what the average team would produce. Therefore, most of the Q that has come accross my tables in the 25 contests we have judged has been 7 or higher.
Also if i score below a 7, I will give a coment card so the team isn't left scratching their head as I have since we started competing this year.
As a couple of Reps have told us in the judges meeting before hand, ther is NO limit to how many 9's you can give out in a contest. If everything presented to you is top shelf product, it is a 9 and I agree. We are judging on criteria, not comparison.

I have been instructed on several occasions that 5 is the average, other times it was said to be 6.
Where I kind of take issue is the idea that just because a "team" cooked it that it would automatically start at any number. It's either good or not according to my palate and is scored that way obviously in degrees dependent on how good or how bad it is. If some meat comes across my plate that tastes real heavy of "jet fuel" or "creosote" (over smoked) then the likelihood of it getting a 6 or 7 out of me is slim to none. With that being said do I still score it a 5 or 6 to not insult someone?
Just to be clear, I don't have any problem handing out 9's to everyone who's food I'm judging IF the product deserves it and conversely, no judge should be discouraged or afraid to hand out 3's if so deserved but unfortunately that's the current culture in some places.

[gettinbasted]

The extra information was very useful. Unfortunately my pork was judged at table 10.... we won the most difficult table at least!

Without the new scoring I might be tinkering with my pork technique, but looking at the extra info I will probably stay the course.

[Ringburner]

Sounds like judging is almost as complex as cooking!

Sent from my GT-I9100 using Tapatalk 2

[THE Meatman]

looks nice and clean...easy to understand

[McEvoy AZ]

I may have missed this earlier in the thread, as I have not read the whole thing. It seems to me that what they are doing could be achieved by making sure there is a short commit on every score. Might even come up with a coding system that would expedite the scoring, as in the number 1 means to salty. With this system you would know why a judge scored you the way they did.
I was at a contest that another sanctioning body put on where most judges commented and I found I agreed with some of the judges and some of the judges contidicted other judges. This way you could look at the comment and decide for yourself if the judge was on point or not.
I think there is always going to be luck in this as we are facing human judges and thank god that is the case or the same teams would always win and what would be the point of showing up if you are not one of the winning teams.

[bover]

Originally Posted by Funtimebbq

That was suggested many pages ago but will only work after judges have had several contests for the system to calculate their scoring pattern. Unfortunately, by the time that happens, all judges will be scoring so high, it will not matter. Once judges see themselves being sent to re-education camps, those who are on the bubble will get the message and start scoring higher, defeating the system.

It stands to reason that judges that overscore would be handled in the same manner as those that underscore. Anyone consistently straying from the norm should be subjected to re-education.

[bover]

I'm also in the camp of not including the alt number on the overall report. What purpose does it serve? To me it just seems like an opportunity for collusion at future events assuming the same renumbering methodology is used year after year.