New KCBS Scoring Program Test

[Sauced!]

I saw this posted by Pellet Envy on Facebook and didnt see a thread on here so I figured I would share. I like the idea. I know FBA does this and I think KCBS would benefit from it.

Here is the comment Rod posted along with a couple of pics of the scoresheets

"At this past weekend's KCBS sanctioned contest in Quapaw, Oklahoma, the new scoring system was unveiled for beta testing on an actual contest and teams received print-outs from the new program along side the traditional score sheets.

Some interesting data at first glance. Table numbers of where your food was judged are shown for all teams and all categories, so you can see the teams on your table. Scores by judge for each category are listed, along with each of those judges average score, so you can see how the score you received compares to the other scores that judge gave on that day. With some simple calculations, you can also prove the "good table, bad table" theory. In this case, 23 of the top 40 places were awarded by 3 tables, while only 6 of the top forty places were awarded by 4 tables. Translation: If you his tables 1 or 10 this weekend, odds were strongly against you winning this contest."

Overall
http://i271.photobucket.com/albums/j...55402537_n.jpg

Team
http://i271.photobucket.com/albums/j...62904238_n.jpg

Chicken
http://i271.photobucket.com/albums/j...83624169_n.jpg

Ribs
http://i271.photobucket.com/albums/j...624169_n-1.jpg

Pork
http://i271.photobucket.com/albums/j...94621823_n.jpg

Brisket
http://i271.photobucket.com/albums/j...77830575_n.jpg

[Sledneck]

About friggin time

[pat]

Looks like a great improvement...if it happens.

[Fatback Joe]

I like it.

That (and no garnish ) was one thing I really liked about FBA over KCBS. Nice to know if you won your table.

[Sauced!]

Funny part is a BOD member posted a comment saying the BOD had no knowledge of this test being done and that they had not even seen the program yet.....Once again another example of the dysfunction of the BOD.

[Teamfour]

Preface: I don't compete yet so take my comments with that in mind.

All this does is provide data that throws a now proven variable into your cook. While you may have suspected poor judging did you in, now you know. So, do you adjust your cook profile for next comp, or leave it the same since the judging was most likely suspect?

I hope KCBS would use the data to identify judges who may need further training. Personally, I think some teams may use the data to hunt down the judges and "retrain" them personally.

[carlyle]

At first glance, this looks like a good improvement.

Rather than bash the BOD, give them credit for sticking to this project even after it became more difficult that it first appeared.

As an organizer and judge chair, I am really excited about getting this new detailed information to evaluate the job my judges are doing. I can see using the data along with assigned seating in advance of the following years contest as an aid in deploying the returning judging resources evenly to avoid "good table/ bad table".

As a judge, getting feedback with data, will help evaluate my performance relative to the rest of the table more concretely than by just talking between categories.

Anxious for the program to be rolled out.

[carlyle]

Teamfour,

I hope the program will be blinded to preserve judge anonymity the same way

renumbering turn in boxes keeps judges from knowing what team they are judging.

Retaliation and confrontation would be bad for bbq and for KCBS.

[SCSmoke]

I for one really like it. I hope it gets implemented.

[Teamfour]

Originally Posted by carlyle

Teamfour,

I hope the program will be blinded to preserve judge anonymity the same way

renumbering turn in boxes keeps judges from knowing what team they are judging.

Retaliation and confrontation would be bad for bbq and for KCBS.

That part of my post was sincerely tongue-in-cheek. Honestly, I love data and hope this implemented. My point is teams will have to make even more educated decisions on what changes to make based on scores.

[tmcmaster]

Originally Posted by Sauced!

Funny part is a BOD member posted a comment saying the BOD had no knowledge of this test being done and that they had not even seen the program yet.....Once again another example of the dysfunction of the BOD.

The reason (well, ONE of) I am no longer a member of KCBS.

Originally Posted by carlyle

Retaliation and confrontation would be bad for bbq and for KCBS.

We all may joke about that sort of thing, but I really can NOT see anyone that I have EVER met in BBQ actually doing anything like that.

[Big Mike]

I like the Team Detail Report. I like the fact that I can see each judges avg score and who was at your table. The only thing that confuses me is the "Rank" column. What is that supposed to show. It is obviously not his rank at that table. Anybody have an idea

[Bourbon Barrel BBQ]

As much as I love the feedback I see this as creating a serious amount of complaints to reps and organizers. Its like eating at a real dive of a restaurant. Sometimes you just don't want to see whats going on behind the scenes. Im hoping for the best though.

[Lake Dogs]

Originally Posted by Bourbon Barrel BBQ

As much as I love the feedback I see this as creating a serious amount of complaints to reps and organizers. Its like eating at a real dive of a restaurant. Sometimes you just don't want to see whats going on behind the scenes. Im hoping for the best though.

Many of us with years and years of experience in cookoffs (not just BBQ) have known this and some of us have preached and tried to inform teams of this. There is a luck factor involved. Even if you train and train and get very well "educated" judges, group a few together with the same tendencies and same biases (we all have them; we're human) and the same types of preferences, and what you'll get are "good" tables and "bad" tables. It happens. I'm sure there's some scientific law here that applies.

For this reason, MANY sanctioning bodies force tables to change judges (mix up the mix) every category. You'll still end up with good and bad tables, but they've shifted and mixed.

There's still the luck of who else's BBQ landed on your table. Trust me, even if you judge each on their own merits, a judge, after tasting 3 or 4 fairly average or below average entries gets one good one, watch that score. It'll go WAY up. Deserved that high? Possibly; possibly not. Same on the other side, if yours is 4th in their tasting and they've had 3 in front of you that were off-the-chart awesome and yours is then very good too, the odds of getting that perfect score are fairly low.

It happens. We're dealing with humans.

For these reasons, many/most seasoned competitors will counsel new competitors to not read too much into any 1 competition scores, and dont change recipes, etc. until you've had 3 or 4 or 5 competitions saying the same thing. THEN and only then should you consider making changes.

Kudos to KCBS for showing this (like many others have done for a while).

[TooSaucedToPork]

Love it. It solves the age old problem of guessing what went wrong. It let's us zero in on and focus on improving our product. By seeing who was against you, you can tell what judges like and don't like. That is...If this actually makes it out of beta without politics breaking it down into a useless program.