New KCBS scoresheet

[The_Kapn]

I have not looked at the new KCBS scoresheet in detail, but it sounds like what FBA has provided for years.
When we competed, we found the data useful, but only in the "macro" sense.

As many have said--if we wound up on a low scoring table, we looked at our numbers as such--took it with a grain of salt.

If we got great numbers, but so did every one else--we took as a blessing.

If our table killed us, but not the others, we looked at working on our cooking.

What is interesting is to track the individual judge scores.
At one event, we got nailed with a 10-7.5-7.5 from one judge. Totally out of whack with the other 10's and 9.5's. Cost us some serious $$.

So, we looked at the other scores from this judge and they were all--100%--10-7.5-7.5 except for one entry that got a perfect 10-10-10 from all judges.
Having judged a bit, I know exactly what happened. This Dude or Dudette knew that if they gave 10's on appearance, the Reps could not fault them.
When the super entry came in, the other, shall I call them "seasoned judges", clearly showed their "awe" and "Mr 10-7.5-7.5" figured he/she needed to score up.

We retired from competing for many reasons, this was just a piece of the puzzle.

That info should be available to you KCBS folks now.

Good Luck y'all!!

TIM

[Hawg Father of Seoul]

Originally Posted by ique

Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables

Originally Posted by Smokin' Joe

This should happen!

Originally Posted by musicmanryann

My thinking as well. Once the last judge checks in the rep clicks a button and the computer assigns judges to each table virtually instantly. Again, I'm not sure if the new program is keeping track of scoring across multiple contests or not, but if it is this would seem relatively simple.

I know that this sounds like a good common sense approach, but it is a bad idea statistically speaking. I call this "inverse statistical regression". If you understand statistical regression, you can explain how I am wrong AND I'll read your response. It may also introduce bbq to the criterion problem.

The issue would be compounded if the judges knew where their average scores fell.

Some how I bet no one will heed this warning. It something has face validity, to hell with construct validity.

[boogiesnap]

i can only imagine that because the statistics are manipulated and not random they are no longer valid. and now that they are no longer valid, they are no longer useful because they were manipulated.

and once you force a smoothing of data, well, we'll probably all get sevens every category, every entry, kinda thing.

was i close????

[sdbbq1234]

Ok, so, why did KCBS institute this new system of delivering the results to the competitors? Was it for us to try and micro manage our cooking? Was it for them to track the judges?

I am only hoping it was for them to track the judges.

But then, what are they going to do about it?

Really, why provide the competitors this more in depth information?

wallace

[Slamdunkpro]

Originally Posted by sdbbq1234

Really, why provide the competitors this more in depth information?

wallace

Drive forum & FB traffic?

[G$]

Originally Posted by Vince RnQ

Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.

Actually, I am not sure I disagree, but we must be careful not to give inordinate importance to a sample size of ... one. Honestly, I am chewing on the ramifications associated with seating of judges based on historical averages, and I don't think I know what my opinion is on it yet.

[G$]

Originally Posted by Slamdunkpro

There is no value because (1) you don't know the circumstances of those tables and (2) you'll most likely never encounter that set of variables (judges at that table, other entries at that table) again. Back to the roulette example. The odds of any one number coming up are 37-1; if 18 comes up 6 times in a row what are the odds it will come up 18 the next spin? 37-1. The odds always remain the same because they reset every spin. Your contest history within the aspect of judges and tables and all variables (N) is always N=1. It resets every event.

The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.

[sdbbq1234]

Originally Posted by Vince RnQ

Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.

Good idea, but I believe it would be an administrative nightmare for the reps, at the present time as you point out. Maybe down the road after this new system has had time to settle in, and the reps programs on the laptops at the comps could/would accommodate this, I agree.

wallace

[Slamdunkpro]

Originally Posted by G$

The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.

Blackjack shoes have a known history and it's possible to predict the outcome with ever increasing accuracy as N approaches zero. the same would be true if your turn in's hit the same table of judges week after week after week. They don't.

The constant in the equation is actually the food as most of us vary our recipes very little and have gotten reasonably successful at producing a constant product from week to week. The main variable is the judge.

[kellym]

Originally Posted by Vince RnQ

So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?

That's hard to say Vince. I know the experience level sort helps a lot but it's not the perfect answer. You know how obsessive I am about my sort and I've modified it since I've seen you and it's even more granular. I did a poll of the cumulative number of contests judged for the judges at each table. The lowest table had a total of 101 contests and the highest table had 113 contests. This gave a 12% variance between the highest and lowest tables which is not a bad sort. I still had one tough table where the highest score was a 10th place chicken. I had one moderately warm table but nothing I would consider super hot. I've had contests where there were no hot or cold tables using this method.
Sorting based on historical average could be interesting. It would have to be tested as I don't believe a model can be built to test this because of the human factor but someone may prove me wrong. I don't know if I would rather see this or have some re-training for the judges that are consistently off target on their judging.
Since I don't have access to historical data I guess it doesn't matter anyway. lol
Thanks Vince.

[billygbob]

While I believe all cook teams should have members that judge at least on occasion, I have found over the years as a judge that they are the toughest/lowest scorers. Reps should account for who are mostly cook team members at contests and distribute them as they would VIPs or newbies.

[SaucyWench]

I'm wondering if there is renumbering of tables and judge seats on the printouts for anonymity, like there is with team entry boxes, so we keep the true double-blind. At comps that have tents for judging, it wouldn't be too hard for someone to at least figure out the table numbers. I'd hate to lose one of my cook friends because they figured out my table and seat number and my score didn't agree with what they thought it should be.

And I wonder how KCBS is going to re-educate outliers. So far, continuing education consists of one on-line test which I easily ace. Tracking won't do much good if they don't have a plan for what's next.

[boogiesnap]

the problem i'm having with sorting through historical data is we would essentially be removing that "i didn't like it" spike in scoring.

clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.

i think it may do a real disservice to the competitors and the sport to start sorting judges.

[sdbbq1234]

[QUOTE=boogiesnap;2554257] clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.[/ QUOTE]

+1. That goes for me.

wallace

[Andrews]

One variable that I don't think has been brought up is the celebrity judges.
To me, that is a problem, you can re-educate existing judges all you want, how do you factor in a high percentage of judges who are judging their first contest whith no historical data on how they might score entries?