New KCBS scoresheet

Balls Casten said:
Come on now ... would you really have "come away wondering what went wrong" "started to mess around with my recipe"? OR would you have chalked it up to a bad table and made plans for Rockford.

(poking the bear) You finished in 6th place do you think you should have finished higher?
Click to expand...


I would have wondered what went wrong. I would have been 90% sure it was a bad table, but I would have still wondered if I really did cook as good of a brisket as I thought I did. After two or three bad contests I would start looking at making changes. I don't know statistically what the odds would be of hitting the bad table in three straight contests, but at least now it can be confirmed.

I don't think I should have finished higher or lower as 4/5 teams above me also hit table 2 and it is all part of the "luck of the draw". I'm not complaining about coming in 6th (actually really happy I did :grin:) or complaining about the judges. The fact is that table 2 was a low scoring table at that contest and hitting that table or not hitting it could have drastically changed the outcome for teams.
 
Slamdunkpro said:
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.
Click to expand...

It's perhaps useful if you want to convince yourself your bad day was the result of hitting "bad" tables and that a successful team had a good day because they hit "good" tables. A single day's i.e. comp's worth of results are not enough to determine if you were unlucky, the other guy was lucky or maybe you just didn't cook as well as you thought. Anyway, I can't influence what table I end up on, right? And even if I could how would I know which is good or bad?

After I get about 20 of these news scoresheets in hand, maybe I'll take a look and see if I see any "trends". But it's kind of like the time I had this ginormous 119 cut brisket and the fat on the deckle looked eerily like a face. Just because it looked like a face didn't mean there was actually a face there. Just because I think I see any trends doesn't mean there is anything of any real significance there. The only "trend" I am going to worry about is how I finish in each category and the overall. If I'm doing OK then I'm happy. If I start stinking it up, then I'll look at making changes. Really, it's all about how you finish on that day and in those circumstances. Once that set of entries, those judges and that set of competitors is dispersed it's back to zero.
 
Balls Casten said:
I agree if thats all its used for.

But what if some of us dont want you guys having more info? Maybe this additional info should only go to teams that have not won a contest. :mrgreen:
Click to expand...

Thats the beauty of it though. Everyone comes away with the same info.
 
Last edited:
G$ said:
How many contests are needed to make that number statistically meaningful?
Click to expand...

Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.
 
gettinbasted said:
I don't advocate any kind of judge training program based on their score history. G$ makes the perfect point as to why. There just won't be enough data. Judge reeducation is a dangerous path to go down.

BBQ judging is subjective. I have received a 180 and a 160 on essentially the same piece of meat at two competitions. The truth was probably somewhere in the middle. I would love to have that 160 be a 170, but I don't want to give that 180 pin back!
Click to expand...

Respectfully, I disagree. Reeducation is hardly dangerous. If someone is doing something that is way outside the norm, it could very well be due to a lack of understanding and reeducation can easily and quickly solve that problem. Remember, we're not talking about reeducating judges that fall within a statistical norm but rather those who fall well outside of what could be considered a normal variance. A judge who regularly scores everything high or everything low is not really judging anyway.
 
I have not looked at the new KCBS scoresheet in detail, but it sounds like what FBA has provided for years.
When we competed, we found the data useful, but only in the "macro" sense.

As many have said--if we wound up on a low scoring table, we looked at our numbers as such--took it with a grain of salt.

If we got great numbers, but so did every one else--we took as a blessing.

If our table killed us, but not the others, we looked at working on our cooking.

What is interesting is to track the individual judge scores.
At one event, we got nailed with a 10-7.5-7.5 from one judge. Totally out of whack with the other 10's and 9.5's. Cost us some serious $$.

So, we looked at the other scores from this judge and they were all--100%--10-7.5-7.5 except for one entry that got a perfect 10-10-10 from all judges.
Having judged a bit, I know exactly what happened. This Dude or Dudette knew that if they gave 10's on appearance, the Reps could not fault them.
When the super entry came in, the other, shall I call them "seasoned judges", clearly showed their "awe" and "Mr 10-7.5-7.5" figured he/she needed to score up. 8)

We retired from competing for many reasons, this was just a piece of the puzzle.

That info should be available to you KCBS folks now.

Good Luck y'all!!

TIM
 
ique said:
Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables
Click to expand...

Smokin' Joe said:
This should happen!
Click to expand...

musicmanryann said:
My thinking as well. Once the last judge checks in the rep clicks a button and the computer assigns judges to each table virtually instantly. Again, I'm not sure if the new program is keeping track of scoring across multiple contests or not, but if it is this would seem relatively simple.
Click to expand...

I know that this sounds like a good common sense approach, but it is a bad idea statistically speaking. I call this "inverse statistical regression". If you understand statistical regression, you can explain how I am wrong AND I'll read your response. It may also introduce bbq to the criterion problem.

The issue would be compounded if the judges knew where their average scores fell.

Some how I bet no one will heed this warning. It something has face validity, to hell with construct validity.
 
i can only imagine that because the statistics are manipulated and not random they are no longer valid. and now that they are no longer valid, they are no longer useful because they were manipulated.

and once you force a smoothing of data, well, we'll probably all get sevens every category, every entry, kinda thing.

was i close????:becky:
 
Ok, so, why did KCBS institute this new system of delivering the results to the competitors? Was it for us to try and micro manage our cooking? Was it for them to track the judges?

I am only hoping it was for them to track the judges.

But then, what are they going to do about it?

Really, why provide the competitors this more in depth information?

wallace
 
Vince RnQ said:
Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.
Click to expand...

Actually, I am not sure I disagree, but we must be careful not to give inordinate importance to a sample size of ... one. Honestly, I am chewing on the ramifications associated with seating of judges based on historical averages, and I don't think I know what my opinion is on it yet.
 
Slamdunkpro said:
There is no value because (1) you don't know the circumstances of those tables and (2) you'll most likely never encounter that set of variables (judges at that table, other entries at that table) again. Back to the roulette example. The odds of any one number coming up are 37-1; if 18 comes up 6 times in a row what are the odds it will come up 18 the next spin? 37-1. The odds always remain the same because they reset every spin. Your contest history within the aspect of judges and tables and all variables (N) is always N=1. It resets every event.
Click to expand...

The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.
 
Vince RnQ said:
Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.
Click to expand...

Good idea, but I believe it would be an administrative nightmare for the reps, at the present time as you point out. Maybe down the road after this new system has had time to settle in, and the reps programs on the laptops at the comps could/would accommodate this, I agree.

wallace
 
G$ said:
The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.
Click to expand...
Blackjack shoes have a known history and it's possible to predict the outcome with ever increasing accuracy as N approaches zero. the same would be true if your turn in's hit the same table of judges week after week after week. They don't.

The constant in the equation is actually the food as most of us vary our recipes very little and have gotten reasonably successful at producing a constant product from week to week. The main variable is the judge.
 
Vince RnQ said:
So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?
Click to expand...
That's hard to say Vince. I know the experience level sort helps a lot but it's not the perfect answer. You know how obsessive I am about my sort and I've modified it since I've seen you and it's even more granular. I did a poll of the cumulative number of contests judged for the judges at each table. The lowest table had a total of 101 contests and the highest table had 113 contests. This gave a 12% variance between the highest and lowest tables which is not a bad sort. I still had one tough table where the highest score was a 10th place chicken. I had one moderately warm table but nothing I would consider super hot. I've had contests where there were no hot or cold tables using this method.
Sorting based on historical average could be interesting. It would have to be tested as I don't believe a model can be built to test this because of the human factor but someone may prove me wrong. I don't know if I would rather see this or have some re-training for the judges that are consistently off target on their judging.
Since I don't have access to historical data I guess it doesn't matter anyway. lol
Thanks Vince.
 
Cooks as judges

While I believe all cook teams should have members that judge at least on occasion, I have found over the years as a judge that they are the toughest/lowest scorers. Reps should account for who are mostly cook team members at contests and distribute them as they would VIPs or newbies.
 
I'm wondering if there is renumbering of tables and judge seats on the printouts for anonymity, like there is with team entry boxes, so we keep the true double-blind. At comps that have tents for judging, it wouldn't be too hard for someone to at least figure out the table numbers. I'd hate to lose one of my cook friends because they figured out my table and seat number and my score didn't agree with what they thought it should be.

And I wonder how KCBS is going to re-educate outliers. So far, continuing education consists of one on-line test which I easily ace. Tracking won't do much good if they don't have a plan for what's next.
 
the problem i'm having with sorting through historical data is we would essentially be removing that "i didn't like it" spike in scoring.

clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.

i think it may do a real disservice to the competitors and the sport to start sorting judges.
 
boogiesnap said:
clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.[/ QUOTE]

+1. That goes for me.

wallace
Click to expand...
 
One variable that I don't think has been brought up is the celebrity judges.
To me, that is a problem, you can re-educate existing judges all you want, how do you factor in a high percentage of judges who are judging their first contest whith no historical data on how they might score entries?
 
You must log in or register to reply here.

Similar threads

H
Used Napoleon Prestige 500SIB
Replies
5
Views
391
cholloway
cholloway
N
American Royal Open Contest - Incorrect Results
Replies
6
Views
4K
Bill-Chicago
Bill-Chicago
Phubar
"Happy Phriday Pharkers,what's Smoking this Weekend?"
2
Replies
24
Views
649
LYU370
LYU370
G
KCBS membership required
Replies
6
Views
685
Smokin D@wg@77
S
BBQchef33
I forgot the new avatar library.
Replies
8
Views
75
Big George's BBQ
Big George's BBQ
Back
Top