New KCBS scoresheet

Admittedly I am not the mathematician that many of you are, I understand and agree with you that there is not enough of a constant in the equation to utilize the new data to to predict a statistical probability of future outcomes. The new data can however provide and excellent analytical review of an event that has already taken place and I think that is where it's value is.

The new scoring system will allow me to see for example that over the course of the 24 entries judged by table x only 1 top ten score in any category. Understandably, I don't get to sample any of the food but I can further determine that out of the 24 entries judged 3 were submitted by current or former Jack/AR champs, 10 were from teams currently in the Top 50 in there respective category, 1 was a Sam's regional champion and 5 others were from 3 teams with 11 combined GC's this year. Looking at the quality of the teams submitting these entries, I think it is very reasonable to think those results are the more likely the result of a scoring anomaly than deviation of your process/recipe. Now I can see that the brisket I way overcooked but finished 2nd with was more than likely due to me hitting the table that put 14 entries into the top ten from 11 teams that have never had top 10 than judges suddenly liking brisket cooked to 215*.
 
Here's a thought, how about they make a nine table and a 8 table and so on and you just take your chance on where you land?
Then you guys can figure the statistical chances of which table you would hit based on the order you hit the turn in table and throw in the odds of which hand you use to carry your box?
Go for it.
Ed
 
boogiesnap said:
i can only imagine that because the statistics are manipulated and not random they are no longer valid. and now that they are no longer valid, they are no longer useful because they were manipulated.

and once you force a smoothing of data, well, we'll probably all get sevens every category, every entry, kinda thing.

was i close????:becky:
Click to expand...

Correct, but currently the statistics are not random and are being manipulated. Reps currently sort the judges and seat them based on their experience under the assumption that experience is a factor in how a judge scores.

We are not talking about normalizing how an individual judge scores, but rather the table as a whole, and across all the tables at a contest. You may still the straight 777s, but it would more likely be from one judge at all four tables you hit, rather than one table with four judges killing you. :mrgreen:
 
ModelMaker said:
Here's a thought, how about they make a nine table and a 8 table and so on and you just take your chance on where you land?
Then you guys can figure the statistical chances of which table you would hit based on the order you hit the turn in table and throw in the odds of which hand you use to carry your box?
Go for it.
Ed
Click to expand...

Do you have such a problem with the way we do it now? I personally think that the way it is currently set up, as the number of contests you cook increases the closer you get to your true ranking.
 
musicmanryann said:
Correct, but currently the statistics are not random and are being manipulated. Reps currently sort the judges and seat them based on their experience under the assumption that experience is a factor in how a judge scores.

We are not talking about normalizing how an individual judge scores, but rather the table as a whole, and across all the tables at a contest. You may still the straight 777s, but it would more likely be from one judge at all four tables you hit, rather than one table with four judges killing you. :mrgreen:
Click to expand...

understood, but that is sorting by person. using historical data the reps would be sorting by numbers.

quite different IMHO.

doesn't really make a difference though probably because there is always the human factor. however, if you balance the tables by the numbers, it is possible, over time, every entry in every category, from every table COULD start getting the exact same totals.
 
boogiesnap said:
understood, but that is sorting by person. using historical data the reps would be sorting by numbers.

quite different IMHO.

doesn't really make a difference though probably because there is always the human factor. however, if you balance the tables by the numbers, it is possible, over time, every entry in every category, from every table COULD start getting the exact same totals.
Click to expand...

It would be quite different if they sorted by age, eye color or their birth month. However, the sorting by person is done under the premise that their experience determines how they score. Individuals' scoring trends are still there in the sorting even though the numbers are not necessarily known. On the other hand, I know Reps who have a good idea who the black sheep are when they regularly attend contests, and more than a couple of organizers who blacklist judges who consistently score above or below the rest of the table. So perhaps more than a little is known, but probably not consistent.

I do believe there is a correlation between scoring and experience, and am thankful for the reps who go to great lengths to ensure tables are sorted thoroughly based on this variable. Hopefully, as data is collected from the new scoring system, light will be shed on the nature of the correlation, and yield more information that may help refine the table sorting process. In the end we all want the best food, and not luck to win the day.
 
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).
 
Kit R said:
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).
Click to expand...

Now there's an idea worth serious consideration!:clap:

wallace
 
Kit R said:
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).
Click to expand...

I think you summed it up well. You can manipulate the judges or the tables, but when you do both you fark the goose.
 
Kit R said:
Then the distribution of the judges WOULD be truly random
Click to expand...

You answered it yourself.

Many competitors do EVERYTHING THEY CAN to remove any trace of randomness to their whole process. THEY DO NOT WANT RANDOM JUDGING. They want what they consider fair and equal judging.

For the record, I am not personally saying this is my point of view, but it is a point of view of many very excellent competition barbeque teams.
 
G$ said:
You answered it yourself.

Many competitors do EVERYTHING THEY CAN to remove any trace of randomness to their whole process. THEY DO NOT WANT RANDOM JUDGING. They want what they consider fair and equal judging.

For the record, I am not personally saying this is my point of view, but it is a point of view of many very excellent competition barbeque teams.
Click to expand...

yes, fair and equal per judge. we're talking per table, know what i mean?

the use of random in this thread isn't what your'e thinking.
 
boogiesnap said:
yes, fair and equal per judge. we're talking per table, know what i mean?

the use of random in this thread isn't what your'e thinking.
Click to expand...


Inigo Montoya said:
You keep using that word. I do not think it means what you think it means.
Click to expand...

I know very well what the folks that want what they would term "fair judge seating" (and as a result "fair turn in distribution") based on average scores means, and there is nothing random about it. Point blank, they don't want random. And maybe that is best, but it is not random.
 
Just throwing this out there .... Distribute all the "low" scoring judges and all the "low" scoring judges get thrown out? ?????
 
  • Thanks
Reactions: G$
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????
 
Smokinclone said:
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????
Click to expand...

That is a good point, Darren. Seems it would be best to not include such outliers in the analysis. If the score contains a score less than or equal to 1, it would receive a NULL value in the database. Or we use the median value of scores, rather than average to compensate.
 
Smokinclone said:
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????
Click to expand...

First of all, congrats on your GC in Holbrook! Sorry that we didn't see you in person after the awards ceremony. I also want to say that my analysis of the score sheet for that event in no way reflects negatively on your team's performance. You were in the lead after every category and I think, based upon your brisket score, that even if yours and the Hill's briskets had both missed Table 4 that you would still have taken GC but that the margin of victory would have been narrower. Regardless, great work, great cooking and a great win!

I think you bring up a good point about how to factor DQs into the equation but I also think that DQs will represent such a small percentage of judge's historical average so as not to require much weight. That will, of course, remain to be seen.

As for Holbrook, I only see one DQ on the score sheet and that was in the Brisket category as a late entry. That entry was assigned to Table 4. While the KCBS scoring average did include that DQ in the judge's contest scoring average, I did not include it when I ran the placing averages for that table since clearly a late entry has nothing to do with the judges. Regardless, Table 4 accounted for the last non-DQ placing in every category and for 10 of the 20 lowest non-DQ placings across the categories.
 
You must log in or register to reply here.

Similar threads

H
Used Napoleon Prestige 500SIB
Replies
5
Views
391
cholloway
cholloway
N
American Royal Open Contest - Incorrect Results
Replies
6
Views
4K
Bill-Chicago
Bill-Chicago
Phubar
"Happy Phriday Pharkers,what's Smoking this Weekend?"
2
Replies
24
Views
651
LYU370
LYU370
G
KCBS membership required
Replies
6
Views
687
Smokin D@wg@77
S
BBQchef33
I forgot the new avatar library.
Replies
12
Views
126
BBQchef33
BBQchef33
Back
Top