PDA

View Full Version : New KCBS scoresheet


big matt
07-14-2013, 07:56 PM
Anyone get a new printout yet?..thoughts?

Bentley
07-14-2013, 08:24 PM
What did they change?

MarleyMan
07-14-2013, 08:27 PM
Edit - See below :thumb:

big matt
07-14-2013, 08:44 PM
They now have more info like what table you hit,how you did on that table and it shows low scoring tables too..I liked it.

MarleyMan
07-14-2013, 08:55 PM
It is a lot nicer being able to compare tables and what teams you were up against on the same table etc...

I like the format much better and it is cleaner than the old "typewriter" printouts.

It is the same format as what is shown in this thread.

http://www.bbq-brethren.com/forum/showthread.php?t=143325

K-Train
07-14-2013, 09:00 PM
I like it

Scottie
07-14-2013, 09:14 PM
About damn time....

oig2scout
07-14-2013, 09:18 PM
I like it, I had great ribs and was the #1 on that table but they trashed everyone on the table. Looking over the other categories with one exemptions they beat up on all their boxes. Proves the long held theory..."sometimes it IS the table you end up on!

fnbish
07-14-2013, 09:25 PM
Gave us more to over analyze on the ride home yesterday :becky:. I like it. Very cool to see how you faired at he table an who else was at the table.

CBQ
07-14-2013, 10:38 PM
The judges that give EVERYONE low scores really stand out now. A lot more data to look at now. You can see what teams you were up against, whether you went against those teams at multiple tables, how each judge scored each team.

It also gives us more stuff to wonder about :becky: Like in Troy, NY today we were ranked 1st on every table our product landed on, but 4th overall in the contest. Was it the product? Did we just hit a low scoring table? Our ribs were middle of the pack overall, but still won the table that they were on. How much did the table we hit influence the result?

There is a lot more data, and a lot more possibilities to consider now, and I'm sure there will be many discussions. :popcorn:

Basted moral support guy
07-14-2013, 11:14 PM
It made us dislike table five.

Balls Casten
07-15-2013, 05:35 AM
The judges that give EVERYONE low scores really stand out now. :

and I hope the judges that hand out high scores stand out as well.

Chipper
07-15-2013, 05:43 AM
We like it. Much better than the old one.

Pig Headed
07-15-2013, 06:39 AM
We liked the new format, now if it could only teach me how to cook brisket

EMTTLC
07-15-2013, 06:55 AM
Now they need to get a print out in each judges hand.

bignburlyman
07-15-2013, 07:00 AM
It made us dislike table five.


Oh snap! Now everyone will hate table 5 as much as they hate judge #6, lol.

CBQ
07-15-2013, 07:54 AM
and I hope the judges that hand out high scores stand out as well.

Aren't many of those in New England :becky: A team from here goes down south, and it adds 20-30 points to your score.


I really do like the new system, gives us a lot of good feedback. They also handed the teams a note saying that comment cards were not integrated with the new software, and would (as least temporarily) stop being used. (Our rep told us the new numbering scheme made it harder to track the comment cards.) I do like the cards, but they don't get used often enough, and the new scoring system does give you a lot of useful information, so it's not a bad trade off. I wouldn't mind seeing them come back, but I suspect KCBS wanted to drop them anyway. I'm sure that they could have figured out how to track the cards if they wanted to.

Now the question is: is this information also going to be used to evaluate judge's performance? Judges that, for example, give everyone a 7 all the time (or a 9) will be exposed here. Maybe those judges need to attend the CBJ course again. Mike from Lakeside is now retired from BBQ and making beer. He tells me that beer judges have to pass a certification exam, one that is quite difficult. Maybe a "lite" version of that asking questions about the KCBS rules, even an online test taken periodically, would be a good thing. Taste is subjective, but people mark down for things they shouldn't like the number of pieces (over 6), smoke ring, meat temp - a periodic validation that people understand the rules wouldn't be a bad thing.

One thing we heard in Troy from a judge, after the turns ins. They heard another judge say they mark down for less than 8 pieces of meat, because they want the "table captains to have a choice too." I'm sorry, but that is a little over the top for me. 6 pieces is the requirement. I judged this contest last year, and spoke directly to a judge that was marking down for 6 pieces. I explained that for chicken we liked to include the biggest, meatiest pieces of chicken we could find to give the judges the most flavor, and if we find what we want, we can't fit more than 6 in the box. We prep more pieces, and it doesn't mean we don't have more that are turn in quality, but we can only fit six unless they want us to do the little trimmed balls 'o chicken thing. THAT judge, at least, later told me she hadn't thought of it that way and now no longer marks down for that.

Anyway, the now scoring system could be used to detect "problem judges", will be interesting to see what happens with it.

Eggspert
07-15-2013, 08:12 AM
I really liked the new printouts and getting more information. I thought it was interesting for us this weekend at a smaller contest we were frequently with the same teams at the table. This makes sense, since they try to not give the same table the same teams food. We had a couple of judges really give low scores that were not consistent with the rest of the table and I had hoped that would be consistent with other teams, but it did not appear so this time. Any time you get more information, it's helpful.

Eggspert

Balls Casten
07-15-2013, 08:40 AM
.. we were frequently with the same teams at the table. This makes sense, since they try to not give the same table the same teams food. We had a couple of judges really give low scores that were not consistent with the rest of the table and I had hoped that would be consistent with other teams, but it did not appear so this time. Any time you get more information, it's helpful.

Eggspert

In principle I agree but I really feel for the contest reps and responsible parties at contests that will have to field all the complaints this will generate. Information can be a liability not a benefit.

drbbq
07-15-2013, 08:59 AM
We had a couple of judges really give low scores that were not consistent with the rest of the table and I had hoped that would be consistent with other teams, but it did not appear so this time.
Eggspert

So these judges gave you low scores and not the other teams? I'm guessing they didn't like your entry.

Slamdunkpro
07-15-2013, 09:21 AM
On the judge's average score, what is N? is N that judge's average for that particular contest or is N the cumulative average score for that judge over their entire history?

ThomEmery
07-15-2013, 09:41 AM
Mike Davis posted a picture of his print out in his facebook feed
Lotta Bull had a great weekend for sure
Thanks for for putting it up there Mike and Debbie

Funtimebbq
07-15-2013, 10:10 AM
With at least 13 tables, wonder why the reps and table captains allowed two of his meats on the same table.

Kit R
07-15-2013, 10:25 AM
We really like it. A few observations:

At the risk of sounding obvious, the information contained in the new scoresheet has always existed but was not reported. I've heard a lot of grumbling about "bad tables", "Judge #5", "hard a** judges", etc. in the three or so years since we started competing, and now you can see exactly what table you landed on, who you were scored with, what the judges gave you, how your score compares to that judges' average score, and how you ranked on that table. If someone is of a mind to say they "got screwed by bad tables" I guess they've got something with which to try and make that argument. I look at it the other way: I've heard people say too many times to count "my _______ was terrible" and then they get a call in that category. Maybe on those occasions you thought your entry was bad, maybe it was. Maybe you landed on a "good" table and lucked out. Bottom line, over time the "good/bad" table/judge (if there really is such a thing) will even itself out. The very best teams win week in/week out because they cook consistently well, regardless of who is judging their food. You might suffer a little on a given week, or you might catch a break. But if you bring it strong every time the results will reflect that.

As far as the concept of good/bad tables, I'm not convinced there really is such a thing. Our practice is to turn in certain entries early/late in the allowable windows, and I am certain other teams do the same. I think this could result in multiple entries from more experienced teams getting turned in together, which could then result in some tables getting mostly experienced team entries and others getting newer teams' boxes. Experienced would reasonably translate to better scores on average, which would look like a "good table". But what if those experienced team entries ended up on a table of very tough judges? Might it then look like a "bad" table? And then I have seen/heard of boxes getting reshuffled onto different tables upon turn in so the judging tables don't necessarily get entries in the order in which they're turned in. All this means to me is anyone can read and interpret the data in any way they wish and justify any conclusion they wish to draw.

We're going to try not to use the additional data to justify a bad cook. Emphasis on "try". We are going to keep a close eye on how our scores compare with each individual judge's average score. If we're doing better than average on that number then I fel like we're on the right track. It will also be interesting how we stack up against the strongest competitors if/when that happens. All in all a giant leap forward in data analysis and a useful tool.

Eggspert
07-15-2013, 10:28 AM
So these judges gave you low scores and not the other teams? I'm guessing they didn't like your entry.

Yes, that is what I know now. Before I assumed maybe the judge scored low on all teams. Now that I know how the judge scored the other entries it helps me understand that that this time the judge did not like our entry. You can't please everyone! I do wish that they could get comment cards going again and would mandate if you give a 6 or lower that you fill one out. We got a 6, when the rest of the judges scored 8-9, and we got a call on the entry??? Wierd.

Eggspert

Kit R
07-15-2013, 10:32 AM
On the judge's average score, what is N? is N that judge's average for that particular contest or is N the cumulative average score for that judge over their entire history?


I figure it's probably for the particular contest. It could be cumulative, but I see a potential data collection nightmare. The reps would have to have a datafile for every judge containing every score they'd ever submitted for every contest they'd judged. Sure, it could be done. But if you're sitting in a tent in Gettysburg running the scoring program, how would you access that datafile and integrate it into results for that day?

Scottie
07-15-2013, 10:49 AM
With at least 13 tables, wonder why the reps and table captains allowed two of his meats on the same table.



I think there were 78 teams. You would think they wouldn't hit the same table though

drbbq
07-15-2013, 11:42 AM
Yes, that is what I know now. Before I assumed maybe the judge scored low on all teams. Now that I know how the judge scored the other entries it helps me understand that that this time the judge did not like our entry. You can't please everyone! I do wish that they could get comment cards going again and would mandate if you give a 6 or lower that you fill one out. We got a 6, when the rest of the judges scored 8-9, and we got a call on the entry??? Wierd.

Eggspert

I got ya. As for the 6, it'd be great to know but of course that score got dropped.

Muzzlebrake
07-15-2013, 11:43 AM
It also gives us more stuff to wonder about :becky: Like in Troy, NY today we were ranked 1st on every table our product landed on, but 4th overall in the contest. Was it the product? Did we just hit a low scoring table? Our ribs were middle of the pack overall, but still won the table that they were on. How much did the table we hit influence the result?


I think finishing 1st at each table you should expect to place in the top 5 of a field the size of the one in Troy. The rib table was a killer, we were on the same table finished 2nd on the table which was good enough for 20th O/A. I agree they werent my best ribs but there is no way there were 19 better rib entries at that contest



As far as the concept of good/bad tables, I'm not convinced there really is such a thing. Our practice is to turn in certain entries early/late in the allowable windows, and I am certain other teams do the same. I think this could result in multiple entries from more experienced teams getting turned in together, which could then result in some tables getting mostly experienced team entries and others getting newer teams' boxes. Experienced would reasonably translate to better scores on average, which would look like a "good table". But what if those experienced team entries ended up on a table of very tough judges? Might it then look like a "bad" table?

I disagree, I think it will point out that low scoring table that can kill your scores. Defining that table as wither bad or tough is up to you but I think it wil highlight the table that is out of the norm in either direction. I think an additional statistic ranking each table with the average score given would be really helpful.

Take table 4 from the Troy contest Chris and I landed on together in ribs. In that category we were 1 & 2 on the table which was 13th & 20th O/A. During the entire contest only one team recieved a score from that table good enough for a top 10. Table 2 on the other hand had a minimum of 2 entries in the top 5 of each category.

Granted I don't think there is going to be anything we as cooks can do about this but I do think it will help you get a better understanding of whether your score is a byproduct of your cooking or the judges judging. That goes both ways for good and bad scores, how many times have you turned in something you thought was just downright bad and then heard your name called? I know I will use this data as a point of reference before i go changing anything in my process.

Dan - 3eyzbbq
07-15-2013, 11:47 AM
I like it. We finished Gettysburg with 1st Chicken, 2nd ribs, 2nd pork and 15th place brisket.

Now, for the breakdown. We won each table we landed on, except brisket where we were 2nd. The highest score on brisket was 158, and finished 13th.

I don't know if its a "bad" table, but certainly we all scored low on that table.

Goddahavit
07-15-2013, 11:51 AM
We liked the new format, now if it could only teach me how to cook brisket


or pork for us ha ha....

its ok, i can see this causing some table bashing and such, but hey more ways to blame others for my failure to cook is always a good thing right ?? lol ha ha...

Slamdunkpro
07-15-2013, 11:58 AM
I like it. We finished Gettysburg with 1st Chicken, 2nd ribs, 2nd pork and 15th place brisket.

Now, for the breakdown. We won each table we landed on, except brisket where we were 2nd. The highest score on brisket was 158, and finished 13th.

I don't know if its a "bad" table, but certainly we all scored low on that table.
Clearly I'm missing something - How do you know what tables everyone else landed on? (other than asking each team) I only see what tables we landed on.

Dan - 3eyzbbq
07-15-2013, 12:01 PM
Look at each sheets breakdown, next to the judges scores. You, for instance, never landed on table 2.

Smitty250
07-15-2013, 12:02 PM
Clearly I'm missing something - How do you know what tables everyone else landed on? (other than asking each team) I only see what tables we landed on.

If you look on the individual sheets for each meat it will show to the right of each team what table they were on.

Stoke&Smoke
07-15-2013, 12:13 PM
so when is this to be rolled out across KCBS? Sure wish we had ours from the 3 we've done so far!

J&B'sBBQ
07-15-2013, 12:27 PM
so when is this to be rolled out across KCBS? Sure wish we had ours from the 3 we've done so far!

This past weekend was the first time we've seen them. I'm sure you'll see it now when you compete KCBS. It was full of great information for us to analyze our scores.

G$
07-15-2013, 12:27 PM
With at least 13 tables, wonder why the reps and table captains allowed two of his meats on the same table.

Thinking out loud here, but the later the category(eg brisket), the harder it is to avoid a duplication.

musicmanryann
07-15-2013, 12:28 PM
so when is this to be rolled out across KCBS? Sure wish we had ours from the 3 we've done so far!

My understanding was this past weekend.

I think it is farking awesome. I learned more the past two days pouring over the score-sheets than I have the past five years. Thank you KCBS!:eusa_clap

CBQ
07-15-2013, 12:48 PM
I do think it will help you get a better understanding of whether your score is a byproduct of your cooking or the judges judging.

Exactly right. There is a fair bit of KCBS bashing on here (and I'm not immune to that myself on occasion) but I think KCBS does deserve some kudos here.

The new KCBScore appears to accomplish what it set out to do. It gives the teams more context about their scores. Sean had some good observations about our rib scores and having that info available is valuable to the teams as we ponder the questions about why we got the scores we did. Did we hit a bad table or a good one? Do we have a problem with what we turned in it not? Was a generous table masking a problem we do have? The new system does give us insight we didn't have before.

ique
07-15-2013, 12:54 PM
Take table 4 from the Troy contest Chris and I landed on together in ribs. In that category we were 1 & 2 on the table which was 13th & 20th O/A. During the entire contest only one team recieved a score from that table good enough for a top 10. Table 2 on the other hand had a minimum of 2 entries in the top 5 of each category.


Yeah, I think its important to look at all four categories to determine if a table is "bad". When looking at just one category, there are clearly two options, the table is scoring low or the cooks deserved 13th and 20th place.

Its unlikely that across four categories all the table winning entries are out of the top 10. But for a single category it should be expected.

A great improvement.

BMerrill
07-15-2013, 01:02 PM
I competed at this same contest last weekend as Lotta Bull.

From the printout, Lotta Bull was placed on Table 13 twice.
With 78 teams this should not happen.
Table #13 was the highest scoring table with 7 top ten calls.
This information is provided on the score sheets.

Table numbers for 1st-10th of each catagory
CK, 4, 2, 3, 12, 7, 9, 2, 2, 11, 12
RI, 1, 1, 12, 8, 4, 13, 13, 11, 1,13
PK, 2, 13, 5, 4, 3, 7, 1, 12, 13,13
BK, 10,12, 5, 8, 1, 13, 8, 12, 2, 1

Top 10 calls by table.
Table 1, calls 6
Table 2, calls 5
Table 3, calls 2
Table 4, calls 3
Table 5, calls 2
Table 6, calls 0
Table 7, calls 2
Table 8, calls 3
Table 9, calls 1
Table 10, calls 1
Table 11, calls 2
Table 12, calls 6
Table 13, calls 7

My entries landed on tables 9, 5, 6, 10; the 4 lowest scoring tables for top 10 calls. The lowest scoring judge was Judge 1, on Table 9 with average of 25.1633.
On chicken this judge gave us 6,6,7 = 25.1428
The other judges
#2 9,8,8 = 32.56
#3 9,9,9 = 36.00
#4 8,8,7 = 30.8572
#5 9,8,8 = 32.5600
#6 9,8,9 = 33.7028
His/her score appears to be completely out of line with the other judges. Here are all of the scores this judge gave out for the day:

CK= 28.5600, 34.8800, 25.1428, 28.0000, 20.0000, 26.2972
RI = 29.1428, 25.1200, 19.4400, 24.0000, 15.3828, 18.2972
PK= 19.4400, 29.1428, 25.1200, 24.0000, 15.3828, 18.2972
BK= 27.4400, 27.4172, 32.0000, 29.7028, 30.2972, 24.0000

This judge should be on KCBS's radar.

We were told at the cooks meeting.
The blind number will no longer be on the print out.
Are using the new 2013 scoring weights.
Comments cards have been temporary stopped.
The Judge Average Score is from all 4 catagories.
KCBS will be tracking judges.

Muzzlebrake
07-15-2013, 02:10 PM
Its unlikely that across four categories all the table winning entries are out of the top 10. But for a single category it should be expected.

A great improvement.

I completely agree. I think in this case you can clearly see the trends of different tables. Just as table 4 seems to be on the low end of the scoring, Table 2 seems to be the on the upper end. I think it sucks that my stuff it table 4 and its more personal but I'm not sure that I'm not more upset table 2 was what appears to be an "easy" table. I think it's more unlikely that table happens to have at least 2 of the 5 best cooked entries in each category.

Outnumbered
07-15-2013, 02:18 PM
I competed at this same contest last weekend as Lotta Bull.

From the printout, Lotta Bull was placed on Table 13 twice.
With 78 teams this should not happen.
Table #13 was the highest scoring table with 7 top ten calls.
This information is provided on the score sheets.

Table numbers for 1st-10th of each catagory
CK, 4, 2, 3, 12, 7, 9, 2, 2, 11, 12
RI, 1, 1, 12, 8, 4, 13, 13, 11, 1,13
PK, 2, 13, 5, 4, 3, 7, 1, 12, 13,13
BK, 10,12, 5, 8, 1, 13, 8, 12, 2, 1

Top 10 calls by table.
Table 1, calls 6
Table 2, calls 5
Table 3, calls 2
Table 4, calls 3
Table 5, calls 2
Table 6, calls 0
Table 7, calls 2
Table 8, calls 3
Table 9, calls 1
Table 10, calls 1
Table 11, calls 2
Table 12, calls 6
Table 13, calls 7

My entries landed on tables 9, 5, 6, 10; the 4 lowest scoring tables for top 10 calls. The lowest scoring judge was Judge 1, on Table 9 with average of 25.1633.
On chicken this judge gave us 6,6,7 = 25.1428
The other judges
#2 9,8,8 = 32.56
#3 9,9,9 = 36.00
#4 8,8,7 = 30.8572
#5 9,8,8 = 32.5600
#6 9,8,9 = 33.7028
His/her score appears to be completely out of line with the other judges. Here are all of the scores this judge gave out for the day:

CK= 28.5600, 34.8800, 25.1428, 28.0000, 20.0000, 26.2972
RI = 29.1428, 25.1200, 19.4400, 24.0000, 15.3828, 18.2972
PK= 19.4400, 29.1428, 25.1200, 24.0000, 15.3828, 18.2972
BK= 27.4400, 27.4172, 32.0000, 29.7028, 30.2972, 24.0000

This judge should be on KCBS's radar.

We were told at the cooks meeting.
The blind number will no longer be on the print out.
Are using the new 2013 scoring weights.
Comments cards have been temporary stopped.
The Judge Average Score is from all 4 catagories.
KCBS will be tracking judges.

Great stuff here. Thanks.

Balls Casten
07-15-2013, 02:41 PM
My understanding was this past weekend.

I think it is farking awesome. I learned more the past two days pouring over the score-sheets than I have the past five years. Thank you KCBS!:eusa_clap

Great just what we need ... you with more information to improve your cooking. :-)
Careful what you wish for people.

tigerpaw
07-15-2013, 02:53 PM
As a judge I think this is what I was looking for when I first became one - how do I stack up in my opinions versus that of my table mates? Am I scoring a point or two high across the board? Am I scoring a point or two low across the board? It would have been helpful to know early on rather then having my scores be off kilter to those other judges at my table.

I would have liked to have seen my scores from my first say 10 events versus those at my table for comparison. Would have possibly sped me up to being in tune to other maybe more experienced judges.

As for comment cards - I like them as a judge. I feel I can at least tell a team why I gave them a score that may not be to that of the other judges for whatever reason it may be. This eliminates me from ever being referred to as Judge 6. I can point out that the rib I selected was overdone, my brisket burnt end was all gristle or fat, my chicken piece was mushy, my pork was tough and dry, whatever the reason MY selection may not have been the same as the other 5 judges and thus my marking down on your entry. I am NOT judge 6. You may be team 6 for putting a less then stellar piece of meat in the box and I unfortunately had the bad luck of getting it. I have also filled out a card for a perfect score and told them thanks for making this what it was for me to judge today. Yes my 9's told them I loved all aspects of it. But a note also to say thanks. And also sometimes a 8 or a 7 will get a card from me to just point out what they could have done to bring it to a high score. "Loved the tenderness and appearance. And love spice but this was a little bit more than needed for this entry. Took away from the meats taste".

So I hope the comment cards do come back as for right now these new scoring systems tell you that you got a 9 - 6 - 8 or a 8 - 6 - 5 but none of that tells you why.

musicmanryann
07-15-2013, 03:20 PM
As for comment cards - I like them as a judge. I feel I can at least tell a team why I gave them a score that may not be to that of the other judges for whatever reason it may be. This eliminates me from ever being referred to as Judge 6. I can point out that the rib I selected was overdone, my brisket burnt end was all gristle or fat, my chicken piece was mushy, my pork was tough and dry, whatever the reason MY selection may not have been the same as the other 5 judges and thus my marking down on your entry. I am NOT judge 6. You may be team 6 for putting a less then stellar piece of meat in the box and I unfortunately had the bad luck of getting it. I have also filled out a card for a perfect score and told them thanks for making this what it was for me to judge today. Yes my 9's told them I loved all aspects of it. But a note also to say thanks. And also sometimes a 8 or a 7 will get a card from me to just point out what they could have done to bring it to a high score. "Loved the tenderness and appearance. And love spice but this was a little bit more than needed for this entry. Took away from the meats taste".

So I hope the comment cards do come back as for right now these new scoring systems tell you that you got a 9 - 6 - 8 or a 8 - 6 - 5 but none of that tells you why.

What we were told this weekend, is that they are coming back as soon as they can develop a system for numbering the cards that maintains the tru double-blind system they are trying to implement.

Vince RnQ
07-15-2013, 04:42 PM
We competed in Holbrook, AZ this past weekend and I love the new score sheet. Lot's of great info. I also took a little time and used the new information about tables to crunch a few numbers. Here is what I've learned about that event.

There were 27 teams at the event and 5 tables of judges. The actual CBJ percentage is unknown but I know it is less than 100% per the contest reps.

First I figured the Average Placing for Each Meat by Table and then Overall

Table 1
Chicken - 9.83
Ribs - 13.60
Pork - 4.80
Brisket - 13.60
Overall - 10.43

Table 2
Chicken - 16.00
Ribs - 16.17
Pork - 14.60
Brisket - 15.20
Overall - 15.55

Table 3
Chicken - 7.40
Ribs - 9.50
Pork - 14.67
Brisket - 8.60
Overall - 10.23

Table 4
Chicken - 19.80
Ribs - 20.40
Pork - 22.17
Brisket - 20.00
Overall - 20.67

Table 5
Chicken - 17.40
Ribs - 10.80
Pork - 12.00
Brisket - 10.67
Overall - 12.62

From this, it is very easy to tell that the hot tables were Tables 1 & 3, that Table 5 just a bit cooler, that Table 2 was just about at the average and that Table 4 was the Table of Death.

I also calculated the Number of Top 10 Calls by Table and what percentage that represented of all the Top 10 Calls

Table 1
# of all T-10 Calls - 13
% of all T-10 Calls - 32.50%

Table 2
# of all T-10 Calls - 5
% of all T-10 Calls - 12.50%

Table 3
# of all T-10 Calls - 13
% of all T-10 Calls - 32.50%

Table 4
# of all T-10 Calls - 2
% of all T-10 Calls - 5.00%

Table 5
# of all T-10 Calls - 7
% of all T-10 Calls - 17.50%

From this you can see that 65.00% of all the Top 10 calls came from Tables 1 & 3 and that 82.50% of all the Top 10 calls came from Tables 1,3 & 5.

I then calculated the Number of Bottom 10 Calls by Table and what percentage that represented of all the Bottom 10 Calls. (This contest had one team that was DQ'd for being late so there are only 39 places used for this set of numbers.)

Table 1
# of all T-10 Calls - 5
% of all T-10 Calls - 12.82%

Table 2
# of all T-10 Calls - 10
% of all T-10 Calls - 25.64%

Table 3
# of all T-10 Calls - 5
% of all T-10 Calls - 12.82%

Table 4
# of all T-10 Calls - 17
% of all T-10 Calls - 43.59%

Table 5
# of all T-10 Calls - 2
% of all T-10 Calls - 5.13%

This data shows that 69.23% of all the Bottom 10 calls came from Tables 2 & 4 with 43.59% of those calls coming from Table 4 alone. Table 4 was indeed the Table of Death at this contest.

One would expect that in any field of teams there will be a good blend of experience levels and that since all the boxes should be hitting different tables the numbers should average out pretty close to each other by the end of the event assuming that the Reps took proper care in balancing the tables. It is impossible for anyone other than the Reps and the Judges to know if this was done. I have been told that many Reps balance the tables based upon experience but it would seem that this new scoring program would give Reps the ability to balance the CBJs at a contest based upon their historical scoring average instead of expereince which might be a very good thing.

Now, I can be a bit cynical at times and all this new data leads me to the cynical conclusion of wondering how long we will be able to see the data points of which table every entry at a contest landed. I wonder because it, for the first time ever, gives conclusive information regarding the quality of the work done by the Reps and the Judges and that means a whole new level of public accountability for those two groups. I hope all the information stays visible and that it leads to improvement but only time will tell.

One last thing about this contest that I think is interesting: After the first three categories, the eventual GC and RGC were less than 4 points apart and neither team had landed on Table 4. In Brisket, the GC landed on Table 3 and the RGC landed on Table 4. The final margin of victory was 16 points. I can't help but wonder what would have happened if they and both missed Table 4 or both landed on Table 4.

big matt
07-15-2013, 04:57 PM
We competed in Holbrook, AZ this past weekend and I love the new score sheet. Lot's of great info. I also took a little time and used the new information about tables to crunch a few numbers. Here is what I've learned about that event.

There were 27 teams at the event and 5 tables of judges. The actual CBJ percentage is unknown but I know it is less than 100% per the contest reps.

First I figured the Average Placing for Each Meat by Table and then Overall

Table 1
Chicken - 9.83
Ribs - 13.60
Pork - 4.80
Brisket - 13.60
Overall - 10.43

Table 2
Chicken - 16.00
Ribs - 16.17
Pork - 14.60
Brisket - 15.20
Overall - 15.55

Table 3
Chicken - 7.40
Ribs - 9.50
Pork - 14.67
Brisket - 8.60
Overall - 10.23

Table 4
Chicken - 19.80
Ribs - 20.40
Pork - 22.17
Brisket - 20.00
Overall - 20.67

Table 5
Chicken - 17.40
Ribs - 10.80
Pork - 12.00
Brisket - 10.67
Overall - 12.62

From this, it is very easy to tell that the hot tables were Tables 1 & 3, that Table 5 just a bit cooler, that Table 2 was just about at the average and that Table 4 was the Table of Death.

I also calculated the Number of Top 10 Calls by Table and what percentage that represented of all the Top 10 Calls

Table 1
# of all T-10 Calls - 13
% of all T-10 Calls - 32.50%

Table 2
# of all T-10 Calls - 5
% of all T-10 Calls - 12.50%

Table 3
# of all T-10 Calls - 13
% of all T-10 Calls - 32.50%

Table 4
# of all T-10 Calls - 2
% of all T-10 Calls - 5.00%

Table 5
# of all T-10 Calls - 7
% of all T-10 Calls - 17.50%

From this you can see that 65.00% of all the Top 10 calls came from Tables 1 & 3 and that 82.50% of all the Top 10 calls came from Tables 1,3 & 5.

I then calculated the Number of Bottom 10 Calls by Table and what percentage that represented of all the Bottom 10 Calls. (This contest had one team that was DQ'd for being late so there are only 39 places used for this set of numbers.)

Table 1
# of all T-10 Calls - 5
% of all T-10 Calls - 12.82%

Table 2
# of all T-10 Calls - 10
% of all T-10 Calls - 25.64%

Table 3
# of all T-10 Calls - 5
% of all T-10 Calls - 12.82%

Table 4
# of all T-10 Calls - 17
% of all T-10 Calls - 43.59%

Table 5
# of all T-10 Calls - 2
% of all T-10 Calls - 5.13%

This data shows that 69.23% of all the Bottom 10 calls came from Tables 2 & 4 with 43.59% of those calls coming from Table 4 alone. Table 4 was indeed the Table of Death at this contest.

One would expect that in any field of teams there will be a good blend of experience levels and that since all the boxes should be hitting different tables the numbers should average out pretty close to each other by the end of the event assuming that the Reps took proper care in balancing the tables. It is impossible for anyone other than the Reps and the Judges to know if this was done. I have been told that many Reps balance the tables based upon experience but it would seem that this new scoring program would give Reps the ability to balance the CBJs at a contest based upon their historical scoring average instead of expereince which might be a very good thing.

Now, I can be a bit cynical at times and all this new data leads me to the cynical conclusion of wondering how long we will be able to see the data points of which table every entry at a contest landed. I wonder because it, for the first time ever, gives conclusive information regarding the quality of the work done by the Reps and the Judges and that means a whole new level of public accountability for those two groups. I hope all the information stays visible and that it leads to improvement but only time will tell.

One last thing about this contest that I think is interesting: After the first three categories, the eventual GC and RGC were less than 4 points apart and neither team had landed on Table 4. In Brisket, the GC landed on Table 3 and the RGC landed on Table 4. The final margin of victory was 16 points. I can't help but wonder what would have happened if they and both missed Table 4 or both landed on Table 4.
You make some great points Vince..in the meeting the reps spoke about the table of death and how they try to avoid that..guess it didn't happen but I guess you can't prevent it 100%..I also noticed a lot of us hit the same tables together(not the same table twice)and wondered if that was normal?..hitting tables with the same teams throughout all meats.

boogiesnap
07-15-2013, 05:00 PM
i only saw the new score sheet for couple minutes(it wasn't mine).

after all that work, would it be possible to narrow down what was going on at table four by judge?

like say, one or two individuals were actually bringing the whole table down?

i'd find it hard to beleive all six judges at a table were lowballers.

i'd also find it hard to beleive all six entries for all four categories were subpar that hit table 4.

Smitty250
07-15-2013, 05:08 PM
[QUOTE=Vince RnQ;2552676]
There were 27 teams at the event and 5 tables of judges. The actual CBJ percentage is unknown but I know it is less than 100% per the contest reps.
QUOTE]

Vince - I spoke with one of the judges after the comp and she said it was 65% CBJ. 35% were VIP's from the town of Holbrook - Approx 2 non CBJ's per table!

mobow
07-15-2013, 05:11 PM
i only saw the new score sheet for couple minutes(it wasn't mine).

after all that work, would it be possible to narrow down what was going on at table four by judge?

like say, one or two individuals were actually bringing the whole table down?

i'd find it hard to beleive all six judges at a table were lowballers.

i'd also find it hard to beleive all six entries for all four categories were subpar that hit table 4.

I have judged contest where all the entries for a category were nothing to get excited about as well as some where all the samples were very good. Keith

boogiesnap
07-15-2013, 05:14 PM
I have judged contest where all the entries for a category were nothing to get excited about as well as some where all the samples were very good. Keith

understood and agreed, but ALL the entries for ALL the categories?

i mean, really, the only two stable tables were two and five. one and three were high and four was low.

Funtimebbq
07-15-2013, 05:22 PM
Thinking out loud here, but the later the category(eg brisket), the harder it is to avoid a duplication.

The more tables a contest has, the easier it is to make sure a team does not hit the same table more than once. Think about it, if a contest has 13 tables, only 3 of those had that team's 3 prior turn-ins (less than 25%). If there were only 4 tables, 75% would have had a team's 3 prior meats.

I also believe table captains are given sheets to mark off and check to make sure they don't get the same boxes twice. Maybe that only happens with certain KCBS reps, I'm not sure.

Scottie
07-15-2013, 05:23 PM
[QUOTE=Vince RnQ;2552676]
There were 27 teams at the event and 5 tables of judges. The actual CBJ percentage is unknown but I know it is less than 100% per the contest reps.
QUOTE]

Vince - I spoke with one of the judges after the comp and she said it was 65% CBJ. 35% were VIP's from the town of Holbrook - Approx 2 non CBJ's per table!


Tucson Sam's had 3 new judges per table. My table was the only table with only 2 brand new judges. But we also had 2 judges that it was only their second contest judging... so if you wonder why scores are crazy.

To just got out of the first round is the hardest at Sam's and a lot of good teams get burned by I experienced judging. No thank you.

Smitty250
07-15-2013, 05:26 PM
What was interesting to me was that we were told in the cooks meeting that this was a true double blind system. After the competition I was looking over the new score sheet with a fellow competitor and his family member judged the comp. She also looked at my score sheet (first time she has seen it as well) and said she can tell me exactly what she scored me (and any other competitor for that matter) if she sees thier score sheet since she knew her table number and seat number.

She seemed to think the old way was more "blind" as she never remembered the alternate numbers that the teams were given upon turn-in and our old score cards did not say what table number our entry was on.

I have never judged so I don't know what goes on "behind the curtain" but it seems to me like the old score sheets were more "blind" than the new ones? Does this make sense?

Don't get me wrong - as a competitor I really like the new score sheets especially seeing what other teams were on my same table.

boogiesnap
07-15-2013, 05:37 PM
[quote=Smitty250;2552711]


Tucson Sam's had 3 new judges per table. My table was the only table with only 2 brand new judges. But we also had 2 judges that it was only their second contest judging... so if you wonder why scores are crazy.

To just got out of the first round is the hardest at Sam's and a lot of good teams get burned by I experienced judging. No thank you.

What table number were you? Just curious.

Smitty250
07-15-2013, 05:43 PM
[quote=Scottie;2552733]

What table number were you? Just curious.

Scottie is talking about a different competition earlier this year in AZ. He didn't judge Holbrook.

sdbbq1234
07-15-2013, 05:43 PM
I think the new system will be a good thing as long as KCBS uses it to re-educate (if needed) judges on what the rules are.

As for me, the last comp Hoggin Up in Winchester, VA, it would have been great to have this in place as far as our chicken goes.

Our scores were:
999
998
999
987
778
978

And yes, we used that "chicken style" that everyone seems to shy away from. I think you know what I mean. But to get 9's across the board for appearance except for 1 judge, I ain't sure about that. And the reason I say that is last year when we judged Smokin' On The Bay, as we were leaving the judging tent area, I overheard a couple of the judges talking about the certain "chicken style". One said to the other, "If I ever see another chicken ball again, I swear I will give it the lowest score I can! I hate that crap"

I wish I had more balls (not chicken balls :evil:) at the time and told him he was messed up (although I would have used a few different words). That was bull crap as he was not giving everyone a fair shot. Preconceived bias about what was being presented is farked!

No excuses for my scores; they are what they are and I'll take what I get. I just hope that the new system will help weed-out some of this. But like most things in my life, reality is what it is: Schit in one hand and wish in the other; see which one fills up first.

I look forward to our next comp. :mrgreen:

wallace

kellym
07-15-2013, 05:46 PM
The more tables a contest has, the easier it is to make sure a team does not hit the same table more than once. Think about it, if a contest has 13 tables, only 3 of those had that team's 3 prior turn-ins (less than 25%). If there were only 4 tables, 75% would have had a team's 3 prior meats.

I also believe table captains are given sheets to mark off and check to make sure they don't get the same boxes twice. Maybe that only happens with certain KCBS reps, I'm not sure.
You're correct, the more tables, the eaiser to prevent duplicates. If there are at least four tables there should be no duplicates although with this few tables this takes a bit of planning and coordination but it's possible.
I believe ALL Reps are to use the tracking sheets. If they don't, how do they prevent duplicates? You can have a table captain OCCASIONALLY not cross off all the boxes on his tracking slip and then pick up a duplicate box however this should be extremely rare.

Vince RnQ
07-15-2013, 06:29 PM
i only saw the new score sheet for couple minutes(it wasn't mine).

after all that work, would it be possible to narrow down what was going on at table four by judge?

like say, one or two individuals were actually bringing the whole table down?

i'd find it hard to beleive all six judges at a table were lowballers.

i'd also find it hard to beleive all six entries for all four categories were subpar that hit table 4.

Since we are able to see the Judge Average Score for the entire contest, another thing that the new score sheet shows is that every judge at Table 4 in Holbrook had a contest average of 29.1912 or lower. Of the remaining three tables that I can see, (Tables 1, 2 & 5. We did not hit Table 3 so I have no data for their average scores on my sheet.), only 1 of those 18 judges had an average score below 30.0000 and 10 of those 18 had averages of 32.000 or higher. The actual average scores for the judges at Table 4 were as follows:

Judge 1 - 29.0094
Judge 2 - 28.7486
Judge 3 - 28.5371
Judge 4 - 26.2769
Judge 5 - 26.8790
Judge 6 - 29.1912

Their combined average for the contest was 28.1070

The combined average for the other tables I can see are:

Table 1 - 32.4984
Table 2 - 31.0785
Table 5 - 31.9779

That's a huge difference.

Vince RnQ
07-15-2013, 06:31 PM
You're correct, the more tables, the eaiser to prevent duplicates. If there are at least four tables there should be no duplicates although with this few tables this takes a bit of planning and coordination but it's possible.
I believe ALL Reps are to use the tracking sheets. If they don't, how do they prevent duplicates? You can have a table captain OCCASIONALLY not cross off all the boxes on his tracking slip and then pick up a duplicate box however this should be extremely rare.

So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?

BigZ
07-15-2013, 06:35 PM
This is going to be a newby comment so bare with me, (I have never been in a competition yet, next month will be first) with out knowing what the "old" sheet was I see that the new one can help you/kcbs figure out which Judge you don't want to have or that needs more training. Since you don't get to choose your table and the same Judge's don't go to all of the comps, how does this "help" you as a cook to improve with out the comment cards? I can see knowing your score can tell you what area you were good or bad in though.

gettinbasted
07-15-2013, 06:38 PM
Since we are able to see the Judge Average Score for the entire contest, another thing that the new score sheet shows is that every judge at Table 4 in Holbrook had a contest average of 29.1912 or lower. Of the remaining three tables that I can see, (Tables 1, 2 & 5. We did not hit Table 3 so I have no data for their average scores on my sheet.), only 1 of those 18 judges had an average score below 30.0000 and 10 of those 18 had averages of 32.000 or higher. The actual average scores for the judges at Table 4 were as follows:

Judge 1 - 29.0094
Judge 2 - 28.7486
Judge 3 - 28.5371
Judge 4 - 26.2769
Judge 5 - 26.8790
Judge 6 - 29.1912

Their combined average for the contest was 28.1070

The combined average for the other tables I can see are:

Table 1 - 32.4984
Table 2 - 31.0785
Table 5 - 31.9779


That's a huge difference.

Table 5 was the "table of death" at wellsville this weekend. It was good to see that our 19th placed chicken was the highest score at the table for chicken and that our 19th place finish in that category was the highest place of any single item that hit table 5.

At past competitions I would have had many hours of hand wringing and analyzing to explain a subpar finish, but this time I could safely mark it up to landing on the wrong table.

oig2scout
07-15-2013, 07:13 PM
As another candidate for table 5 at Wellsville I agree, it was a tough table at best. However, if we as cooks are held to a standard, then some how judges should be held to a standard of being fair in the overall scheme of things and if historically they are continually out of sync with other judges over time then they need to either be re-educated or politely taken aside and talk to about their quality standards. With that said the nearby judge or 2 time judge will still have issues but if the rep keeps them to one per table then it really doesn't matter as their potential low score will be dropped. Overall the new scoring sheet is a vast improvement...good job KCBS and everyone who took countless hours to make it happen!

bbq.tom
07-15-2013, 07:45 PM
However, if we as cooks are held to a standard, then some how judges should be held to a standard of being fair in the overall scheme of things and if historically they are continually out of sync with other judges over time then they need to either be re-educated or politely taken aside and talk to about their quality standards.

It is my understanding that THAT is exactly what this KCBSCORE system will be used for. KCBS can look at the results and see which judge needs re-education or whatever.

G'pa Herb
07-15-2013, 08:22 PM
What was interesting to me was that we were told in the cooks meeting that this was a true double blind system. After the competition I was looking over the new score sheet with a fellow competitor and his family member judged the comp. She also looked at my score sheet (first time she has seen it as well) and said she can tell me exactly what she scored me (and any other competitor for that matter) if she sees thier score sheet since she knew her table number and seat number.

She seemed to think the old way was more "blind" as she never remembered the alternate numbers that the teams were given upon turn-in and our old score cards did not say what table number our entry was on.

I have never judged so I don't know what goes on "behind the curtain" but it seems to me like the old score sheets were more "blind" than the new ones? Does this make sense?

Don't get me wrong - as a competitor I really like the new score sheets especially seeing what other teams were on my same table.

I was a table captain at Holbrook, and I believe that the CBJ % was less than 65%. There were at least 3 VIPs at most of the tables. On my table, there were 3, so to help the teams and scoring, I held a mini judges class before the judging began, and then again before each meat turn in. The non CBJs scoring was right in line with the CBJs on my table, +/- a point here and there.

The new score sheet is very revealing, judges are no longer anonymous, and names could be easily placed to tables and seat numbers by those in the judging area.

My advice would be to not use Holbrook as a baseline for the new scoring system because of the low CBJ percentage. Just for the record, even with only 5 tables, the Table Captains took the time to make sure no table judged the same teams entrys twice.

BMerrill
07-15-2013, 08:26 PM
I like the new scoring sheets. Gives us more information.
I cooked the Greenwood contest with a very good friend who is a CMJ.
In his opinion of the scores:
Chicken was about right.
Ribs and pork were way too low. They were very good entries just hit the dead tables.
Brisket was about right. We knew it was not going to place very high even before slicing.

Only time will tell if KCBS uses the results to improve judging.

gettinbasted
07-15-2013, 09:33 PM
As another candidate for table 5 at Wellsville I agree, it was a tough table at best. However, if we as cooks are held to a standard, then some how judges should be held to a standard of being fair in the overall scheme of things and if historically they are continually out of sync with other judges over time then they need to either be re-educated or politely taken aside and talk to about their quality standards. With that said the nearby judge or 2 time judge will still have issues but if the rep keeps them to one per table then it really doesn't matter as their potential low score will be dropped. Overall the new scoring sheet is a vast improvement...good job KCBS and everyone who took countless hours to make it happen!

There was also a "money" table at wellsville. 3 of the 4 category winners came off of table 8. I'm guessing we won't be arguing to have those judges reeducated!

Curling Q
07-15-2013, 10:25 PM
My understanding was this past weekend.

I think it is farking awesome. I learned more the past two days pouring over the score-sheets than I have the past five years. Thank you KCBS!:eusa_clap


I agree, been pouring over them too from the same contest. It is no secret good cooks still win that is why you kick our butt every time.:wink:

boogiesnap
07-15-2013, 10:46 PM
it may have been understood and i didn't know that i was wrong, but i always kinda thought if there was a tough or hot table or two all the teams would hit them once(except for the big events)so it would kinda average out over the four categories and it becomes kinda "fair".

vince and gettin basted clearly contradict that sentiment.

but, then again, i can't argue that the great teams don't almost always perform well. kinda contradicts the contradiction...

a paradox it is.

SaucyWench
07-15-2013, 10:57 PM
Really interested in how this is going to work. I thought Score was implemented to help KCBS track judges' scoring over the long run, not to tell teams how a particular judge at a particular table scored on average that particular day. Hmmm...

TailGateJoecom
07-15-2013, 11:02 PM
.....I also noticed a lot of us hit the same tables together(not the same table twice)and wondered if that was normal?..hitting tables with the same teams throughout all meats.

Are you consistently submitting your entry at the beginning or end of the submission window?

Also, I wonder, do the better teams normally turn in at the beginning, middle, or end of the turn-in time?

TailGateJoecom
07-15-2013, 11:09 PM
So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?

That is an interesting idea. If they could maybe have something where they could put a judge's historical average in and have it spit out table assignments where the average historical scoring ends up being about the same for each table, so that no one table is weighted with very high or very low scorers.

I would think this wouldn't be too difficult now that all this info is being recorded and tracked.

TailGateJoecom
07-15-2013, 11:19 PM
This is going to be a newby comment so bare with me, (I have never been in a competition yet, next month will be first) with out knowing what the "old" sheet was I see that the new one can help you/kcbs figure out which Judge you don't want to have or that needs more training. Since you don't get to choose your table and the same Judge's don't go to all of the comps, how does this "help" you as a cook to improve with out the comment cards? I can see knowing your score can tell you what area you were good or bad in though.

I am a newb too, having only competed in one contest. Here is my take on how this could have helped me in understanding where I am in my development.

I think my ribs were pretty good, I had 2 judges give me 999. I ended up coming in 28th i think out of 42 in ribs. Now, were my ribs really that good from the 2 judges going all 9's, or did those judges score everyone high? Did the judges who scored me low in ribs score everyone low? Now, if the high scoring judges didn't score everyone else high, and the low scoring judges pretty much scored everyone else low, then I can reasonably deduce that my ribs were likely pretty good and got the luck of a bad judge who scored everyone down.

Also, did anyone get a call in ribs from my table? What place did the highest place rib score on my table land overall? Was I actually the highest rib score of my table yet only came in 28th which leads me to think maybe I hit a low scoring table?

Now, as for my brisket, it was my second time EVER cooking brisket. I thought it was terrible. It looked ok, but overall, it was a bad entry. I came in 32 out of 42. A bad score, but I can't imagine how bad the 10 guys below me were. Maybe I lucked into a high scoring table, and I can deduce that my 32nd was a gift.

Like I said, i am a newb as well, and I am sure others can chime in with other ways to use these score sheets, but this is just one way I see its benefits. I would still like comment cards thought.

G$
07-15-2013, 11:31 PM
The more tables a contest has, the easier it is to make sure a team does not hit the same table more than once. Think about it, if a contest has 13 tables, only 3 of those had that team's 3 prior turn-ins (less than 25%). If there were only 4 tables, 75% would have had a team's 3 prior meats.

I also believe table captains are given sheets to mark off and check to make sure they don't get the same boxes twice. Maybe that only happens with certain KCBS reps, I'm not sure.

What you said and what I said are not mutually exclusive.

Also, and I am not going to do the statistical anlysis here, but in general one might want to be careful when pointing out tings like "GC never hit table 3, and table 3 was the TOD (table o' death)". There can be a causal relationship there, POTENTIALLY.

G$
07-15-2013, 11:33 PM
There was also a "money" table at wellsville. 3 of the 4 category winners came off of table 8. I'm guessing we won't be arguing to have those judges reeducated!

So.... If the best four turn ins happen to arrive at the same table, wouldn't we expect it to be, what you have termed a "money" table?

G$
07-15-2013, 11:35 PM
So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?

How many contests are needed to make that number statistically meaningful?

CBQ
07-16-2013, 01:21 AM
This is going to be a newby comment so bare with me, (I have never been in a competition yet, next month will be first) with out knowing what the "old" sheet was I see that the new one can help you/kcbs figure out which Judge you don't want to have or that needs more training. Since you don't get to choose your table and the same Judge's don't go to all of the comps, how does this "help" you as a cook to improve with out the comment cards? I can see knowing your score can tell you what area you were good or bad in though.

We don't really know what, if anything, KCBS will do with regard to using KCBScore to change judge behavior. Maybe nothing. What it DOES do is give the cooks a lot more information about their performance. Did you have a problem with your product, or did you hit an unlucky table? While the new system won't give you a conclusive answer, it gives you more insight into how you really did and what the judges might be thinking.

gettinbasted
07-16-2013, 07:40 AM
So.... If the best four turn ins happen to arrive at the same table, wouldn't we expect it to be, what you have termed a "money" table?

It is absolutely possible that the "money" table had the top entries in each category and that the "table of death" had terrible food across the board. At an 8 table contest, however, I would consider it unlikely. With only 5 or 6 entries hitting each table per category, any conclusion drawn can be dead wrong.

It is what it is. We are all aware that a good or bad table can exist when we turn in our food. That is BBQ, and as Boogiesnap pointed out, it will even out over the course of a contest and a season.

gettinbasted
07-16-2013, 08:06 AM
How many contests are needed to make that number statistically meaningful?

I don't advocate any kind of judge training program based on their score history. G$ makes the perfect point as to why. There just won't be enough data. Judge reeducation is a dangerous path to go down.

BBQ judging is subjective. I have received a 180 and a 160 on essentially the same piece of meat at two competitions. The truth was probably somewhere in the middle. I would love to have that 160 be a 170, but I don't want to give that 180 pin back!

BMerrill
07-16-2013, 08:29 AM
At Greenwood, SC, out of the top 10 overall teams, only Lotta Bull hit the same table twice.

musicmanryann
07-16-2013, 08:55 AM
I'm with Vince. At our contest this weekend there was clearly and bad table and a great table (from a cooks perspective ;-)). Fortunately for ours most, if not all, hit every table once. I have been saying for a couple of years now that sorting judges based on how they actually score, rather than on the number of contests they have competed would be a much more reliable method for equalizing the scoring across tables. You have judges that score high, score low, and score middle of the road no matter their experience level. Experience does matter a great deal, but there are still bad judges with a lot of experience and good judges with little experience.

It seems this new scoring system would open the door for tracking and sorting judges based on their variance from the mean. The question is whether or not the new system is currently keeping track of judges' scoring across multiple contests.

Muzzlebrake
07-16-2013, 09:34 AM
I'm going to use this data to help me make adjustments to things within my control. This past week for instance I know my ribs and pork tanked. I didn't really like either one but it didn't think they were as bad as they placed. Looking at the scores I can now see my ribs hit a low scoring table and the pork just tanked on its own merit.

I would have thought the judges scoring statistics would be very useful to them just as Tigerpaw mentioned.
I'm not sure that we need to seat judges according to their scoring averages to even out the tables though. In our contest this weekend, the top 10 overall I think 7 of the teams had hit the low scoring table at least once so it seems to have balanced out in the end. On the other hand, the GC didn't hit it once and won by 10 points, 2-4 all hit that table and were only separated by 5 points.

Who knows? I think more data is definitely need before any adjustments are made. The rest of the year should give you a good sample range to work with.

mobow
07-16-2013, 09:41 AM
You also have to keep in mind that the low table may be low or it may be accurate. The high table may be accurate or it may be inflated. More information can be good but it can also increase the odds of a faulty conclusion. Keith

Outnumbered
07-16-2013, 09:45 AM
I was a table captain at Holbrook, and I believe that the CBJ % was less than 65%. There were at least 3 VIPs at most of the tables. On my table, there were 3, so to help the teams and scoring, I held a mini judges class before the judging began, and then again before each meat turn in. The non CBJs scoring was right in line with the CBJs on my table, +/- a point here and there.

The new score sheet is very revealing, judges are no longer anonymous, and names could be easily placed to tables and seat numbers by those in the judging area.

My advice would be to not use Holbrook as a baseline for the new scoring system because of the low CBJ percentage. Just for the record, even with only 5 tables, the Table Captains took the time to make sure no table judged the same teams entrys twice.

I commend you for that and this is not a critique of how you handled. And thank you for handling it this way as it seems to be the best way to do it. Just using it to illustrate my point.

Assume that the non-CBJs score a point lower than the rest of the table for whatever reason. They throw the lowest out, so we can assume it's the low score, so you have two non-CBJ who score a point lower on across the board on taste from a purely arbitrary reason (at least theoretically here). You now have lowered the scores from that table by 5 points per category, per team. That's a 20 point swing for all 4 meats and can mean the difference between a GC and middle of the pack finish for overall.

Of course, the same can be said for the other extreme as well. Maybe you did a middle of the road cook, but you hit the right table and you mediocre stuff got GC.

But as gettin'basted says, this new system at least gives you the information to know if your entry was bad or if your luck was.

Thanks to KCBS for making this change. I think it's great.

Alexa RnQ
07-16-2013, 09:49 AM
You also have to keep in mind that the low table may be low or it may be accurate. The high table may be accurate or it may be inflated. More information can be good but it can also increase the odds of a faulty conclusion. Keith
I think tracking across all categories and an accumulated body of several contests will reduce the faulty conclusion. A judge may land on a lucky table that gets all great entries, or on a poor table that gets all scary entries, but the odds of that happening in all four categories in contest after contest over time are pretty remote.

bignburlyman
07-16-2013, 09:57 AM
With all the talk of now being able to see more data and more of the tables scores, and how this may help balance the judges better, no one has mentioned the 800 pound gorilla in the room---the judges will most likely never see any of the score sheets. How will this help any judge to realize they are too harsh or too generous unless KCBS taps them on the shoulder 5 contests down the line and say "Your scores do not fit the mean average of the other judges"?

There are a lot of great judges out there, and there are a few total dooshbags, most of my contests judged are in Kansas, and I know a couple of judges that I prefer to not sit at the same table with them. Now expand that couple of judges nationwide and there really are quite a few that are not that great. Someone once told me to remember that even though medical doctors are mostly very well respected, half of them graduated in the bottom 50% of their class.

Slamdunkpro
07-16-2013, 10:25 AM
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.

gettinbasted
07-16-2013, 10:31 AM
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.

Agreed. Just another thing to discuss over a beer after awards.

Muzzlebrake
07-16-2013, 10:45 AM
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that's from awards since if you consistently get scores over the judge's mean average you should be getting walks.

So you don't see any value in knowing that the product you turned in scored where it did due to it hitting either a good/hot table or bad/cold table. I think everyone has had that turn in they thought was great that tanked or sucked and they walked and wondered why. Granted I don't think there is much you can do about it but I see value in at least knowing that.

Slamdunkpro
07-16-2013, 11:00 AM
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.

So you don't see any value in knowing that the product you turned in scored where it did due to it hitting either a good/hot table or bad/cold table. I think everyone has had that turn in they thought was great that tanked or sucked and they walked and wondered why. Granted I don't think there is much you can do about it but I see value in at least knowing that.

There is no value because (1) you don't know the circumstances of those tables and (2) you'll most likely never encounter that set of variables (judges at that table, other entries at that table) again. Back to the roulette example. The odds of any one number coming up are 37-1; if 18 comes up 6 times in a row what are the odds it will come up 18 the next spin? 37-1. The odds always remain the same because they reset every spin. Your contest history within the aspect of judges and tables and all variables (N) is always N=1. It resets every event.

Balls Casten
07-16-2013, 11:05 AM
So you don't see any value in knowing that the product you turned in scored where it did due to it hitting either a good/hot table or bad/cold table. I think everyone has had that turn in they thought was great that tanked or sucked and they walked and wondered why. Granted I don't think there is much you can do about it but I see value in at least knowing that.

I would disagree ... I think it tells you what kind of food was at that table. All of these remarks are being made without seeing or tasting the entries.

.. and again I think the new scoring is good. I am just afraid of how it will be used to single out judges and influence their scoring.

jaestar
07-16-2013, 11:39 AM
I would have left the Sam's Club contest last week wondering what went wrong with my brisket even though I cooked a good brisket. Based on past contest I figured a top 5 finish (my average in brisket this year is 168 over 14 contests) as this was one of the better briskets I have cooked this year, but finished 20th OA.

The new sheets showed that my brisket was at table 2. Table 2 had two total top 10 finishes. A 7th in ribs by Pig Skin and an 8th in pork. I talked to Scott before awards and he said he was happy with his ribs...which usually is a bad thing for the rest of us :razz:

At the brisket table I was with Tippy Canoe. Joe said that he cooked a good brisket too but finished 15th OA.

Table 4 had the following rankings (29 teams):

Chicken: 21, 22, 25, 26, 28, 29
Ribs: 7, 21, 23, 27, 29
Pork: 8, 15, 22, 26, 27, 28
Brisket: 15, 20, 26, 27, 28, 29

Now let's play this out another contest or two and say you have the bad luck of hitting the bad table. In the past, I would not have known that. I would probably have started to mess around with my recipe and then waste a couple of more contests trying to correct something that wasn't wrong in the first place.

Smokin' Joe
07-16-2013, 11:48 AM
In the past, I would not have known that. I would probably have started to mess around with my recipe and then waste a couple of more contests trying to correct something that wasn't wrong in the first place.

This is spot on in my opinion and the reason that I think this system is a huge improvement.

As jaestar pointed out at the Woodbury MN event of the bottom 4 placings in each category 12 of 16 came off of table 2. Thats either a trend or a very scary coincidence. (for the record, I'm not complaining about judges, I'm just saying I enjoy having a few additional facts to base any changes or feedback on)

Balls Casten
07-16-2013, 12:24 PM
I would have left the Sam's Club contest last week wondering what went wrong with my brisket even though I cooked a good brisket. Based on past contest I figured a top 5 finish (my average in brisket this year is 168 over 14 contests) as this was one of the better briskets I have cooked this year, but finished 20th OA.

The new sheets showed that my brisket was at table 2. Table 2 had two total top 10 finishes. A 7th in ribs by Pig Skin and an 8th in pork. I talked to Scott before awards and he said he was happy with his ribs...which usually is a bad thing for the rest of us :razz:

At the brisket table I was with Tippy Canoe. Joe said that he cooked a good brisket too but finished 15th OA.

Table 4 had the following rankings (29 teams):

Chicken: 21, 22, 25, 26, 28, 29
Ribs: 7, 21, 23, 27, 29
Pork: 8, 15, 22, 26, 27, 28
Brisket: 15, 20, 26, 27, 28, 29

Now let's play this out another contest or two and say you have the bad luck of hitting the bad table. In the past, I would not have known that. I would probably have started to mess around with my recipe and then waste a couple of more contests trying to correct something that wasn't wrong in the first place.

Come on now ... would you really have "come away wondering what went wrong" "started to mess around with my recipe"? OR would you have chalked it up to a bad table and made plans for Rockford.

(poking the bear) You finished in 6th place do you think you should have finished higher?

Balls Casten
07-16-2013, 12:28 PM
I'm just saying I enjoy having a few additional facts to base any changes or feedback on

I agree if thats all its used for.

But what if some of us dont want you guys having more info? Maybe this additional info should only go to teams that have not won a contest. :mrgreen:

ique
07-16-2013, 12:37 PM
Sorry havent read the entire thread but...

The next thing to add for each judge is "scoring deviation".

Take the average score of the other 5 judges and compare it judge 6 score. Judge6 ScoreDev could be +7% or -5% for example. After say 10 contests judged we can identify judges that score much higher or much lower than average and try to address it. I know at least a couple judges in NEBS land with a ScoreDev of -35% :wink:

Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables

Smokin' Joe
07-16-2013, 12:42 PM
Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables
This should happen!

Untraceable
07-16-2013, 12:50 PM
would it be easier to handicap tables at the end?

nevermind. if every table was even, there would be even more .0002 spreads

musicmanryann
07-16-2013, 01:52 PM
Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables

My thinking as well. Once the last judge checks in the rep clicks a button and the computer assigns judges to each table virtually instantly. Again, I'm not sure if the new program is keeping track of scoring across multiple contests or not, but if it is this would seem relatively simple.

jaestar
07-16-2013, 02:07 PM
Come on now ... would you really have "come away wondering what went wrong" "started to mess around with my recipe"? OR would you have chalked it up to a bad table and made plans for Rockford.

(poking the bear) You finished in 6th place do you think you should have finished higher?


I would have wondered what went wrong. I would have been 90% sure it was a bad table, but I would have still wondered if I really did cook as good of a brisket as I thought I did. After two or three bad contests I would start looking at making changes. I don't know statistically what the odds would be of hitting the bad table in three straight contests, but at least now it can be confirmed.

I don't think I should have finished higher or lower as 4/5 teams above me also hit table 2 and it is all part of the "luck of the draw". I'm not complaining about coming in 6th (actually really happy I did :grin:) or complaining about the judges. The fact is that table 2 was a low scoring table at that contest and hitting that table or not hitting it could have drastically changed the outcome for teams.

Kit R
07-16-2013, 02:33 PM
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.

It's perhaps useful if you want to convince yourself your bad day was the result of hitting "bad" tables and that a successful team had a good day because they hit "good" tables. A single day's i.e. comp's worth of results are not enough to determine if you were unlucky, the other guy was lucky or maybe you just didn't cook as well as you thought. Anyway, I can't influence what table I end up on, right? And even if I could how would I know which is good or bad?

After I get about 20 of these news scoresheets in hand, maybe I'll take a look and see if I see any "trends". But it's kind of like the time I had this ginormous 119 cut brisket and the fat on the deckle looked eerily like a face. Just because it looked like a face didn't mean there was actually a face there. Just because I think I see any trends doesn't mean there is anything of any real significance there. The only "trend" I am going to worry about is how I finish in each category and the overall. If I'm doing OK then I'm happy. If I start stinking it up, then I'll look at making changes. Really, it's all about how you finish on that day and in those circumstances. Once that set of entries, those judges and that set of competitors is dispersed it's back to zero.

WineMaster
07-16-2013, 02:41 PM
I agree if thats all its used for.

But what if some of us dont want you guys having more info? Maybe this additional info should only go to teams that have not won a contest. :mrgreen:

Thats the beauty of it though. Everyone comes away with the same info.

Vince RnQ
07-16-2013, 04:38 PM
How many contests are needed to make that number statistically meaningful?

Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.

Vince RnQ
07-16-2013, 04:43 PM
I don't advocate any kind of judge training program based on their score history. G$ makes the perfect point as to why. There just won't be enough data. Judge reeducation is a dangerous path to go down.

BBQ judging is subjective. I have received a 180 and a 160 on essentially the same piece of meat at two competitions. The truth was probably somewhere in the middle. I would love to have that 160 be a 170, but I don't want to give that 180 pin back!

Respectfully, I disagree. Reeducation is hardly dangerous. If someone is doing something that is way outside the norm, it could very well be due to a lack of understanding and reeducation can easily and quickly solve that problem. Remember, we're not talking about reeducating judges that fall within a statistical norm but rather those who fall well outside of what could be considered a normal variance. A judge who regularly scores everything high or everything low is not really judging anyway.

The_Kapn
07-16-2013, 05:10 PM
I have not looked at the new KCBS scoresheet in detail, but it sounds like what FBA has provided for years.
When we competed, we found the data useful, but only in the "macro" sense.

As many have said--if we wound up on a low scoring table, we looked at our numbers as such--took it with a grain of salt.

If we got great numbers, but so did every one else--we took as a blessing.

If our table killed us, but not the others, we looked at working on our cooking.

What is interesting is to track the individual judge scores.
At one event, we got nailed with a 10-7.5-7.5 from one judge. Totally out of whack with the other 10's and 9.5's. Cost us some serious $$.

So, we looked at the other scores from this judge and they were all--100%--10-7.5-7.5 except for one entry that got a perfect 10-10-10 from all judges.
Having judged a bit, I know exactly what happened. This Dude or Dudette knew that if they gave 10's on appearance, the Reps could not fault them.
When the super entry came in, the other, shall I call them "seasoned judges", clearly showed their "awe" and "Mr 10-7.5-7.5" figured he/she needed to score up. 8)

We retired from competing for many reasons, this was just a piece of the puzzle.

That info should be available to you KCBS folks now.

Good Luck y'all!!

TIM

Hawg Father of Seoul
07-16-2013, 05:14 PM
Or maybe even better write a judge seating program that mixes together judges so high scoring and low scoring judges are evenly distributed across the tables

This should happen!

My thinking as well. Once the last judge checks in the rep clicks a button and the computer assigns judges to each table virtually instantly. Again, I'm not sure if the new program is keeping track of scoring across multiple contests or not, but if it is this would seem relatively simple.

I know that this sounds like a good common sense approach, but it is a bad idea statistically speaking. I call this "inverse statistical regression". If you understand statistical regression, you can explain how I am wrong AND I'll read your response. It may also introduce bbq to the criterion problem.

The issue would be compounded if the judges knew where their average scores fell.

Some how I bet no one will heed this warning. It something has face validity, to hell with construct validity.

boogiesnap
07-16-2013, 05:25 PM
i can only imagine that because the statistics are manipulated and not random they are no longer valid. and now that they are no longer valid, they are no longer useful because they were manipulated.

and once you force a smoothing of data, well, we'll probably all get sevens every category, every entry, kinda thing.

was i close????:becky:

sdbbq1234
07-16-2013, 05:52 PM
Ok, so, why did KCBS institute this new system of delivering the results to the competitors? Was it for us to try and micro manage our cooking? Was it for them to track the judges?

I am only hoping it was for them to track the judges.

But then, what are they going to do about it?

Really, why provide the competitors this more in depth information?

wallace

Slamdunkpro
07-16-2013, 05:57 PM
Really, why provide the competitors this more in depth information?

wallace
Drive forum & FB traffic?

G$
07-16-2013, 06:13 PM
Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.

Actually, I am not sure I disagree, but we must be careful not to give inordinate importance to a sample size of ... one. Honestly, I am chewing on the ramifications associated with seating of judges based on historical averages, and I don't think I know what my opinion is on it yet.

G$
07-16-2013, 06:19 PM
There is no value because (1) you don't know the circumstances of those tables and (2) you'll most likely never encounter that set of variables (judges at that table, other entries at that table) again. Back to the roulette example. The odds of any one number coming up are 37-1; if 18 comes up 6 times in a row what are the odds it will come up 18 the next spin? 37-1. The odds always remain the same because they reset every spin. Your contest history within the aspect of judges and tables and all variables (N) is always N=1. It resets every event.

The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.

sdbbq1234
07-16-2013, 06:20 PM
Obviously the more data, the better the representation but I would be all for using that method of table balancing immediately. As the data set grows, the balance will be better and better.

If you have a different opinion, I would like to hear it.

Good idea, but I believe it would be an administrative nightmare for the reps, at the present time as you point out. Maybe down the road after this new system has had time to settle in, and the reps programs on the laptops at the comps could/would accommodate this, I agree.

wallace

Slamdunkpro
07-16-2013, 06:39 PM
The potential diffence, SDP, is that fair roulette spins or dice throws on the level are, as you know, independant events. Deals from Blackjack shoes are not....

Notice I said potential, because, the constant in the equation is the judge, the variable is all the things you mentioned and implied: entries, other judges, weather, a cold, etc etc.

Many here would argue that the past reults from the constant (the individual judge and his score) is enough data to use to predict what the future outcome is more likely to be.
Blackjack shoes have a known history and it's possible to predict the outcome with ever increasing accuracy as N approaches zero. the same would be true if your turn in's hit the same table of judges week after week after week. They don't.

The constant in the equation is actually the food as most of us vary our recipes very little and have gotten reasonably successful at producing a constant product from week to week. The main variable is the judge.

kellym
07-16-2013, 06:42 PM
So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?
That's hard to say Vince. I know the experience level sort helps a lot but it's not the perfect answer. You know how obsessive I am about my sort and I've modified it since I've seen you and it's even more granular. I did a poll of the cumulative number of contests judged for the judges at each table. The lowest table had a total of 101 contests and the highest table had 113 contests. This gave a 12% variance between the highest and lowest tables which is not a bad sort. I still had one tough table where the highest score was a 10th place chicken. I had one moderately warm table but nothing I would consider super hot. I've had contests where there were no hot or cold tables using this method.
Sorting based on historical average could be interesting. It would have to be tested as I don't believe a model can be built to test this because of the human factor but someone may prove me wrong. I don't know if I would rather see this or have some re-training for the judges that are consistently off target on their judging.
Since I don't have access to historical data I guess it doesn't matter anyway. lol
Thanks Vince.

billygbob
07-16-2013, 07:32 PM
While I believe all cook teams should have members that judge at least on occasion, I have found over the years as a judge that they are the toughest/lowest scorers. Reps should account for who are mostly cook team members at contests and distribute them as they would VIPs or newbies.

SaucyWench
07-16-2013, 07:42 PM
I'm wondering if there is renumbering of tables and judge seats on the printouts for anonymity, like there is with team entry boxes, so we keep the true double-blind. At comps that have tents for judging, it wouldn't be too hard for someone to at least figure out the table numbers. I'd hate to lose one of my cook friends because they figured out my table and seat number and my score didn't agree with what they thought it should be.

And I wonder how KCBS is going to re-educate outliers. So far, continuing education consists of one on-line test which I easily ace. Tracking won't do much good if they don't have a plan for what's next.

boogiesnap
07-16-2013, 07:56 PM
the problem i'm having with sorting through historical data is we would essentially be removing that "i didn't like it" spike in scoring.

clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.

i think it may do a real disservice to the competitors and the sport to start sorting judges.

sdbbq1234
07-16-2013, 08:09 PM
[QUOTE=boogiesnap;2554257] clearly the most skilled teams have overcome that with their cooking and enjoy success, and those that haven't aspire to it, learn more, cook better, and hopefully enjoy the journey.[/ QUOTE]

+1. That goes for me.

wallace

Andrews
07-16-2013, 08:44 PM
One variable that I don't think has been brought up is the celebrity judges.
To me, that is a problem, you can re-educate existing judges all you want, how do you factor in a high percentage of judges who are judging their first contest whith no historical data on how they might score entries?

Muzzlebrake
07-16-2013, 08:58 PM
Admittedly I am not the mathematician that many of you are, I understand and agree with you that there is not enough of a constant in the equation to utilize the new data to to predict a statistical probability of future outcomes. The new data can however provide and excellent analytical review of an event that has already taken place and I think that is where it's value is.

The new scoring system will allow me to see for example that over the course of the 24 entries judged by table x only 1 top ten score in any category. Understandably, I don't get to sample any of the food but I can further determine that out of the 24 entries judged 3 were submitted by current or former Jack/AR champs, 10 were from teams currently in the Top 50 in there respective category, 1 was a Sam's regional champion and 5 others were from 3 teams with 11 combined GC's this year. Looking at the quality of the teams submitting these entries, I think it is very reasonable to think those results are the more likely the result of a scoring anomaly than deviation of your process/recipe. Now I can see that the brisket I way overcooked but finished 2nd with was more than likely due to me hitting the table that put 14 entries into the top ten from 11 teams that have never had top 10 than judges suddenly liking brisket cooked to 215*.

ModelMaker
07-17-2013, 08:14 AM
Here's a thought, how about they make a nine table and a 8 table and so on and you just take your chance on where you land?
Then you guys can figure the statistical chances of which table you would hit based on the order you hit the turn in table and throw in the odds of which hand you use to carry your box?
Go for it.
Ed

musicmanryann
07-17-2013, 08:27 AM
i can only imagine that because the statistics are manipulated and not random they are no longer valid. and now that they are no longer valid, they are no longer useful because they were manipulated.

and once you force a smoothing of data, well, we'll probably all get sevens every category, every entry, kinda thing.

was i close????:becky:

Correct, but currently the statistics are not random and are being manipulated. Reps currently sort the judges and seat them based on their experience under the assumption that experience is a factor in how a judge scores.

We are not talking about normalizing how an individual judge scores, but rather the table as a whole, and across all the tables at a contest. You may still the straight 777s, but it would more likely be from one judge at all four tables you hit, rather than one table with four judges killing you. :mrgreen:

Hawg Father of Seoul
07-17-2013, 09:09 AM
Here's a thought, how about they make a nine table and a 8 table and so on and you just take your chance on where you land?
Then you guys can figure the statistical chances of which table you would hit based on the order you hit the turn in table and throw in the odds of which hand you use to carry your box?
Go for it.
Ed

Do you have such a problem with the way we do it now? I personally think that the way it is currently set up, as the number of contests you cook increases the closer you get to your true ranking.

boogiesnap
07-17-2013, 09:11 AM
Correct, but currently the statistics are not random and are being manipulated. Reps currently sort the judges and seat them based on their experience under the assumption that experience is a factor in how a judge scores.

We are not talking about normalizing how an individual judge scores, but rather the table as a whole, and across all the tables at a contest. You may still the straight 777s, but it would more likely be from one judge at all four tables you hit, rather than one table with four judges killing you. :mrgreen:

understood, but that is sorting by person. using historical data the reps would be sorting by numbers.

quite different IMHO.

doesn't really make a difference though probably because there is always the human factor. however, if you balance the tables by the numbers, it is possible, over time, every entry in every category, from every table COULD start getting the exact same totals.

musicmanryann
07-17-2013, 09:47 AM
understood, but that is sorting by person. using historical data the reps would be sorting by numbers.

quite different IMHO.

doesn't really make a difference though probably because there is always the human factor. however, if you balance the tables by the numbers, it is possible, over time, every entry in every category, from every table COULD start getting the exact same totals.

It would be quite different if they sorted by age, eye color or their birth month. However, the sorting by person is done under the premise that their experience determines how they score. Individuals' scoring trends are still there in the sorting even though the numbers are not necessarily known. On the other hand, I know Reps who have a good idea who the black sheep are when they regularly attend contests, and more than a couple of organizers who blacklist judges who consistently score above or below the rest of the table. So perhaps more than a little is known, but probably not consistent.

I do believe there is a correlation between scoring and experience, and am thankful for the reps who go to great lengths to ensure tables are sorted thoroughly based on this variable. Hopefully, as data is collected from the new scoring system, light will be shed on the nature of the correlation, and yield more information that may help refine the table sorting process. In the end we all want the best food, and not luck to win the day.

Kit R
07-17-2013, 10:06 AM
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).

Balls Casten
07-17-2013, 10:09 AM
The sorting of judges by experience is a logical act. No one wants 6 first time judges at one table.

Kit R
07-17-2013, 10:12 AM
The sorting of judges by experience is a logical act. No one wants 6 first time judges at one table.

Even if they gave all 9s? :decision:

sdbbq1234
07-17-2013, 10:49 AM
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).

Now there's an idea worth serious consideration!:clap:

wallace

Hawg Father of Seoul
07-17-2013, 01:40 PM
I had a thought. This is an honest, and non-sarcastic, proposal. What if judges took a number from a hat from 1 to x (with X being the total number of tables) as they entered the tent and they were seated at the number of the table they pulled? No additional sorting by experience, no breaking up spouses, etc. Then the distribution of the judges WOULD be truly random with zero chance of manipulation, smoothing of data, etc. The luck of the draw might result in a table from H-E-Double Hockey Sticks, and it might create a really favorable table, but over time it'd probably even out. Assuming there were enough tables, if reps made sure teams didn't hit a table more than once it would be random luck whether or not you hit a super or awful table. This is assuming there really is such a thing as good or bad tables as opposed to it just looking that way when you persuse the data of a single comp. My theory is this is pretty much what happens already, but maybe if you take any manipulation out of the equation it does become 100% random. If KCBS wanted they could still use the detailed scoresheets to identify outlier judges and take whatever action deemed appropriate (counseling, reeducation, exile to the salt mines of Urumqi China or complimentary lifetime membership in the American Vegan Society).

I think you summed it up well. You can manipulate the judges or the tables, but when you do both you fark the goose.

G$
07-17-2013, 02:02 PM
Then the distribution of the judges WOULD be truly random

You answered it yourself.

Many competitors do EVERYTHING THEY CAN to remove any trace of randomness to their whole process. THEY DO NOT WANT RANDOM JUDGING. They want what they consider fair and equal judging.

For the record, I am not personally saying this is my point of view, but it is a point of view of many very excellent competition barbeque teams.

boogiesnap
07-17-2013, 04:14 PM
You answered it yourself.

Many competitors do EVERYTHING THEY CAN to remove any trace of randomness to their whole process. THEY DO NOT WANT RANDOM JUDGING. They want what they consider fair and equal judging.

For the record, I am not personally saying this is my point of view, but it is a point of view of many very excellent competition barbeque teams.

yes, fair and equal per judge. we're talking per table, know what i mean?

the use of random in this thread isn't what your'e thinking.

G$
07-17-2013, 04:51 PM
yes, fair and equal per judge. we're talking per table, know what i mean?

the use of random in this thread isn't what your'e thinking.


You keep using that word. I do not think it means what you think it means.

I know very well what the folks that want what they would term "fair judge seating" (and as a result "fair turn in distribution") based on average scores means, and there is nothing random about it. Point blank, they don't want random. And maybe that is best, but it is not random.

boogiesnap
07-17-2013, 04:56 PM
fair enough.

Balls Casten
07-17-2013, 10:38 PM
Just throwing this out there .... Distribute all the "low" scoring judges and all the "low" scoring judges get thrown out? ?????

Smokinclone
07-18-2013, 08:50 AM
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????

musicmanryann
07-18-2013, 09:24 AM
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????

That is a good point, Darren. Seems it would be best to not include such outliers in the analysis. If the score contains a score less than or equal to 1, it would receive a NULL value in the database. Or we use the median value of scores, rather than average to compensate.

Vince RnQ
07-18-2013, 09:26 AM
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????

First of all, congrats on your GC in Holbrook! Sorry that we didn't see you in person after the awards ceremony. I also want to say that my analysis of the score sheet for that event in no way reflects negatively on your team's performance. You were in the lead after every category and I think, based upon your brisket score, that even if yours and the Hill's briskets had both missed Table 4 that you would still have taken GC but that the margin of victory would have been narrower. Regardless, great work, great cooking and a great win!

I think you bring up a good point about how to factor DQs into the equation but I also think that DQs will represent such a small percentage of judge's historical average so as not to require much weight. That will, of course, remain to be seen.

As for Holbrook, I only see one DQ on the score sheet and that was in the Brisket category as a late entry. That entry was assigned to Table 4. While the KCBS scoring average did include that DQ in the judge's contest scoring average, I did not include it when I ran the placing averages for that table since clearly a late entry has nothing to do with the judges. Regardless, Table 4 accounted for the last non-DQ placing in every category and for 10 of the 20 lowest non-DQ placings across the categories.

musicmanryann
07-18-2013, 09:33 AM
What's with table 4?? I think at least three of the contests people are talking about here, table 4 was the low table. Coincidence???:mrgreen:

Vince RnQ
07-18-2013, 09:34 AM
It's a conspiracy, I tell ya!

big matt
07-18-2013, 11:26 AM
Here is the one thing that is missing when we start throwing average scores out. In Holbrook, AZ there were 3 DQ's this weekend, 2 raw chickens and 1 foreign object. Not sure what tables they hit since I don't have my score sheet in front of me. Those scores are included in the averages which skew the data. A judge has no control over that DQ and what score he gives it. Should the new system not include that score when figuring the averages only?? Otherwise if you're a judge and go to every contest and your table gets a DQ won't it skew your judge average down and make the judge look like he's scoring low when really he has no control over that DQ, they are just following the rules???????

I guess this thread is proof of what you said would happen.

Smokin' Joe
07-18-2013, 12:35 PM
I'll admit that all the statistics stuff is over my head, but I don't understand how random is better than applying what we know about a judges history? I look at it this way -

1) if it gives me more information about why my stuff scored the way it did, thats a good thing (perfect? no, but I believe more is better).

2) I believe some people arent qualified to be KCBS CBJ's - If this can help indentify them for re-training/elimination, then I support it. This isn't about all judges giving everyone a 9, its about the few judges who consistenly score 3-4 full points under everyone else. There arent millions of them, but they do exist

I understand point 2 might not be popular and could get a little scary with the wrong implemention, but I think its necessary for the continued growth of KCBS.

big matt
07-18-2013, 12:38 PM
I'll admit that all the statistics stuff is over my head. I look at it this way -

1) if it gives me more information about why my stuff scored the way it did, thats a good thing (perfect? no, but I believe more is better).

2) I believe some people arent qualified to be KCBS CBJ's - If this can help indentify them for re-training/elimination, then I support it. This isn't about all judges giving everyone a 9, its about the few judges who consistenly score 3-4 full points under everyone else. There arent millions of them, but they do exist

I feel exactly the same way Joe.

Slamdunkpro
07-18-2013, 01:17 PM
I'll admit that all the statistics stuff is over my head, but I don't understand how random is better than applying what we know about a judges history? I look at it this way -

1) if it gives me more information about why my stuff scored the way it did, thats a good thing (perfect? no, but I believe more is better).

2) I believe some people arent qualified to be KCBS CBJ's - If this can help indentify them for re-training/elimination, then I support it. This isn't about all judges giving everyone a 9, its about the few judges who consistenly score 3-4 full points under everyone else. There arent millions of them, but they do exist

I understand point 2 might not be popular and could get a little scary with the wrong implemention, but I think its necessary for the continued growth of KCBS.
1. I agree if the information is valid. If it's just the illusion of information (again think roulette history boards) then acting on it may be detrimental.
2. The KCBS position seems to be that if the check clears that person is qualified to be a judge. Again the age old question - How many people have failed the CBJ class?

What about the few judges that constantly score 2 to 3 points higher than the rest of the table? Will they be retrained? I can see the future and it's score cards full of 8,8,8.:wacko:

Smokin' Joe
07-18-2013, 01:23 PM
What about the few judges that constantly score 2 to 3 points higher than the rest of the table? Will they be retrained? I can see the future and it's score cards full of 8,8,8.:wacko:

No, I like those judges so they can stay :shock: :mrgreen: In all seriousness, yes they should also be retrained or removed. Again, implementation is key here. Otherwise the movement to 888 in a real concern

musicmanryann
07-18-2013, 01:43 PM
Everybody scoring 888's would be a concern if you looked at a particular judge's average score across contests. If you compared their score against the average of the other 5 judges at their table every contest, this wouldn't be an issue. This is what ique mentioned earlier in the discussion. A judge could be indexed on how they deviate from the other judges at their table for each entry.

TheJackal
07-18-2013, 02:01 PM
How is Average Judge Score on the sheet defined? I thought it would be that judge's average score for the 6 entries in that category only. However, I did the math from my chicken table form last weekend and it doesn't add up. Is it that judge's average for the entire contest across all 24 samples?

Candy Sue
07-18-2013, 02:13 PM
I asked for clarification on Judge Average Score.

It's the sum of that judge's scoring on that day at that contest for all 4 categories.

musicmanryann
07-18-2013, 02:13 PM
How is Average Judge Score on the sheet defined? I thought it would be that judge's average score for the 6 entries in that category only. However, I did the math from my chicken table form last weekend and it doesn't add up. Is it that judge's average for the entire contest across all 24 samples?

Yessir!

TheJackal
07-18-2013, 02:25 PM
Isn't that comparing apples to oranges (or chicken to ribs?!?)? That may help with tracking judges but it is slightly less meaningful to me as a competitor when analyzing my own scores then.

kellym
07-18-2013, 02:50 PM
I'll admit that all the statistics stuff is over my head, but I don't understand how random is better than applying what we know about a judges history? I look at it this way -

1) if it gives me more information about why my stuff scored the way it did, thats a good thing (perfect? no, but I believe more is better).

2) I believe some people arent qualified to be KCBS CBJ's - If this can help indentify them for re-training/elimination, then I support it. This isn't about all judges giving everyone a 9, its about the few judges who consistenly score 3-4 full points under everyone else. There arent millions of them, but they do exist

I understand point 2 might not be popular and could get a little scary with the wrong implemention, but I think its necessary for the continued growth of KCBS.
You also have the judges that never score under a 7. I actually had a CBJ tell me he gives a 7 just for turning in because he doesn't want to disrespect the team. Fortunately he wasn't judging (and never will) at one of my contests. No amount of discussion would change his mind either.

kellym
07-18-2013, 03:02 PM
Everyone should be aware of how the team name is recorded. The team name you supply to the organizer is what will print on your results sheet. The team name associated with your KCBS membership number that you supplied to the organizer is what will display in the online results screen. If the wrong team name is displayed online contact Karrin in the KCBS office. She can change your team name and correct the online results to show the proper name. Her email is kmurphy@kcbs.us

Pappy Q
07-18-2013, 06:21 PM
Damn, I knew I should've paid closer attention in that college statistic class 35 years ago

Muzzlebrake
07-18-2013, 07:43 PM
You guys have convinced me, statistical analysis has no place in sports. And to think I have been wasting my time all these years paying attention to things like batting averages, ERA, yards per rush/catch, completion percentage, goals against average........

sdbbq1234
07-18-2013, 07:52 PM
You guys have convinced me, statistical analysis has no place in sports. And to think I have been wasting my time all these years paying attention to things like batting averages, ERA, yards per rush/catch, completion percentage, goals against average........

On any given Sunday.......... :doh:

wallace

Candy Sue
07-19-2013, 08:59 AM
I asked for clarification on Judge Average Score.

It's the sum of that judge's scoring on that day at that contest for all 4 categories.

I'm wrong! The average is for all categories at that event that that judge judged.

Tiki Wolf
07-19-2013, 09:13 AM
I've read through most of this, and as a competitor, yeah, I'd like to see who I went up against on a table and how I did. But if I get a 6 or a 7, it's still only a number and doesn't tell my why it was a 6 or 7. Was it bland, or was it too spicy, salty, sweet, tangy, overdone, underdone? And if it's a 9, what was it that made it a 9?

I'd much rather see mandatory comment cards for each entry. That's the valuable feedback I'm looking for. I know that would be a logistical nightmare, the judging would take forever and the awards wouldn't be until after midnight.

Just my $0.02. I'll go back to sleep now....

TheJackal
07-19-2013, 09:47 AM
I'm wrong! The average is for all categories at that event that that judge judged.

Wow. So if that judge also sampled a chef's choice, sauce, and dessert entries they would be included in the average? :shock:

Candy Sue
07-19-2013, 10:38 AM
That's correct! Brings up another statistical thing though, how many contests will have ancillaries that are judged by the same judges?

CaptTable
07-19-2013, 11:08 AM
Candy,

Wouldn't it work better if we just don't use judge's numbers for ancillaries. that way it wouldn't skew anything. Just put 0's like you would for celebrity judges.

Candy Sue
07-19-2013, 01:02 PM
Wouldn't that be hard for reps if the contest is set up and dessert follows the main 4? I suspect that ancillaries probably aren't part of same judge seating for most contests. I like the idea though.

CaptTable
07-19-2013, 02:13 PM
I believe as long as you mark these categories as "Do Not Count" you will be able to omit judge's numbers. You migght check for us. That would solve that problem.

billygbob
07-19-2013, 08:06 PM
I'm wrong! The average is for all categories at that event that that judge judged.

How can the KCBS President not know how the new scoring system works? Should not every member of the Board know all details of how it works and be able to justify their decision to have it work that way , or how they disagreed with the implementation?

Amazing!

So sad....:cry:

YankeeBBQ
07-19-2013, 09:04 PM
How can the KCBS President not know how the new scoring system works? Should not every member of the Board know all details of how it works and be able to justify their decision to have it work that way , or how they disagreed with the implementation?

Amazing!

So sad....:cry:

Do you think the board of directors of any major company knows the complete details of how their products work ? They have technical people for that. I worked at a biomedical research facility for 18 years. We had technical, political and business people on our board of directors. The board is there to make major decisions and steer the direction of the company or organization. It's not the boards job to know the detail of everything going on. It's the boards job to make sure the company has people in place to do that.

billygbob
07-19-2013, 09:51 PM
Do you think the board of directors of any major company knows the complete details of how their products work ? They have technical people for that. I worked at a biomedical research facility for 18 years. We had technical, political and business people on our board of directors. The board is there to make major decisions and steer the direction of the company or organization. It's not the boards job to know the detail of everything going on. It's the boards job to make sure the company has people in place to do that.

You have a point for a major company. However, KCBS is NOT a major company with hundreds or thousands of employees, multiple divisions, and geographically dispersed business units. The board has chosen to release a new scoring system impacting all cooks and judges without guides or documentation. The KCBS BOD is closely involved with the most minute aspects of operations (read any BOD meeting minutes) so I would think any of them should understand how contests are scored by SCORE. It is the biggest change in KCBS contests in a decade (since "all entries start with a '999'"). How are cooks and judges supposed to understand SCORE when the BOD doesn't.

Big Poppa
07-19-2013, 10:40 PM
Candy Sue is president...she does not get paid...she volunteers for all of us in bbq. I for one am happy she has posted here....

Hawg Father of Seoul
07-19-2013, 10:53 PM
How can the KCBS President not know how the new scoring system works? Should not every member of the Board know all details of how it works and be able to justify their decision to have it work that way , or how they disagreed with the implementation?

Amazing!

So sad....:cry:

I dislike these statements.

Thank you Candy Sue. You are MUCH more transparent than we deserve and for that I say BRAVO!!!

Muzzlebrake
07-20-2013, 12:09 AM
You have a point for a major company. However, KCBS is NOT a major company with hundreds or thousands of employees, multiple divisions, and geographically dispersed business units.


It isn't? I think it meets the majority if not all of the items you mentioned. Perhaps not hundreds or thousand but certainly scores of employees spread out over multiple divisions across not just the US cut internationally as well.

CivilWarBBQ
07-20-2013, 01:36 AM
Tracking judges is not a tool for cooks.

It should be used by KCBS to detect when retraining is needed and perhaps a tool for organizers to use in selecting judges.

Alexa RnQ
07-20-2013, 08:12 AM
If the persons who claim to understand the needs of competitors admittedly don't understand the ramifications of their new system that so directly and vitally impacts competitors....

Fat Freddy
07-21-2013, 08:53 AM
Had my first experience with the new score sheets at this weekends Sams club contest in Des Moines. I had teamed up with fellow brethren LeatherheadIowa. I did chicken and ribs while he did big meats.

At awards we got a 9th place call in ribs so I was pleased but after I got a chance to look over new scoresheet, I think I am even more happy with our 11th place finish in chicken.

We were on table 1. We won the table in chicken. But in all 4 categories combined there was not a single top 10 call in any category. The very best category finish in all 4 meats was my 11th place chicken. I think table 1 second best finisher was 14th in pork and down from there.

I was happy with my scores so definitely not complaining but it sure seems that table 1 was a bad table for most if not all. Seems to me that if someone finished best at the table no matter what the category they should have at least gotten a top 10 call in that category. For me I am still very happy with 11th but what about those teams that landed on table 1 twice, they were messed up from the beginning.

Mad About Que
07-21-2013, 09:25 AM
okay... so I've read about 9 pages of this thread.. and your all wrong. here's the best idea. hand over the scoring aspect to a current Reality TV production crew. Have the judges discuss and vote on each entry. Think a bunch of tables of American Idol judging. oh, theres a thought.. we could walk in down a long hallway with odd ambient lighting and ominous tones sit with our box and let them pick it apart....

but for real, scoring gets to be a tricky beast, and smarter folks than me should be figuring that out. a few years ago in Charlotte Backyard contest, they had it all farked up. some tables with 4 judges, some with 6. then they just averaged the score. worked out for me when some BOA number nerd figured out that there was an issue and I ended up getting a phone call, a check and a trophy for 2nd place ribs. The team that had been the 2nd place ribs got to keep their prizes and was upset the next year and accused me of being one of the whiners. "Dude, someone called me and told me, someone else called me and mailed me a check.".... But crap like that in contests where city event planners and the fire chief are planning the event and setting judging. KCBS/MBM and the like will tweek it, but someone will never be happy. I know statistics will show a few things, but really it's all a page out of the book "Things That Losers Say"...

and I know this is a touchy subject, and of course i'm just talking smack. I have no idea what i'm talking about. my first KCBS event is next week and I'm just hoping that I make turn in's, much less who was on what table. Now remind me of my sarcastic undertone NEXT week when i'm on here complaining about math errors in my 89th place whatever.. hahahahahaa

CivilWarBBQ
07-21-2013, 11:37 AM
I was happy with my scores so definitely not complaining but it sure seems that table 1 was a bad table for most if not all. Seems to me that if someone finished best at the table no matter what the category they should have at least gotten a top 10 call in that category. For me I am still very happy with 11th but what about those teams that landed on table 1 twice, they were messed up from the beginning.

No, your reasoning is flawed. Think of it this way:

A contest has 6 tables. With random selection, it turns out that five brand new teams hit table 1, along with your box. Each of those five new teams are still learning, so they get appropriately low scores. Your product is better, nothing great but it still easily "wins the table". Does that automatically make it a top ten entry?

It's the X factor of the quality of entries each table receives to judge that defeats this idea. Put another way, if you are the best of the worst, that doesn't mean you deserve to walk.

big matt
07-21-2013, 03:02 PM
I asked this earlier but got no response..is it normal to hit all the tables with the same group of teams here is an example from last weekend in Az..we hit every table with the same teams except 1 team on pork and brisket..does this happen every time?

Rookie'48
07-21-2013, 04:04 PM
[QUOTE=Fat Freddy;2559345] . . .but what about those teams that landed on table 1 twice . . . [QUOTE]

Freddy, the way it's set up you should never have that happen unless you're at a contest with LESS than four tables of judges.

Every Table Captain has a sheet where he or she scratches off each team's number as they get that box. This will stop your entries from hitting the same table twice.

I hope that this helps explain things a little bit :grin:.

Fat Freddy
07-21-2013, 05:24 PM
[QUOTE=Fat Freddy;2559345] . . .but what about those teams that landed on table 1 twice . . . [QUOTE]

Freddy, the way it's set up you should never have that happen unless you're at a contest with LESS than four tables of judges.

Every Table Captain has a sheet where he or she scratches off each team's number as they get that box. This will stop your entries from hitting the same table twice.

I hope that this helps explain things a little bit :grin:.

Thanks Dave, not having a copy of the sheet for myself, my teammate had his and the person I rode home with has his. Seems like there is some real question how it all workes, but I have also found out you have already heard about most of the questions,lol.

Like I say I am not unhappy at all, just for the short amount of time I could look at the sheet seemed confusing. Maybe after this coming weekend I will understand completely when I see my disasters I figure is coming.

LindaM
07-21-2013, 06:20 PM
I competed at this same contest last weekend as Lotta Bull.

From the printout, Lotta Bull was placed on Table 13 twice.
With 78 teams this should not happen.
Table #13 was the highest scoring table with 7 top ten calls.
This information is provided on the score sheets.

Table numbers for 1st-10th of each catagory
CK, 4, 2, 3, 12, 7, 9, 2, 2, 11, 12
RI, 1, 1, 12, 8, 4, 13, 13, 11, 1,13
PK, 2, 13, 5, 4, 3, 7, 1, 12, 13,13
BK, 10,12, 5, 8, 1, 13, 8, 12, 2, 1

Top 10 calls by table.
Table 1, calls 6
Table 2, calls 5
Table 3, calls 2
Table 4, calls 3
Table 5, calls 2
Table 6, calls 0
Table 7, calls 2
Table 8, calls 3
Table 9, calls 1
Table 10, calls 1
Table 11, calls 2
Table 12, calls 6
Table 13, calls 7

My entries landed on tables 9, 5, 6, 10; the 4 lowest scoring tables for top 10 calls. The lowest scoring judge was Judge 1, on Table 9 with average of 25.1633.
On chicken this judge gave us 6,6,7 = 25.1428
The other judges
#2 9,8,8 = 32.56
#3 9,9,9 = 36.00
#4 8,8,7 = 30.8572
#5 9,8,8 = 32.5600
#6 9,8,9 = 33.7028
His/her score appears to be completely out of line with the other judges. Here are all of the scores this judge gave out for the day:

CK= 28.5600, 34.8800, 25.1428, 28.0000, 20.0000, 26.2972
RI = 29.1428, 25.1200, 19.4400, 24.0000, 15.3828, 18.2972
PK= 19.4400, 29.1428, 25.1200, 24.0000, 15.3828, 18.2972
BK= 27.4400, 27.4172, 32.0000, 29.7028, 30.2972, 24.0000

This judge should be on KCBS's radar.

We were told at the cooks meeting.
The blind number will no longer be on the print out.
Are using the new 2013 scoring weights.
Comments cards have been temporary stopped.
The Judge Average Score is from all 4 catagories.
KCBS will be tracking judges.

and now the question is how many of those low scoring judges are Master Judges. They typically score much lower than new judges.

swamprb
07-21-2013, 11:51 PM
Got to experience judging my first KCBS comp in many years today and was amazed at how efficient the HJ/Reps were in tallying the results as we turned in the scoresheets. Was able to take a look at results from a previous comp from the Rep using the new software and was pretty impressed at how the scoresheets looked.
Took a look at some of the teams results and liked the table averages, (and how I scored with my table mates) heard some sour grapes complaints from one that felt getting 3 turn ins on 1 table was unfair, but not the GC that had the same results. Very small event with only 13 teams and @75% KCBS CBJ's in attendance.
Awards were earlier than announced, which I think made everyone happy as well.

Rich Parker
07-22-2013, 05:34 AM
I asked this earlier but got no response..is it normal to hit all the tables with the same group of teams here is an example from last weekend in Az..we hit every table with the same teams except 1 team on pork and brisket..does this happen every time?

I noticed that this weekend as I was on the same table with one team three times. I think we are going to see the box dance (folks walking up with only certain teams) a little more often as others realize it.

Funtimebbq
07-22-2013, 09:55 AM
I asked this earlier but got no response..is it normal to hit all the tables with the same group of teams here is an example from last weekend in Az..we hit every table with the same teams except 1 team on pork and brisket..does this happen every time?

Hi Matt,
The fewer number of tables, the more likely it will happen. There were only 5 tables at your AZ contest so not much opportunity for switching around among the table captains.

JD McGee
07-22-2013, 09:58 AM
Anyone get a new printout yet?..thoughts?

I like it! :thumb: Here's a pic of our pork sheet from yesterday...small contest...only two tables.

https://sphotos-a.xx.fbcdn.net/hphotos-ash4/1330_639829522695269_1841346598_n.jpg

Diva
07-22-2013, 10:06 AM
I find the sheets to be quite informative, don't get me wrong, however, I think they rushed putting the program out, considering training for it occurred the second week of July. According to the mp3 of the July BOD meeting..... With 6 votes for and 4 votes against on July 10th that the program be officially used starting on the 11th, that doesn't seem to give contest reps an ample amount of time to get accustomed to the program. According to the meeting mp3, the reps have had the program for 2 months prior to the training, but, what good does that really do? If you're not trained on it, how do you practice with it???

They've been working on it for three years, what would have been the problem to wait 19 more days and start it on August 1st? Not everyone is as computer savvy as the technology committee.

MoKanMeathead
07-22-2013, 11:45 AM
We had our fist experience with the new score sheets this past weekend. I do like the additional information - HOWEVER - our scores were wrong and the score of at least one other team was wrong.

Here were the issues with our scores:
1. If you add up the 5 highest individual judges scores for pork the total is 162.24. When you look at the Pork scoring sheet the total shows 163.9544. The score for Judge 2 is wrong.
2. When you look at the pork score on the overall sheet it says 162.8000 – No idea where that came from.
3. The Team Detail Report shows we were on table 4 for pork – the pork score sheet shows we were on table 7.

There was another team that won Ribs with a 178.8572 but they actually should have had a 180!! 5 Judges gave them 999.

Bottom Line - Check your new score sheets CAREFULLY!

mobow
07-22-2013, 12:00 PM
We had our fist experience with the new score sheets this past weekend. I do like the additional information - HOWEVER - our scores were wrong and the score of at least one other team was wrong.

Here were the issues with our scores:
1. If you add up the 5 highest individual judges scores for pork the total is 162.24. When you look at the Pork scoring sheet the total shows 163.9544. The score for Judge 2 is wrong.
2. When you look at the pork score on the overall sheet it says 162.8000 No idea where that came from.
3. The Team Detail Report shows we were on table 4 for pork the pork score sheet shows we were on table 7.

There was another team that won Ribs with a 178.8572 but they actually should have had a 180!! 5 Judges gave them 999.

Bottom Line - Check your new score sheets CAREFULLY!

Great. (sarcasm)

bdawgbbq
07-22-2013, 12:02 PM
Anyone have input on how to use this sheet to improve your product? We just finished our first comp and left scratching our heads after seeing a 5 and 9 in a single category.

Alexa RnQ
07-22-2013, 03:14 PM
We had our fist experience with the new score sheets this past weekend. I do like the additional information - HOWEVER - our scores were wrong and the score of at least one other team was wrong.

Here were the issues with our scores:
1. If you add up the 5 highest individual judges scores for pork the total is 162.24. When you look at the Pork scoring sheet the total shows 163.9544. The score for Judge 2 is wrong.
2. When you look at the pork score on the overall sheet it says 162.8000 No idea where that came from.
3. The Team Detail Report shows we were on table 4 for pork the pork score sheet shows we were on table 7.

There was another team that won Ribs with a 178.8572 but they actually should have had a 180!! 5 Judges gave them 999.

Bottom Line - Check your new score sheets CAREFULLY!

Which contest was this????

Diva
07-22-2013, 06:01 PM
We had our fist experience with the new score sheets this past weekend. I do like the additional information - HOWEVER - our scores were wrong and the score of at least one other team was wrong.

Here were the issues with our scores:
1. If you add up the 5 highest individual judges scores for pork the total is 162.24. When you look at the Pork scoring sheet the total shows 163.9544. The score for Judge 2 is wrong.
2. When you look at the pork score on the overall sheet it says 162.8000 No idea where that came from.
3. The Team Detail Report shows we were on table 4 for pork the pork score sheet shows we were on table 7.

There was another team that won Ribs with a 178.8572 but they actually should have had a 180!! 5 Judges gave them 999.

Bottom Line - Check your new score sheets CAREFULLY!

We too had the issue with #3. I sent the entire Board an email this morning.

Diva
07-22-2013, 06:04 PM
Which contest was this????

Glasgow, Missouri

SaucyWench
07-22-2013, 06:27 PM
So the new scoring system is getting scores wrong, tables wrong, and the judge tracking system may be tracking the wrong judges? Huh

MikeJ65
07-23-2013, 10:02 AM
I like it, I had great ribs and was the #1 on that table but they trashed everyone on the table. Looking over the other categories with one exemptions they beat up on all their boxes. Proves the long held theory..."sometimes it IS the table you end up on!

Last weekend I won my rib table by over 7 points, but ended up 16th of 32. That table placed 16, 23, 29, 30, 31, 32. I'm sure it's possible that the four worst boxes ended up on one table, but not real likely. But, hey, I will spend more time on the boxes that fell in the bottom half of the table and not worry as much about ribs.

I do like the format. While more information will give us more reasons to bitch, the table ranking and ability to see average table scores will help me know were I need to improve and what was probably just a gift table or bad table.

MikeJ65
07-23-2013, 10:14 AM
Anyone have input on how to use this sheet to improve your product? We just finished our first comp and left scratching our heads after seeing a 5 and 9 in a single category.

One thing I would look at is how each judge ranked you on the table. If a couple have you at the top and a couple have you at the bottom it could mean that you have a flavor profile that doesn't appeal to everyone or there were problems with a couple of samples. Cooks always think that every sample is the same, but they are not.

Balls Casten
07-23-2013, 10:55 AM
Last weekend I won my rib table by over 7 points, but ended up 16th of 32. That table placed 16, 23, 29, 30, 31, 32. I'm sure it's possible that the four worst boxes ended up on one table, but not real likely. But, hey, I will spend more time on the boxes that fell in the bottom half of the table and not worry as much about ribs.

I do like the format. While more information will give us more reasons to bitch, the table ranking and ability to see average table scores will help me know were I need to improve and what was probably just a gift table or bad table.

Your rib average for the year is 163 (four contests from scoreCard) and you scored a 161.68 in that contest. Your highest score for ribs all year is a 165 ... I would work on ribs too.
Not trying to be hard on you just pointing out the misconception of the "table from hell".

AppalachianAmericanBBQ
07-23-2013, 06:52 PM
I agree with Kit. Let the judges draw a number out of a hat. Whatever number they draw that's the table they sit at period.Then it is truly the "Luck of the Draw" just like it is for us cooks when we turn in our entry.

CivilWarBBQ
07-24-2013, 01:47 AM
I agree with Kit. Let the judges draw a number out of a hat. Whatever number they draw that's the table they sit at period.Then it is truly the "Luck of the Draw" just like it is for us cooks when we turn in our entry.

And when the random selection puts three Masters on one table and three virgins on another, what then?

82's BBQ
07-24-2013, 02:55 AM
We had our fist experience with the new score sheets this past weekend. I do like the additional information - HOWEVER - our scores were wrong and the score of at least one other team was wrong.

Here were the issues with our scores:
1. If you add up the 5 highest individual judges scores for pork the total is 162.24. When you look at the Pork scoring sheet the total shows 163.9544. The score for Judge 2 is wrong.
2. When you look at the pork score on the overall sheet it says 162.8000 No idea where that came from.
3. The Team Detail Report shows we were on table 4 for pork the pork score sheet shows we were on table 7.

There was another team that won Ribs with a 178.8572 but they actually should have had a 180!! 5 Judges gave them 999.

Bottom Line - Check your new score sheets CAREFULLY!

I noticed an error (.oo56) in scoring also in one of my categories. New Palestine, IN

rolfejr
07-24-2013, 06:11 AM
There is no value because (1) you don't know the circumstances of those tables and (2) you'll most likely never encounter that set of variables (judges at that table, other entries at that table) again. Back to the roulette example. The odds of any one number coming up are 37-1; if 18 comes up 6 times in a row what are the odds it will come up 18 the next spin? 37-1. The odds always remain the same because they reset every spin. Your contest history within the aspect of judges and tables and all variables (N) is always N=1. It resets every event.

Well said!
The only constant is you, the odds of seeing the same set of judges under the same circumstances is off the charts. If you adjust your style against one or two contest results you will always be chasing your tail.

We are still fairly new to this, 7 comps, and my thinking is that it will take about 10 - 12 to gather up enough data to start tweaking with confidence.

If you change every time you get a bad score, you will have random products with random success.
The best teams cook the same EVERY TIME until they find a consistent flaw in their process. They fix it and they improve.
That is the key to success, not chasing hot tables.

DawgPhan
07-24-2013, 06:54 AM
So wait....this new scoring program is not calculating scores properly? Is that right?

musicmanryann
07-24-2013, 08:41 AM
This is complete hearsay, and what likely amounts to normal hobnobbing between judges across the tables to pass the time between categories, but it was told to me this past weekend that all the judges were talking about the new system during the contest. That KCBS is tracking their scores across contests. Some judges have already received letters because their judging is way off from the norm and some have been banned. It stuck me as odd and inaccurate, but was curious if anyone could confirm or deny that this was the case?

MikeJ65
07-24-2013, 08:45 AM
Your rib average for the year is 163 (four contests from scoreCard) and you scored a 161.68 in that contest. Your highest score for ribs all year is a 165 ... I would work on ribs too.
Not trying to be hard on you just pointing out the misconception of the "table from hell".

Well, this was my 5th ever comp and I've certainly not perfected ribs (or any other category). I changed my rib cook after Kearney and have gone 11/32, 7/34, and 16/32 with a table win since then. I'm just more concerned with the categories that have been inconsistent and ranking 4th or 5th at the table.

Balls Casten
07-24-2013, 09:10 AM
Well, this was my 5th ever comp and I've certainly not perfected ribs (or any other category). I changed my rib cook after Kearney and have gone 11/32, 7/34, and 16/32 with a table win since then. I'm just more concerned with the categories that have been inconsistent and ranking 4th or 5th at the table.

Mike dont get me wrong, you are doing great! I just ment that although the table stats pointed towards it being a bad table. Other stats said you were judged fairly or at least consistant with your other cooks.

Also did you check your judge 6 scores in ribs? Although it was the score that was thrown out I dont know what scores add up to a 29.1544

Alexa RnQ
07-24-2013, 09:55 AM
This is complete hearsay, and what likely amounts to normal hobnobbing between judges across the tables to pass the time between categories, but it was told to me this past weekend that all the judges were talking about the new system during the contest. That KCBS is tracking their scores across contests. Some judges have already received letters because their judging is way off from the norm and some have been banned. It stuck me as odd and inaccurate, but was curious if anyone could confirm or deny that this was the case?

My cynical side says: http://www.divaherself.com/frozenover_sm.jpg
But if the judges' rumor mill is spinning in that direction, well and good. Just giving thought that it *might* happen might give some judges pause. And for the majority that are handing down thoughtful, well-considered scores, well, what would they have to worry about?

Slamdunkpro
07-24-2013, 10:34 AM
This is complete hearsay, and what likely amounts to normal hobnobbing between judges across the tables to pass the time between categories, but it was told to me this past weekend that all the judges were talking about the new system during the contest. That KCBS is tracking their scores across contests. Some judges have already received letters because their judging is way off from the norm and some have been banned. It stuck me as odd and inaccurate, but was curious if anyone could confirm or deny that this was the case?

I can't imaging KCBS moving that fast on anything (except maybe a penis apron)

billygbob
07-24-2013, 11:08 AM
My cynical side says: http://www.divaherself.com/frozenover_sm.jpg
But if the judges' rumor mill is spinning in that direction, well and good. Just giving thought that it *might* happen might give some judges pause. And for the majority that are handing down thoughtful, well-considered scores, well, what would they have to worry about?

Not every bite in a box is the same. What qualifies an individual to get dinged? What about the judge that gets the one piece of pork 'bark' that is just burnt fat or the one "burnt end" that is not rendered. Scoring them according to their actual taste and tenderness makes them an outlier at that table so they get "written up".

Slamdunkpro
07-24-2013, 11:39 AM
Not every bite in a box is the same. What qualifies an individual to get dinged? What about the judge that gets the one piece of pork 'bark' that is just burnt fat or the one "burnt end" that is not rendered. Scoring them according to their actual taste and tenderness makes them an outlier at that table so they get "written up".

If they actually do this here's your future:

888 888 888 888 888 888

Smitty250
07-24-2013, 12:05 PM
My cynical side says: http://www.divaherself.com/frozenover_sm.jpg
But if the judges' rumor mill is spinning in that direction, well and good. Just giving thought that it *might* happen might give some judges pause. And for the majority that are handing down thoughtful, well-considered scores, well, what would they have to worry about?

I have to disagree with this - When / If KCBS starts sending letters out and banning judges for "not in line" judging then the whole process is useless! Our Q will not be judged fairly by the judges out of fear that they will be written up or banned from judging. What happens when that judge that is handing down thoughtful, well-considered scores gets a sub-par entry? Does he/she not give it a rightful score for fear that thier "average" will go down? Or that judge on infamous Table 5 who thinks that what they cook in their back yard is better than everything there - you know the guy/gal who doesn't give anything higher than a 7. Does he/she start dishing out 8's and 9's for fear of banishment?

Now I am a competitor (never judged) so I have had many moments where I said 'What the hell are those judges thinking? That was the best chicken/ribs/pork/brisket I ever cooked" and ended up middle of the pack or worse. Or even take a 1st in a category one week and the next week not walk doing the same exact thing! So I know and understand the frustrations and confusions when it comes to judging. I think we all need to take a moment and realize that every competition is different - different meat (not the same exact cow, pig, chicken), different tables, different judges, different altitude/humidity/temperature, different teams, etc...But the underlying constant we see is that the "Top" teams continue to win! Not only winning on thier home filed but winning when they travel to other states as well. They are winning where the "difference" is multiplied many times.

I can only see that if KCBS starts "judging the judges" and pushes them to the "norm" then we as competitiors will still not be happy with the outcome! If this happens remember this post at the next comp you go to and there is a 5 way tie for 1st in chicken!

Vince RnQ
07-24-2013, 12:05 PM
Not every bite in a box is the same. What qualifies an individual to get dinged? What about the judge that gets the one piece of pork 'bark' that is just burnt fat or the one "burnt end" that is not rendered. Scoring them according to their actual taste and tenderness makes them an outlier at that table so they get "written up".

We're not talking about a single score in a single category at a single event. The judge's scoring data is useful because it is going to identify those who are outside of the norm on both the high and low sides over a number of categories at a number of contests.

Individual scores are not necessecarily indicative of anyone's average and I certainly wouldn't want to see a judge be confronted for giving an honest score to a single entry.

The real elephant in the room is that for the first time, teams are going to see just how good or how bad the KCBS judging system really is. I have always believed that if a judge cannot articulate their reasoning behind the score he/she gave to an entry they have no business judging in the first place. That's why I have always been a proponent of mandatory comment cards for any score of 5 or lower. Of course, the comment card system is temporarily, (most likely permanently), on hiatus so that really doesn't much matter.

Candy Sue
07-24-2013, 12:26 PM
This is complete hearsay, and what likely amounts to normal hobnobbing between judges across the tables to pass the time between categories, but it was told to me this past weekend that all the judges were talking about the new system during the contest. That KCBS is tracking their scores across contests. Some judges have already received letters because their judging is way off from the norm and some have been banned. It stuck me as odd and inaccurate, but was curious if anyone could confirm or deny that this was the case?

I don't know about any of this! I doubt any other board member does either. So I'd call bs on this one, until a judge showed a bona fide letter.

musicmanryann
07-24-2013, 12:38 PM
I don't know about any of this! I doubt any other board member does either. So I'd call bs on this one, until a judge showed a bona fide letter.

Thanks for the reply Candy! I figured as much, but thought it was interesting it was already being discussed around the tables.

Candy Sue
07-24-2013, 12:43 PM
Big concern is the bunching of the same teams' entries to different tables. Table captains make sure they haven't seen an entry before, but right now, there's no way to keep the same team entries from hitting the same tray going to a different table. Larger the contest, the less it's likely for this to happen. It is worrisome at smaller events though.

gettinbasted
07-24-2013, 12:58 PM
Candy, while we have you here, is there any concern/response to the scoring errors mentioned in this thread? Thanks!

Candy Sue
07-24-2013, 01:54 PM
Candy, while we have you here, is there any concern/response to the scoring errors mentioned in this thread? Thanks!

Responses related to issues at Glasgow have been addressed personally to those that filed an "official" complaint with KCBS. This isn't something that I feel comfortable with addressing here. I can assure that issues were quickly addressed and resolved by folks at KCBS. Results should be correct at kcbs.us now. Any team that cooked this contest can get corrected team detail and tally sheets from the office.

Vince RnQ
07-24-2013, 02:28 PM
Responses related to issues at Glasgow have been addressed personally to those that filed an "official" complaint with KCBS. This isn't something that I feel comfortable with addressing here. I can assure that issues were quickly addressed and resolved by folks at KCBS. Results should be correct at kcbs.us now. Any team that cooked this contest can get corrected team detail and tally sheets from the office.

Candy, the greater issue is that there were scoring/calculation errors and that has now created doubt as to whether there are accuracy issues within the scoring program itself or if it was due to operator/input error. We are all certainly aware that mistakes will happen due to human error and we all know the GIGO (Garbage In, Garbage Out) principle. We also know that KCBS will correct those problems including corrections to monies being properly distributed should that be necessary.

However, we also all want to be confident that the calculations being made within the program itself are correct and accurate. That is the question that most are likely to have after reading of the problems at the Glasgow event.

I don't care to know all the details of what happend at Glasgow and don't expect you to spell them out for us. But, I do expect that if the problems at Glasgow were due to issues with the scoring program itself and not operator/input error, that you would address that matter publicly and in detail.

G$
07-24-2013, 02:34 PM
I can appreciate not commenting on individual team's scores and resolutions related to their inquiries. What can you tell us about steps to prevent errors such as these from happening in the future?

G$
07-24-2013, 02:37 PM
As usual, Vince finished ahead of me and with a more complete entry. :icon_smile_tongue:

gettinbasted
07-24-2013, 03:00 PM
I went over to kcbs.us and had a peak. Apparently my 8th place finish is now a 10th! I have managed to learn how to blow points in the week after my cook is over! Do I just call up to the office to get an explanation of my scores changing after the fact?

gettinbasted
07-24-2013, 03:16 PM
I talked to Karrin up at the office and she got me squared away.

Candy Sue
07-24-2013, 03:36 PM
Candy, the greater issue is that there were scoring/calculation errors and that has now created doubt as to whether there are accuracy issues within the scoring program itself or if it was due to operator/input error. We are all certainly aware that mistakes will happen due to human error and we all know the GIGO (Garbage In, Garbage Out) principle. We also know that KCBS will correct those problems including corrections to monies being properly distributed should that be necessary.

However, we also all want to be confident that the calculations being made within the program itself are correct and accurate. That is the question that most are likely to have after reading of the problems at the Glasgow event.

I don't care to know all the details of what happend at Glasgow and don't expect you to spell them out for us. But, I do expect that if the problems at Glasgow were due to issues with the scoring program itself and not operator/input error, that you would address that matter publicly and in detail.

The Glasgow issue was not an issue with KCBScore. The contest has been re-entered multiple times, results as posted have been duplicated exactly. The program has integrity, it works as expected. If there were issues with KCBScore, I would say so. I have confidence that what happened at Glasgow was an aberration and steps are in place to rectify same going forward.

Vince RnQ
07-24-2013, 03:44 PM
As usual, Vince finished ahead of me and with a more complete entry. :icon_smile_tongue:

On the other hand, you said it just as well but with far fewer words!

AppalachianAmericanBBQ
07-24-2013, 03:52 PM
I may upset some folks but here's a thought. How about everyone is upset with table 4 or judge 6 or whatever the case may be just keep it to themselves. I have read every post on this thread. Some were good and others not so much. You would think that there were black helicopters flying over BBQ contest with sercret service agents talking into a judges earpiece on how to score the event.

Why can't some of you just understand that there is a new score sheet. Period. They aren't using comment cards for right now. Period. That some days you will turn in the same product and get 1st at one comp and 20th at another. Sometimes your food is great and some days it just flat out stinks. How about we all just embrace the changes that are taking place and see how it goes.

It's a new system. There are going to be mistakes. Deal with it and carry on.

Outnumbered
07-24-2013, 04:01 PM
:icon_bugeyed^^^Cause this is a discussion board.

And people are discussing the new format and talking about how it can help them get better, or not. And they are simply saying what they would like to see or not.

But that could just be a guess. :grin:

DawgPhan
07-24-2013, 07:27 PM
The Glasgow issue was not an issue with KCBScore. The contest has been re-entered multiple times, results as posted have been duplicated exactly. The program has integrity, it works as expected. If there were issues with KCBScore, I would say so. I have confidence that what happened at Glasgow was an aberration and steps are in place to rectify same going forward.


So just to be clear, you are saying that the same information was entered into the new KCBScore system multiple times and it output the same results?

Any chance that the same data was entered into the old system and the results matched?

Also you are confirming that the issues from Glasgow were in no way a bug within the software and no reasonable person would think that it was a software bug given the situation?

Candy Sue
07-24-2013, 09:41 PM
Yes

No -- weighting changed slightly with KCBScore. Many times the results would be close, but not necessarily every time. KCBS did run KCBScore at a number of contests pre-release to check this. This is why BQwin and KCBScore can't run side by side. This is why a date certain was picked to switch scoring programs at all contests.

Yes

CivilWarBBQ
07-25-2013, 04:36 AM
Straight, simple answers.

Candy Sue serves the KCBS membership well my friends.

DawgPhan
07-25-2013, 07:26 AM
Yes

No -- weighting changed slightly with KCBScore. Many times the results would be close, but not necessarily every time. KCBS did run KCBScore at a number of contests pre-release to check this. This is why BQwin and KCBScore can't run side by side. This is why a date certain was picked to switch scoring programs at all contests.

Yes


My point is that running the same data through the software is no way to confirm that the scores are actually correct, just that the system will act in the same manner consistently.

I am guessing that someone actually did the math by hand to confirm that the results from KCBScore are accurate, but you saying that KCBScore spits out the same results is VERY different than saying that it spits out the CORRECT scores.

So can you also confirm that the scores coming out of KCBScore are accurate and that has been verified by several different sources. Not just running the contest data back through KCBScore. Again, running the contest data through KCBScore multiple times DOES NOT confirm that the scores are accurate, only that KCBScore acts consistently. Which is a good thing and important, but it does not confirm the accuracy.

Also providing the method used to confirm the results were accurate would be great.


thanks

DawgPhan
07-25-2013, 07:27 AM
Straight, simple answers.

Candy Sue serves the KCBS membership well my friends.


unfortunately simple answers didnt really get to the point of confirming that the scores are accurate only that they are consistent. You know that there is a big difference and running the data through the same system in no way confirms accuracy.

Funtimebbq
07-25-2013, 09:16 AM
I've read all Candy's responses to mean there was a human error in the process and she can't say that because it is a "personnel" issue.

Balls Casten
07-25-2013, 09:20 AM
Anyone know how a 29.1544 is possible with the new weighted scores?

Outnumbered
07-25-2013, 09:30 AM
Anyone know how a 29.1544 is possible with the new weighted scores?
Just got it to work...7,8,6

PeppermonkeyBBQ
07-25-2013, 09:31 AM
I can't imaging KCBS moving that fast on anything (except maybe a penis apron)

I've been reading 14 pages of this thread and finally laughed ... Thanks!

Vince RnQ
07-25-2013, 09:36 AM
Anyone know how a 29.1544 is possible with the new weighted scores?

That score can be calculated using 7-8-6

7 x 0.5600 = 3.9200
8 x 2.2972 = 18.3776
6 x 1.1428 = 6.8568

Total = 29.1544

Rich Parker
07-25-2013, 12:15 PM
My point is that running the same data through the software is no way to confirm that the scores are actually correct, just that the system will act in the same manner consistently.

I am guessing that someone actually did the math by hand to confirm that the results from KCBScore are accurate, but you saying that KCBScore spits out the same results is VERY different than saying that it spits out the CORRECT scores.

So can you also confirm that the scores coming out of KCBScore are accurate and that has been verified by several different sources. Not just running the contest data back through KCBScore. Again, running the contest data through KCBScore multiple times DOES NOT confirm that the scores are accurate, only that KCBScore acts consistently. Which is a good thing and important, but it does not confirm the accuracy.

Also providing the method used to confirm the results were accurate would be great.


thanks

I know you aren't asking me but listening to the bod meeting, I heard that contests were ran through both BQWin and KCBSScore for testing its accuracy.

The bod decided at another time to release the new weighting scale when score was released. This kind of muddy's the water releasing these two changes at the same time considering how big of a change KCBSScore is but "it is what it is".

Candy Sue
07-25-2013, 12:29 PM
I know you aren't asking me but listening to the bod meeting, I heard that contests were ran through both BQWin and KCBSScore for testing its accuracy.

The bod decided at another time to release the new weighting scale when score was released. This kind of muddy's the water releasing these two changes at the same time considering how big of a change KCBSScore is but "it is what it is".

With weighting changes, results won't be exact between the old software and the new software scoring the same data. They can't be exact, there's a different multiplier on Appearance and Tenderness. There was beta testing with BQwin and KCBScore with the weighting being identical and the results were identical. KCBScore works the way it is supposed to work. Can we move on now? Maybe comment cards???

bbq.tom
07-25-2013, 12:57 PM
Can we move on now? Maybe comment cards???

I'd love to see the Comment Cards back for judges to use. I think that Don Harwell had a good idea to tear off the "new" number after re-writing the team number then staple it to the scorecard. Seems pretty simple to me. Now some will say that it only works for small contests as there would be too many teams and cards at larger contests, but aren't there more REPS at larger contests? Seems pretty equivalent to me.

If I give a team a 6 or below, I would expect the team to want to know WHY. Without comment cards they will never know.

Vince RnQ
07-25-2013, 12:58 PM
Candy, I think the accuracy of the scoring program is an issue of serious interest to all competitors and it seems that the discussion of any problems the program may have have hardly been beaten down in this or any other thread. We all want to have complete confidence in the program and you answering questions here goes a long way towards assuring that.

This program represents one of the biggest changes that Teams, Judges and Reps have experienced due to the wonderful sharing of information that it provides to us all. I truly hope that the information that is now being documented and distributed will lead the way for improvements wherever they can be made.

gettinbasted
07-25-2013, 01:21 PM
My scores were some of the ones in question and i feel confident that it was not an issue with the new system.

The human element has always been a source of potential errors and KCBScore now gives us more data to compare. In this case the additional data helped point out some scoring discrepancies. A valuable discussion might be how to minimize or QC the data entry points to remove as much of the human element as possible. A judges score going directly into the system might be one way to reduce errors. Obviously KCBS can't supply 50+ iPads to each competition but a bring your own device setup could have potential. Double entry of scores could also reduce error (and push back awards).

Any other ideas?

CivilWarBBQ
07-26-2013, 01:46 PM
Vince, you obviously want to see the technical details of the new software and its testing methodology in a very granular fashion. I would not expect the KCBS BoD President to be versed in such - I suggest you direct your inquiry to the Technology Committee that has dealt with the software and its developers since inception.

I'm sure a simple phone call to the KCBS office will get you the contact information you need, and you can report back what you learn here.

LindaM
07-27-2013, 12:53 PM
Big concern is the bunching of the same teams' entries to different tables. Table captains make sure they haven't seen an entry before, but right now, there's no way to keep the same team entries from hitting the same tray going to a different table. Larger the contest, the less it's likely for this to happen. It is worrisome at smaller events though.

That's what table captain sheets are for. But then there are some contests that don't bother to use them. Consistency is the key. If everyone uses them, the chances are far less to seeing duplicates to a table. With 14 tables there is no excuse a team would land on the table 2X as I read in another posting