PDA

View Full Version : Table of Death


big matt
11-04-2013, 10:49 AM
We all know about bad tables and how they can kill your day..well this past weekend in Laughlin,Nv we had very bad table..table #10..their highest score of the day was a 41st in pork,47th in chicken,48th in ribs and a whopping 53rd in brisket!..now I can see a low scoring table on one meat but all 4?..can you imagine getting your score sheet and see you won your table but were 53rd?..are you kidding me would be my thought..that 53rd brisket was cokked by the#3 guy in the country..I'm sorry even on his worst day he doesn't cook 53rd place brisket he just doesn't..also the highest scoring team that hit table 10 was 16th overall..that means if you hit that table you were done no matter how well you did..this folks can kill comp bbq and will send people home really pissed off that stuff like this can actually happen..we all know it's a gamble but with the new data out there something has got to be done to correct this problem and it needs to be done ASAP IMO..yes we hit the table but so did 24 other teams and I'm sure they feel the same way.

carlyle
11-04-2013, 11:27 AM
Utilizing the judges that you have at a contest in a way that is fair to everyone is one of the biggest challenges that an organizer has.

Currently the only tools at my disposal are previous experience with individual judges,
how many contests they have done, and any info shared with other organizers.

With the new scoring system providing so much more information, I have hopes that
organizers will get some new tools to help sort our judges and create tables that are better balanced. I hope that is not a naive pipe dream!

And judges need the kind of feedback that the teams are now seeing to evaluate their own performance. Just talking among themselves between categories would not have helped the situation you described, since the whole table was down. Not just one or two individual judges. Ways need to be found to communicate this scoring info to the judges
while maintaining the anonymity. There has to be ways to do that.

IMO- it is just as bad to have a "hot " table as it is to have a "cold " table.

More opinion- no matter how good a team is, there can be an occasional bad result.
That is why we have to actually cook the contests, and not award prizes based on team standings. Probably not the case here, though, since that table was consistently
low across all 4 categories.

Very sorry for your experience and your bad luck to land on that table.

Much thanks for sharing the specifics on the Brethren. The info needs to be out there.

To some degree, this is a preventable situation by creating balanced tables using all the
data available.

KCBS, please help us organizers do just that. Help us create a more level playing field
for everyone. So that the winner is actually the best that day, not just who was the luckiest to hit the best tables.

cpw
11-04-2013, 11:40 AM
The same thing happened at the (I think) Hendersonville Sam's club regional. There was 1 table that had Swamp boys, Killer B's, and a few other big names who all scored at the bottom of the pack in pork. It effectively killed some team's chances of moving on to the nationals. And I will agree that everyone has a bad day, but not that many teams who cook that well.

big matt
11-04-2013, 11:44 AM
Utilizing the judges that you have at a contest in a way that is fair to everyone is one of the biggest challenges that an organizer has.

Currently the only tools at my disposal are previous experience with individual judges,
how many contests they have done, and any info shared with other organizers.

With the new scoring system providing so much more information, I have hopes that
organizers will get some new tools to help sort our judges and create tables that are better balanced. I hope that is not a naive pipe dream!

And judges need the kind of feedback that the teams are now seeing to evaluate their own performance. Just talking among themselves between categories would not have helped the situation you described, since the whole table was down. Not just one or two individual judges. Ways need to be found to communicate this scoring info to the judges
while maintaining the anonymity. There has to be ways to do that.

IMO- it is just as bad to have a "hot " table as it is to have a "cold " table.

More opinion- no matter how good a team is, there can be an occasional bad result.
That is why we have to actually cook the contests, and not award prizes based on team standings. Probably not the case here, though, since that table was consistently
low across all 4 categories.

Very sorry for your experience and your bad luck to land on that table.

Much thanks for sharing the specifics on the Brethren. The info needs to be out there.

To some degree, this is a preventable situation by creating balanced tables using all the
data available.

KCBS, please help us organizers do just that. Help us create a more level playing field
for everyone. So that the winner is actually the best that day, not just who was the luckiest to hit the best tables.

I think you explain it much better than me thanks!..and guys I'm not bitching I feel for all teams that hit that table

Podge
11-04-2013, 12:26 PM
I totally agree. These kind of tables will hurt competition BBQ as a whole. I was too, at Hendersonville sam's. I didn't hit table #1 and GC'ed by 12 points. I could have potentially hit that table and not made it to the Nationals. I was fortunate. Good thing that Rub is such a hell of a good cook to be RGC after hitting that table!

big matt
11-04-2013, 12:55 PM
I totally agree. These kind of tables will hurt competition BBQ as a whole. I was too, at Hendersonville sam's. I didn't hit table #1 and GC'ed by 12 points. I could have potentially hit that table and not made it to the Nationals. I was fortunate. Good thing that Rub is such a hell of a good cook to be RGC after hitting that table!

Same here we have won 3 times since the new data was released and each time we avoided the bad table..but each time we have hit it we've been down in the scoring..and I'm not alone it happenes at every contest..the question is how do we even it out and prevent it from happening?

bbq.tom
11-04-2013, 01:14 PM
From a Table Captain and Judge perspective I'd like to address this issue:
1. MOST of the contests that I TC or judge I am NOT assigned a table, just told to go sit at a table.
2. Where I have been told a particular table to go to it is either based on check-in order or based on number of contests judged.
3. Finally, I've been the TC for a "table of death" and I tried almost every entry, and the judges did NOT under score the entries. Sometimes a team has a bad day no matter how good they usually are.

There definitely needs to be more thought to judge placement at tables.

smokeisgood
11-04-2013, 03:11 PM
Same here we have won 3 times since the new data was released and each time we avoided the bad table..but each time we have hit it we've been down in the scoring..and I'm not alone it happenes at every contest..the question is how do we even it out and prevent it from happening?
The only way to do it is to have only one judging table. Of course, that's a lot of bbq to be judged by a few people....

big matt
11-04-2013, 04:28 PM
From a Table Captain and Judge perspective I'd like to address this issue:
1. MOST of the contests that I TC or judge I am NOT assigned a table, just told to go sit at a table.
2. Where I have been told a particular table to go to it is either based on check-in order or based on number of contests judged.
3. Finally, I've been the TC for a "table of death" and I tried almost every entry, and the judges did NOT under score the entries. Sometimes a team has a bad day no matter how good they usually are.

There definitely needs to be more thought to judge placement at tables.
I see your point and thanks for the input..my problem isn't with one meat scoring low I understand that happenes but when a table scores EVERY meat low we have a problem..at this contest there were 85 teams..the highest scores off table 10 were 41st pork,47th chicken,48th ribs and 53rd in brisket.

Smoke'n Ice
11-04-2013, 04:32 PM
Maybe IBCA has the right answer, half the entries at a table move to the next level, regardless. Takes longer and more product but it could be the only fair way to judge.

Blazing BBQ was third at the Jack and they cook mostly IBCA, just saying! If we had feedback on our placment in IBCA, I would start cooking them more as I live in the area.

bbq.tom
11-04-2013, 04:35 PM
One other possibility is to have two tables judge the same entries. Of course this would necessitate having each team submit 12 samples of each category. Then average the two tables.

Just a thought.

big brother smoke
11-04-2013, 04:38 PM
Just post their names, it would bring an end to that crap real quick :icon_blush:

G'pa Herb
11-04-2013, 05:52 PM
There doesn't seem to be a standard manner of seating judges by contest reps/organizers. There are many differing ways of seating judges, i.e., no two people living at the same address at the same table, no drinking buddies at the same table, just take a seat, play musical chairs based on a 20 question game where you raise your hand, sit in an assigned seat at an assigned table, line up by number of contests judged and get an assigned table, etc. When judging, I have experienced all of the above. Which is best? I have a personal opinion having experienced all of the above options, but my opinion doesn't matter. What does matter is that there is consistency across all competitions, and that KCBS put out a standard policy for seating judges that must be followed by all contest reps/organizers.

landarc
11-04-2013, 06:23 PM
Actually, with the improvements in data, perhaps what Carlyle suggest will be possible, as judges will now have a track record. Over time, even a year, you will be able to manage table assignments based upon ratings of judges. Not whether they are good or bad, but, by handicap, so to speak. That would actually do wonders for evening out scores and eliminating both hot and cold tables. That would be good, across all aspects.

big matt
11-04-2013, 06:38 PM
Actually, with the improvements in data, perhaps what Carlyle suggest will be possible, as judges will now have a track record. Over time, even a year, you will be able to manage table assignments based upon ratings of judges. Not whether they are good or bad, but, by handicap, so to speak. That would actually do wonders for evening out scores and eliminating both hot and cold tables. That would be good, across all aspects.

This sounds right on point to me..bottom line guys is that we all want to be judged fairly and on a level playing field..not sitting there with our fingers crossed and hoping that we don't end up on a terrible table.

Smoke'n Ice
11-04-2013, 06:52 PM
Looking at the results as listed on the KCBS website, the contest was not run according to KCBS rules, there were not enough reps to properly adminsister the contest. I'm not sure why SOW has a bye in this area, but it is a good possibility that given the proper number of reps, they may have been able to administer the seating of judging better. These are things that should be considered when selecting which contest to attend.

bbq.tom
11-04-2013, 07:10 PM
Seems that only a few REPs are "placing" judges at tables. The rest just let judges sit wherever as long as no spouses are sitting together.

Slamdunkpro
11-04-2013, 07:30 PM
Do it the way they assign seats in poker tournaments: When the judge signs in they draw a card with a table# and seat#.

Alexa RnQ
11-04-2013, 07:39 PM
Looking at the results as listed on the KCBS website, the contest was not run according to KCBS rules, there were not enough reps to properly adminsister the contest. I'm not sure why SOW has a bye in this area

There were four reps present, not sure why the website did not reflect that.

I would think it would be entirely possible for the current data being gathered to be applied to establishing a "handicap" or relevance score for judges, and for the judges' seating to be presorted accordingly. It would take some diligence in sign-ups (never a bad thing), and would still be subject to the occasional no-show, but as a cook I'd be willing to give it a spin and see how things shake out. It would provide much-needed consistency in the application of judge sorting across contests and regions.

Slamdunkpro
11-04-2013, 07:56 PM
i would think it would be entirely possible for the current data being gathered to be applied to establishing a "handicap" or relevance score for judges, and for the judges' seating to be presorted accordingly......
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888
888-888-888-888-888-888

Rub
11-04-2013, 08:31 PM
Unfortunately these tables have always been a part of contests but it's just now coming to light in KCBS. The FBA has had this info on the score sheets for over 10 years and I've got to say the reps put a lot of effort into seating judges in the most fair setting they can. Obviously they aren't aware of every judge's tendencies but the effort is a start. I'm surprised to read that it is not policy for the KCBS reps to seat judges every time.

Like someone mentioned earlier, now is the time to manipulate and use the data present on the judges to create a handicap for each. Then once the judge list is set for the contest the software can create a pretty close to even seating arraignment for judging. I don't believe the FBA is doing this but they have a much smaller operating budget than KCBS. I don't think the cooks could ask for any more. I'd love to see it happen.

In the end all the teams want is to be scored and ranked fairly on their skills that day, compared to the rest of the field. Good or bad, as long as it's fair. These death tables don't allow that to happen.

djqualls
11-04-2013, 08:34 PM
We all know about bad tables and how they can kill your day..well this past weekend in Laughlin,Nv we had very bad table..table #10..their highest score of the day was a 41st in pork,47th in chicken,48th in ribs and a whopping 53rd in brisket!..now I can see a low scoring table on one meat but all 4?..can you imagine getting your score sheet and see you won your table but were 53rd?..are you kidding me would be my thought..that 53rd brisket was cokked by the#3 guy in the country..I'm sorry even on his worst day he doesn't cook 53rd place brisket he just doesn't..also the highest scoring team that hit table 10 was 16th overall..


Well Guys The 53rd in Brisket was me. Yes I won my table. Woo F'n Hoo!

I'd love to know who the judges were on this table. I'd take a guess it was a couple MCBJ's and the rest were sponsor employees... Anyway They are a detriment to the sport!

The top six Brisket cooks in the nation were there and it would have been great to see a shootout among us but sadly it didn't happen. The only bright side is that we all kept our same place in TOY so no harm.

I have hit two bad tables this year that I believe cost me the GC but those are the odds.

At the Royal Invitational, I was 1st on Table 5 and 149th in Brisket. And this isn't one of the contests that I think cost me a GC.

Bottom Line, KCBS needs some leadership in the office as well as on their board. The middle tier of cooks will quickly dissolve with this present system.

I'll sum up the judging competency/consistency in KCBS Like This.
To take a line out of an old Bluegrass song.. It Ain't Broke, But Brother Its Badly Bent"!

Bubba
11-05-2013, 03:57 AM
Guys it's the organizer of the contest responsability to get the judges to judge the contest. What I would like to see is that KCBS take over the part of getting judges to judge the contest.

Pappy Q
11-05-2013, 04:57 AM
Guys it's the organizer of the contest responsability to get the judges to judge the contest. What I would like to see is that KCBS take over the part of getting judges to judge the contest.

I have first hand knowledge of a recent contest that had a Table of Death that drastically altered the outcome of the contest as well as a year long points chase. The organizer was very upset about it and went to KCBS to get the names of the judges at that table as he wanted to ban them from the contest (yes it was that bad) but KCBS refused to give him the info. KCBS has a problem that they brought upon themselves by giving us cooks the info to confirm what we always knew. But they don't want to address it and no BOD member is standing up to do anything about it.

Serious Hog
11-05-2013, 06:47 AM
Could table of deaths be popping up because the judges, regardless of experience, are talking too much during judging? As a result they are influencing each other, causing the whole table to scores high or low.

TooSaucedToPork
11-05-2013, 07:05 AM
I have been in 2 discussions on this topic.

My solution...use the new software to track judges. You figure out a mean range score and any judge above is a high scorer. Any judge below is a low scorer, any judge in the middle range is average.

The software could color code them so exact scores are not known to reps.

Then you distribute them equally at the tables. Any new judge is distributed equally among tables.

Pretty simple, control the numbers and you eliminate us teams being able to blame tables or judges...with that taken out of the equation, it's our food that scores low. Not the table.

My 2 cents ;-)

bignburlyman
11-05-2013, 07:13 AM
Could table of deaths be popping up because the judges, regardless of experience, are talking too much during judging? As a result they are influencing each other, causing the whole table to scores high or low.


At all the contests I have judged, there is no table talk while the judging is in process. Everyone does take it serious at the table. That does not mean that the judge scores correctly, but I don't think distractions or talk at the table are the cause of a table of death. In my opinion only----there are BAD judges, if enough bad judges sit at the same table, it will create the table of death. I know a few judges who I don't like to be at the table with, and if I have the choice I will sit as far away as I can from them.

What I forsee happening to correct this? KCBS will take the table number off the scoring sheet, you will not be able to see that info, problem solved! :doh:

Smokin' Gnome BBQ
11-05-2013, 07:22 AM
I think some of the death tables are a result of t.v. shows like pitmasters. We all run to out t.v.s to watch them and with out thinking start to believe that is what bbq is supposed to look like in the box or it should be sweet or if it aint thighs it aint chicken. Bbq was an art now its just us photo coping what we see on these shows. Don't get me wrong I love to watch them over and over again. When was the last time you heard of someone turning in Cornish game hens or went naked in the box at a KCBS event and did well? you wont because that would break the boundaries of what is drilled into both cooks and judges brains.

I hate to say this but I would really hate to hit a rib table that someone turned in McRibs because that's what many people believe is good bbq.

New Pal Frank
11-05-2013, 07:25 AM
Could table of deaths be popping up because the judges, regardless of experience, are talking too much during judging? As a result they are influencing each other, causing the whole table to scores high or low.


It has been my experience that tere is NO talking until ALL the score csards are turned in and the TC clears us for that catagory.

Jorge
11-05-2013, 08:10 AM
I have been in 2 discussions on this topic.

My solution...use the new software to track judges. You figure out a mean range score and any judge above is a high scorer. Any judge below is a low scorer, any judge in the middle range is average.

The software could color code them so exact scores are not known to reps.

Then you distribute them equally at the tables. Any new judge is distributed equally among tables.

Pretty simple, control the numbers and you eliminate us teams being able to blame tables or judges...with that taken out of the equation, it's our food that scores low. Not the table.

My 2 cents ;-)

I think this is probably the most sensible proposal for KCBS to open the internal discussion with.

The biggest issue is the data collected to date. My understanding is that the data has been collected, but there has been no analysis. Newer judges, based on anecdotal evidence, will score lower but have scores that fall in line over time. The alleged 'super judge' will be consistent and skew things some but that will largely be offset by their consistency. The biggest variable will be judges whether they score high or low, that judge much more frequently than the norm.

Without seeing the data, I have no idea whether there is a sufficient sample to make any long term changes now. I understand the need and desire to address the issue. It's important to keep in mind that the solution, could be worse than the existing issue if it's not given the thought it's due.

It's also important to remember that's not always the judges. In the example Matt gave I have a hard time believing those particular cooks earned those scores. That being said, there are cooks that have killed tables for fellow competitors. I've been at a table, and had some brisket that was so foul that crackers, cheese, grapes, and water couldn't completely remove the taste from my mouth. I probably scored higher than I normally would have to give the cooks that followed the benefit of the doubt since it wasn't their responsibility that the entry that preceded theirs was quite possible the worst bbq I've ever tasted.

Jorge
11-05-2013, 08:12 AM
What I forsee happening to correct this? KCBS will take the table number off the scoring sheet, you will not be able to see that info, problem solved! :doh:

My understanding is that they will try to mask the table number on the printed results. Table 5 will become Table Q or something similar.

BKBBQ
11-05-2013, 08:56 AM
Our ribs were on the table with you...cost us a top five overall i would say.

bover
11-05-2013, 09:00 AM
What I would really like to see is KCBS taking full advantage of the data that is now being gathered with the new scoring system and put it to practical use. Some examples of things I see as very viable are:


Create one standardized website for judges to go register for contests
Use the information gathered on judges to generate a balanced seating arrangement for the organizer's use (add $25-$50 to sanctioning fees for this service)
Website would also include a portal so KCBS members could track contests they are registered to judge, contests they have judged, and how their scores compared to others (a simple three-color rating based on a std dev of 1 would likely suffice)
Website portal would also provide archived contest results for cook's research purposes
and so on and so forth...


Now that the data is being gathered, and I'm assuming stored, the mining possibilities are endless. My hope is that the board has the technical vision to get the most out of it so that both judges and cooks can benefit.

TooSaucedToPork
11-05-2013, 09:15 AM
I have been thinking about this for a while…This is what I came up with.

You take ALL scores given in EACH contest over the last 10 years and add them. This gives you a good indication of a perfect average score. Will this math take a long time…yes. Will this be the best way to get a true perfect average score… yes.

You then develop a mean average. This is arrived at by using a cluster. You use the new software to look at percentages.

You take a number, lets say 66.67%, and the 66.67% of scores between the total contest average become your average. Lets say 66.67% of judges fall between 29.24 and 32.30 as their average.

Why 66.67 you ask, because there are 6 table judges and 100 divided by 6 is 16.66666 and 16.666 multiplied by 4 is 66.67

The other 33.33% are at or above 32.31 , and at or below 29.23

Every judge below 29.24 (16.665%) is a lower than average scoring judge. Not a lowballer, or a bad judge…just a lower than average scoring judge.

Every judge above 32.30 (16.665%) is a higher than average scoring judge. Not a highballer, or a great judge…just a higher than average scoring judge.

Everyone in the mean range (middle or average) is an average scoring judge. Not a good or bad judge, not a sheep that goes with the flock…just a judge whose scores fall in line with the mean average.

The beauty of this is it is a living document. It will ebb and flow as averages increase or decrease.

It could even be used to map the average score among judges at a SPECIFIC contest…thus ensuring there are 66.67% average judges, 16.665% High judges, and 16.665% low judges…or 4 average scorers, 1 high, and 1 low among the judges at that specific contest.

Judges will score with their hearts as they always have done and will quite possibly move between catagories and that is fine. It is all about balancing the tables and reducing “tables of death”. But there is NO perfect system…there will still be tables that become mathematical outliers no matter how hard we try. This will reduce drastically but NEVER eliminate bad tables…the only way to ensure that bad tables are forevermore eliminated would be to have 1 table of six judges judge EVERY entry.

It will just bring balance to the force.

Jorge
11-05-2013, 09:38 AM
I have been thinking about this for a while…This is what I came up with.

You take ALL scores given in EACH contest over the last 10 years and add them. This gives you a good indication of a perfect average score. Will this math take a long time…yes. Will this be the best way to get a true perfect average score… yes.

You then develop a mean average. This is arrived at by using a cluster. You use the new software to look at percentages.

You take a number, lets say 66.67%, and the 66.67% of scores between the total contest average become your average. Lets say 66.67% of judges fall between 29.24 and 32.30 as their average.

Why 66.67 you ask, because there are 6 table judges and 100 divided by 6 is 16.66666 and 16.666 multiplied by 4 is 66.67

The other 33.33% are at or above 32.31 , and at or below 29.23

Every judge below 29.24 (16.665%) is a lower than average scoring judge. Not a lowballer, or a bad judge…just a lower than average scoring judge.

Every judge above 32.30 (16.665%) is a higher than average scoring judge. Not a highballer, or a great judge…just a higher than average scoring judge.

Everyone in the mean range (middle or average) is an average scoring judge. Not a good or bad judge, not a sheep that goes with the flock…just a judge whose scores fall in line with the mean average.

The beauty of this is it is a living document. It will ebb and flow as averages increase or decrease.

It could even be used to map the average score among judges at a SPECIFIC contest…thus ensuring there are 66.67% average judges, 16.665% High judges, and 16.665% low judges…or 4 average scorers, 1 high, and 1 low among the judges at that specific contest.

Judges will score with their hearts as they always have done and will quite possibly move between catagories and that is fine. It is all about balancing the tables and reducing “tables of death”. But there is NO perfect system…there will still be tables that become mathematical outliers no matter how hard we try. This will reduce drastically but NEVER eliminate bad tables…the only way to ensure that bad tables are forevermore eliminated would be to have 1 table of six judges judge EVERY entry.

It will just bring balance to the force.

Quick observations:

1) I don't know if data from 10 yrs back exists, and if it's on cards the data input costs would be sizable.

2) In that 10 yr. period you have several different methods involved. Start at 9 and score down, start at 6, etc...

3) There has also been at least one change to weighting that I know of.

4) I'm not comfortable with the methodology of averaging to arrive at the mean. I'm not ruling it out, but I'm not comfortable with it. I'm more comfortable with a relational model to identify consistency and use that to balance the tables.

TooSaucedToPork
11-05-2013, 09:44 AM
Forgot about the weight change...that would totally throw the math.

I'm happy the rest is possible

carlyle
11-05-2013, 10:20 AM
Great conversation!

Hope KCBS is listening.

Would not like to see KCBS take over the judges. Right now, our Reps and I know the judges, for the most part, better than KCBS does.

Really like the color coded idea. We have to use the data we are now getting to make the judging process more fair. And give judges feedback about how they stack up relative to other judges.

TooSaucedToPork
11-05-2013, 10:37 AM
And give judges feedback about how they stack up relative to other judges.

This I wouldn't do...this throws off the math. Every judge would be worried if they are judging entries "correctly"

We don't want 6 average judges...we want 6 balanced judges
judging by the rules, not by fear of being an outlier. That might be a six, but they give it a seven to ensure they aren't a low judge.

Judges never need to see their rank...that inhibits balance in the force.

Dan - 3eyzbbq
11-05-2013, 10:39 AM
Is this really fixable? We are talking humans after all. That awesome judge that is never part of a table of death can get sick and not taste sh*t. Suddenly that judge that knows BBQ can't taste it. Should he pull out of judging, sure, but would he if its say, the Jack?

I just pulled one variable out as an example, there are many. There will always be hot and cold tables IMO.

Jorge
11-05-2013, 10:42 AM
Really like the color coded idea. We have to use the data we are now getting to make the judging process more fair. And give judges feedback about how they stack up relative to other judges.

I don't know. Is everything a 7, 8, or 9 these days? Is what used to be an 8, really a 7 as cooks have become so much better and it has become harder to separate entries based on merit and the existing system? If we have judges going home thinking about two entries that they scored the same, but believing one was better than the other it suggests we may need to address something.

I thought about composing an email to the board this morning with some suggestions, including members I know of that have the background and experience to look at the data (excluding myself!:becky: Work is keeping me plenty busy, and I'm still not done with the family business and other issues that required me to resign from the board).

I think the best solution is probably to put together a proposal and take it to KU, K State, etc. and unleash some grad students with reasonable compensation. It takes it off the plate of the board which doesn't have the time. It should receive ample peer and faculty review if the proposal is complete. It brings in a fresh set of eyes which is hopefully free of bias.

Who wants to set the O/U on how long it takes cooks to bitch and/or judges if the board actually does it?

TooSaucedToPork
11-05-2013, 10:46 AM
Is this really fixable? We are talking humans after all. That awesome judge that is never part of a table of death can get sick and not taste sh*t. Suddenly that judge that knows BBQ can't taste it. Should he pull out of judging, sure, but would he if its say, the Jack?

I just pulled one variable out as an example, there are many. There will always be hot and cold tables IMO.

Like I said...

It is all about balancing the tables and reducing “tables of death”. But there is NO perfect system…there will still be tables that become mathematical outliers no matter how hard we try. This will reduce drastically but NEVER eliminate bad tables… the only way to ensure that bad tables are forevermore eliminated would be to have 1 table of six judges judge EVERY entry.

It will just bring balance to the force.

landarc
11-05-2013, 10:56 AM
I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

goodsmokebbq
11-05-2013, 11:01 AM
The underlying problem is that the number of judges at the table (6) does not reasonably ensure a representative sample of judges. You want a distribution of judges that represents the population at large, a normal distribution, maybe one high, one low and a few average. You know, the good old bell shaped curve. With a large enough sample size you can be reasonably assured that you have a good representative sample. That number depends on the population distribution, but a good rule of thumb is usually around 12 or so. Pretty hard to implement, but it would almost guarantee equality of tables.

We can after the fact “pretend” that the judges are from the same population and adjust the scores for individual table / judge bias. This is pretty standard in the data analysis world, when we are compensating for some bias, particularly when dealing with humans. Not sure how teams would respond to having their scores changed after the fact, but it would accomplish the goal of normalizing the tables / judges.

QN
11-05-2013, 11:03 AM
There were four reps present, not sure why the website did not reflect that.

I would think it would be entirely possible for the current data being gathered to be applied to establishing a "handicap" or relevance score for judges, and for the judges' seating to be presorted accordingly. It would take some diligence in sign-ups (never a bad thing), and would still be subject to the occasional no-show, but as a cook I'd be willing to give it a spin and see how things shake out. It would provide much-needed consistency in the application of judge sorting across contests and regions.

There were actually a total of six KCBS contest reps working this contest; Randy and Carol Bilgler, Wayne and Maria Lohman, and Bill and Debbie Gage. They are all very experienced and they spent quite a bit of time sorting out the judges to try to balance the tables with new, experienced, master judges, etc.

Jorge
11-05-2013, 11:04 AM
I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

That would actually have the effect of potentially creating more of those tables, or more tables that score well above the mean. If boxes are shuffled as they are supposed to be you increase the chance of a team or teams to actually land on those tables more often. Everybody should have the same opportunity, via chance, to land on the same good or bad table. If you change the makeup of the table that chance is gone.

Sawdustguy
11-05-2013, 11:06 AM
I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

I like that idea!!

boogiesnap
11-05-2013, 11:13 AM
that is a good idea.

goodsmokebbq
11-05-2013, 11:15 AM
I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

You could just as easily create a new TOD for each category

TooSaucedToPork
11-05-2013, 11:15 AM
I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

Problem...ensuring you don't create chaos. Both on the scoring, the odds, and lastly in the judges tent.

Half hour between entries, and you have to assign seats 4 times, ensuring each time that the same people do not sit together more than once.

Alexa RnQ
11-05-2013, 11:15 AM
And somebody's gonna complain that they have to move their cooler.

*runs like the wind*

dhuffjr
11-05-2013, 11:21 AM
Who wants to set the O/U on how long it takes cooks to bitch and/or judges if the board actually does it?
Someone will complain. Especially if they don't do as well as they think they should or typically do. It is in our nature :becky:.

goodsmokebbq
11-05-2013, 11:23 AM
Problem...ensuring you don't create chaos. Both on the scoring, the odds, and lastly in the judges tent.

Half hour between entries, and you have to assign seats 4 times, ensuring each time that the same people do not sit together more than once.

The implementation would be very difficult.

What if my team hits the TOD for chicken, then you reshuffle and then I hit the TOD for ribs, reshuffle and I hit a TOD for pork... At least the reps can move the teams to different tables now to avoid hitting it twice.

TooSaucedToPork
11-05-2013, 11:26 AM
The implementation would be very difficult.

What if my team hits the TOD for chicken, then you reshuffle and then I hit the TOD for ribs, reshuffle and I hit a TOD for pork... At least the reps can move the teams to different tables now to avoid hitting it twice.

Exactly...moving judges is not a good solution. It creates more variables than we currently have in the judging process.

Jorge
11-05-2013, 11:26 AM
I like that idea!!

Put on your EE hat and think about that for a minute;) Hope all is well, Guy!

DanaK6JQ
11-05-2013, 11:26 AM
Let's look at the numbers Matt gave us a little closer. First of all, entries are given a point score, they're not ranked. So someone doesn't score 20th place - they score 164.000, for example. Let's look at how Table 10 scored the best entries they had.

According to Matt, the highest scoring entries at Laughlin Table 10 were:

Meat Overall Meat Score
Chicken 47th 164.0000
Ribs 48th 162.8228
Pork 41st 164.5600
Brisket 53rd 157.6800

Those *scores* don't look really insane; with the exception of brisket, they average above all 8s and the brisket score is mostly 8s. The best meats at table 10 were scored 8s or higher.

Let's take it another step: without mentioning team names, let's see how those teams did in other meats:

Team Score/Place (chicken/ribs/pork/brisket)
"47th Chicken" 47,164.000 45,163.4172 69,155.9428 79,142.24
"48th Ribs" 68,159.4284 48,162.8228 59,158.8344 22,167.4172
"41st Pork" 35,167.9656 43,163.9772 41,164.5600 27,165.7028
"53rd Brisk" 21,170.8800 33,165.6800 3,177.1772, 53,157.6800

This is a lot of concentrated data and maybe a little hard to read, but it says a lot. That 47th chicken team had lower _scores_ in every other meat and chicken was their second-best placement.

The 48th ribs team had lower scores in two other meats and ribs was their 2nd-best placement.

The 41st pork team had very consistent scores in the mid-160s and place lower in chicken.

The 53rd brisket team scored better in every other meat; brisket was their lowest score. Maybe Table 10 hates brisket. Maybe the team had a bad cook.

In other words, while it's easy to say that Table 10 was a Table Of Death and want to talk to those judges, it looks like all the other tables, for the most part, were scoring the other entries from these teams pretty similarly.

When you have 85 entries, placing near the middle of a category is gonna put you in 45th place or so.

Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.

Podge
11-05-2013, 11:46 AM
How about no contests over 24 teams. Everybody hits the bad table. Increase the entry to $400-$500. That way, total take in for entries would still equal a 48 team contest for the organizer..

big matt
11-05-2013, 11:54 AM
Let's look at the numbers Matt gave us a little closer. First of all, entries are given a point score, they're not ranked. So someone doesn't score 20th place - they score 164.000, for example. Let's look at how Table 10 scored the best entries they had.

According to Matt, the highest scoring entries at Laughlin Table 10 were:

Meat Overall Meat Score
Chicken 47th 164.0000
Ribs 48th 162.8228
Pork 41st 164.5600
Brisket 53rd 157.6800

Those *scores* don't look really insane; with the exception of brisket, they average above all 8s and the brisket score is mostly 8s. The best meats at table 10 were scored 8s or higher.

Let's take it another step: without mentioning team names, let's see how those teams did in other meats:

Team Score/Place (chicken/ribs/pork/brisket)
"47th Chicken" 47,164.000 45,163.4172 69,155.9428 79,142.24
"48th Ribs" 68,159.4284 48,162.8228 59,158.8344 22,167.4172
"41st Pork" 35,167.9656 43,163.9772 41,164.5600 27,165.7028
"53rd Brisk" 21,170.8800 33,165.6800 3,177.1772, 53,157.6800

This is a lot of concentrated data and maybe a little hard to read, but it says a lot. That 47th chicken team had lower _scores_ in every other meat and chicken was their second-best placement.

The 48th ribs team had lower scores in two other meats and ribs was their 2nd-best placement.

The 41st pork team had very consistent scores in the mid-160s and place lower in chicken.

The 53rd brisket team scored better in every other meat; brisket was their lowest score. Maybe Table 10 hates brisket. Maybe the team had a bad cook.

In other words, while it's easy to say that Table 10 was a Table Of Death and want to talk to those judges, it looks like all the other tables, for the most part, were scoring the other entries from these teams pretty similarly.

When you have 85 entries, placing near the middle of a category is gonna put you in 45th place or so.

Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.your missing the point..anybody that hit table 10 was done regardless of where there other meats finished..it was the worst table of death I've seen since the new data was released..and again I'm not bashing ANYBODY!..there is a problem that is staring us right in the face.its about fixing it not slamming anyone..and never once did I mention "bad" judges.

TooSaucedToPork
11-05-2013, 11:59 AM
Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.


It's not "bad" judges that are the main problem...

It's judges that score too high, and judges that score too low...they are not bad, they are just not average. Most judges are doing their honest best, even the ones that score high and low. By balancing the tables according to a judges scoring trend you take away a chunk of the TOD risk.

Equations have to balance...by setting all tables up with the same makeup of judges, truly excellent entries will prevail and truly average or bad entries will not rise to the top.

Assigning tables by scoring takes away a bit of the luck and replaces it with consistency.And as everyone knows we cooks all strive for consistency.

goodsmokebbq
11-05-2013, 12:08 PM
Procedure for normalizing the tables after the fact:

Calculate the overall mean score for a category for all tables (for example 160)

Calculate the mean score for each category on each table (table 1 170, table 2 150)

Create a table bias factor for each category and table (table 1 bias = -10, table 2 bias= +10)

Adjust a team’s score plus or minus based on which tables they hit. A team the scored 140 on table 2 would be biased up 10 points because table 2 was “on average” 10 points lower.

The only problem is that the average for the table is not very accurate, again because there are only 6 data points for the category. Taking a mean across all the categories for each table would help IF the table is consistent across categories (correlated). But the couple cases I have looked at suggest that there isn’t a large correlation between the categories for a given table. Being low on chicken doesn’t necessarily mean that the table will be low on the other categories (exceptions exist).

big matt
11-05-2013, 12:19 PM
Someone will complain. Especially if they don't do as well as they think they should or typically do. It is in our nature :becky:.Have you hit a table of death while in competition?

DanaK6JQ
11-05-2013, 12:24 PM
It's not "bad" judges that are the main problem...


Interesting point - is there actually a problem? By looking at the broader set of numbers from Laughlin, it isn't objectively clear there is a problem. No matter how hard you try to "even out" the tables, there will always be a low-scoring table, that's the way it works.

The system already favors higher scores by tossing out the lowest-scoring judge at a table. Perhaps that needs to be improved to toss out the highest-scoring judge at the table, also. That would achieve the same result - more "average" scoring - without having to keep track of thousands of judges over the years.

Better yet, toss out the top 2 and bottom 2 scores from each table and just use the middle 2 scores :twisted:

DawgPhan
11-05-2013, 12:25 PM
This is completely half baked idea without the least bit of effort into the math of it, but what if instead of worrying about the judges we just gave a little more wiggle room.

instead of getting judged by 6, include 7 or even 8 pieces in the box. Now you get judged by 7 judges and you drop the 2 lowest. That means that it now takes 3 really bad judges to impact your score.

7 pieces in the box, 7 judges at a table, 7 boxes on the table and now to piggy back on podge's idea, 28 teams to a contest. If you want more teams, run 2 contests on a weekend.

I still think that great reps and organizers are really the best defense against poor judging and trying to create a formula that takes the decision away from them is a bad idea.

dhuffjr
11-05-2013, 12:26 PM
Have you hit a table of death while in competition?
More than likely.

Pretty sure I complained about it too.

big matt
11-05-2013, 12:32 PM
Interesting point - is there actually a problem? By looking at the broader set of numbers from Laughlin, it isn't objectively clear there is a problem. No matter how hard you try to "even out" the tables, there will always be a low-scoring table, that's the way it works.

The system already favors higher scores by tossing out the lowest-scoring judge at a table. Perhaps that needs to be improved to toss out the highest-scoring judge at the table, also. That would achieve the same result - more "average" scoring - without having to keep track of thousands of judges over the years.

Better yet, toss out the top 2 and bottom 2 scores from each table and just use the middle 2 scores :twisted:

Laughlin is a glaring example and this isn't the first time it's happened..it happened at the Royal,Woodlake.ect ect..it hasn't happened at every contest but it happens..I'm sure you think that I started this thread because we didn't fair well..the bottom line is that hitting the table of death ends your day no matter how good your food was or wasn't..that's a problem to me..shouldn't our scores reflect how our food was not if we hit the TOD?

DanaK6JQ
11-05-2013, 12:35 PM
your missing the point..anybody that hit table 10 was done regardless of where there other meats finished..it was the worst table of death I've seen since the new data was released..and again I'm not bashing ANYBODY!..there is a problem that is staring us right in the face.its about fixing it not slamming anyone..and never once did I mention "bad" judges.

Here's something to think about - a table is not just 6 judges. It is 6 judges and 6 entries. The food has more to do with the scores than the judges do.

It's easy to think of a "Table Of Death" as a trap waiting to happen to even the best food in a contest, but that's not what the numbers from Laughlin suggest happened.

big matt
11-05-2013, 12:35 PM
More than likely.

Pretty sure I complained about it too.

Why wouldn't you?..it's this simple to me..if you hit the TOD your day is done as far as GC goes..don't hit it and you've got a shot if you cooked well..and yes I have the sheets to prove it.

goodsmokebbq
11-05-2013, 12:39 PM
The goal of the judging is to estimate the population mean (all the judges in the world) given the sample mean (one table).

Lets look at an example. Lets say the I am competing heads up against Johnny Trigg in ribs (my personal nightmare). In a perfect world to find out who has the best ribs we would be judged by every judge in the world, the largest population possible. The average would be very precise, in fact perfect. We call this the population mean or true mean. Clearly Trigg’s average would be higher than mine and he would win J

Obviously this is impractical and I don’t want to cook that much food J The great thing about statistics is that we can come up with a pretty good estimate (sample mean) of the population mean by taking a random sample (subset of judges) from the population. The mean of a random sample of a population will be equal to the population mean given enough samples. So if we pick enough judges we should be able to approach the population mean for each table. If every table is close to the population mean than they would all be close to each other. Problem is we only choose 6 which is far lower than what statistics say we should be using.

We can view the seating of judges at tables as a random(ish) sample of the judging pool for any given contest. With only six choices per table the odds are pretty high that a number of the tables will be unbalanced.

So let’s say that I land on the random table of high judges and poor Johnny lands on the TOD then I get to brag until I am old about beating Trigg at ribs!

There is really no way around the statistics except to increase the number of judges at each table. Rule of thumb in these things is 12, closer analysis could find the right number. Not realistic I understand, but maybe an increase to 7 or 8 judges could have a real impact while still being somewhat achievable.

big matt
11-05-2013, 12:40 PM
Here's something to think about - a table is not just 6 judges. It is 6 judges and 6 entries. The food has more to do with the scores than the judges do.

It's easy to think of a "Table Of Death" your as a trap waiting to happen to even the best food in a contest, but that's not what the numbers from Laughlin suggest happened.

The numbers and score sheet in front of me say one thing..if you hit table 10 your day was done..it's that simple

landarc
11-05-2013, 12:44 PM
That would actually have the effect of potentially creating more of those tables, or more tables that score well above the mean. If boxes are shuffled as they are supposed to be you increase the chance of a team or teams to actually land on those tables more often. Everybody should have the same opportunity, via chance, to land on the same good or bad table. If you change the makeup of the table that chance is gone.
I understand that the current system is meant to do exactly that, which means nobody ends up at the TOD more than once, and my system randomizes. I just put that out there, as something simpler than trying to normalize scoring after judging, or trying to track judging across many competitions.

To the idea that everyone is going to complain when they don't do well, I can't speak to all of them. But, I have followed Big Matt out west here, and while he doesn't fear conflict, the man knows his cooking, if he tells me he feels his entry wasn't judged fairly, I am inclined to believe him, based on the fact that when he wasn't hitting on all cylinders, he stated so plainly and never accepts excuses.

DanaK6JQ
11-05-2013, 12:47 PM
Laughlin is a glaring example and this isn't the first time it's happened..it happened at the Royal,Woodlake.ect ect..it hasn't happened at every contest but it happens..I'm sure you think that I started this thread because we didn't fair well..the bottom line is that hitting the table of death ends your day no matter how good your food was or wasn't..that's a problem to me..shouldn't our scores reflect how our food was not if we hit the TOD?

First of all, I didn't realize that you'd hit Table 10, I wasn't thinking about that at all. Sorry to hear you didn't fare well, Matt - getting low numbers from the judges always stings.

Are you suggesting the Table of Death is just something that happens regardless of how good the food is? That 6 judges walk in to the tent and sit down at a table and don't much like BBQ that day?

big matt
11-05-2013, 12:49 PM
I understand that the current system is meant to do exactly that, which means nobody ends up at the TOD more than once, and my system randomizes. I just put that out there, as something simpler than trying to normalize scoring after judging, or trying to track judging across many competitions.

To the idea that everyone is going to complain when they don't do well, I can't speak to all of them. But, I have followed Big Matt out west here, and while he doesn't fear conflict, the man knows his cooking, if he tells me he feels his entry wasn't judged fairly, I am inclined to believe him, based on the fact that when he wasn't hitting on all cylinders, he stated so plainly and never accepts excuses.

Thank you Robert for understanding where I'm coming from..our ribs hit that table and were 63rd..were they 63rd place ribs in my book..no but it is what it is..the problem I see is that anyone that hit that table was done before the box was ever opened

big matt
11-05-2013, 12:56 PM
First of all, I didn't realize that you'd hit Table 10, I wasn't thinking about that at all. Sorry to hear you didn't fare well, Matt - getting low numbers from the judges always stings.

Are you suggesting the Table of Death is just something that happens regardless of how good the food is? That 6 judges walk in to the tent and sit down at a table and don't much like BBQ that day?

Yes that's exactly what has happened more than once too..we've won 3 times since the new sheets have came out..each and every time we won we luckily avoided the TOD..each time we hit theTOD we weren't even close in the running..so that's my theory and I've got the score sheets to prove it..hit the TOD and your day is done..shouldn't it be if your food isn't good your done?

TooSaucedToPork
11-05-2013, 01:02 PM
Some of you will deny the existence of a TOD with your last breath. It's your right and that's fine. It could be that in your area of the country they don't happen often, or you have yet to land on one. I have been cooking for a very long time, and they exist. I've been seated at one, my food has landed on one. Not just in KCBS, they exist in every organization that I have cooked under.

Every effort should be given and every possible way should be investigated by KCBS to make the playing field as even as possible. As many aspects of luck as possible should be pulled from the judging process. Does his mean an Inquisition of the judges...No. It means find a way to ensure that EACH judging table is as balanced as humanly possible.

Has that been achieved, you may yes...I believe it hasn't.

big matt
11-05-2013, 01:06 PM
Some of you will deny the existence of a TOD with your last breath. It's your right and that's fine. It could be that in your area of the country they don't happen often, or you have yet to land on one. I have been cooking for a very long time, and they exist. I've been seated at one, my food has landed on one. Not just in KCBS, they exist in every organization that I have cooked under.

Every effort should be given and every possible way should be investigated by KCBS to make the playing field as even as possible. As many aspects of luck as possible should be pulled from the judging process. Does his mean an Inquisition of the judges...No. It means find a way to ensure that EACH judging table is as balanced as humanly possible.

Has that been achieved, you may yes...I believe it hasn't.

Took the words right from my mouth

Podge
11-05-2013, 01:22 PM
I like the 8 judge per table idea. drop the two bottom scores and the highest score.

roksmith
11-05-2013, 01:46 PM
We're gonna need a bigger box :-o

landarc
11-05-2013, 02:11 PM
how many judges does it take to create a TOD? I mean, if one is scoring off, then I can't see it being an issue. But, with the current scoring system, could it be as few as 2, or 3?

The_Kapn
11-05-2013, 02:28 PM
Back in post #21 Rub said that FBA scoresheets have had this info for years.
TRUE.

Before we retired from comps, and I retired from Judging, we came to accept that there was really nothing to be done.
Judges are volunteers and no one seems to want to "rock the boat".
Even a clearly out of range (high or low) judge was seldom, if ever, passed over for future judging.
At one contest, a judge gave 10-7.5-7.5 to each and every entry, except one.
Since FBA counts all scores, and Taste/Tenderness are rated higher than "Appearance/Presentation" he was the "JUdge of Death"!

I spoke with the lead REP, a really good one, and he said nothing could be done because of all the 10's for appearance.

Welcome to the "info age" or "data age" KCBS cooks. 8)

TIM

goodsmokebbq
11-05-2013, 02:28 PM
I like the 8 judge per table idea. drop the two bottom scores and the highest score.

I also don't think we should be dropping any scores either, all it does it make the average scores higher. And on average the higher scoring teams are losing more points than the lower scoring teams so it actually disproportionately helps the lower scoring teams.

If my drop score is a 9 9 9 I lose 36 points. if another team's drop score is 6 6 6 then they lose only 24 points.

Meat Man
11-05-2013, 02:38 PM
We've had the misfortune of hitting the TOD twice this year. It's difficult when teams put so much effort and expense into a comp, and come away after awards feeling badly. Especially when its so obvious. On the other hand we have probably benefited from over generous tables (but I can't think of one) :-D. I realize that the judges are human, and there will always be differences in what someone considers to be fair, good or excellent BBQ. But It would just be better for everyone if the element of luck or unluck didn't play a role in competition BBQ. Maybe a simpler scoring system is the answer.

TooSaucedToPork
11-05-2013, 02:38 PM
I also don't think we should be dropping any scores either, all it does it make the average scores higher. And on average the higher scoring teams are losing more points than the lower scoring teams so it actually disproportionately helps the lower scoring teams.

If my drop score is a 9 9 9 I lose 36 points. if another team's drop score is 6 6 6 then they lose only 24 points.

I disagree...that is there to eliminate the low judge at the table...the outlier. You need that dropped for fairness.

You still get your 180 brother ;-)

The_Kapn
11-05-2013, 02:44 PM
I also don't think we should be dropping any scores either, all it does it make the average scores higher. And on average the higher scoring teams are losing more points than the lower scoring teams so it actually disproportionately helps the lower scoring teams.

If my drop score is a 9 9 9 I lose 36 points. if another team's drop score is 6 6 6 then they lose only 24 points.

Agree!

http://www.bbq-brethren.com/forum/showthread.php?t=96301&highlight=scores

What folks don't understand is that for every place a team moves UP, due to the dropped score, another team moves DOWN one.
If a team moves UP 5 places due to dropped scores, some combination of other teams must move DOWN 5 places.

Just depends on how good your stuff really was to all 6 Judges, not just 5!8)

TIM

ShencoSmoke
11-05-2013, 02:46 PM
how many judges does it take to create a TOD? I mean, if one is scoring off, then I can't see it being an issue. But, with the current scoring system, could it be as few as 2, or 3?

Use this as an example (ribs). 2 decent scores, 4 bad scores, and the 4 bad scores all basically agreed which eachother. And for more perspective, here is how this particular table judged in all 4 categories:

Table scores out of 37 teams:
Chicken: 17,23,25,35,36,37
Ribs: 6,20,21,22,34,36,37
Pork: 27,28,31,32,33,36
Brisket: 8,11,26,33,36,37

87160

Our ribs are not our best category but have averaged roughly around 10th place all year, and we liked them at this contest. At the end of the day the best team won, they also hit this table. I just thought it is textbook TOD example.

Podge
11-05-2013, 03:12 PM
If I had 4 bad scores that agreed with each other, that'd make me think twice about my ribs.

Dan - 3eyzbbq
11-05-2013, 03:20 PM
I disagree...that is there to eliminate the low judge at the table...the outlier. You need that dropped for fairness.



Outliers are on both sides. If you drop the low, you should drop the high as well. He is right IMO.

TooSaucedToPork
11-05-2013, 03:33 PM
Use this as an example (ribs). 2 decent scores, 4 bad scores, and the 4 bad scores all basically agreed which eachother. And for more perspective, here is how this particular table judged in all 4 categories:

Table scores out of 37 teams:
Chicken: 17,23,25,35,36,37
Ribs: 6,20,21,22,34,36,37
Pork: 27,28,31,32,33,36
Brisket: 8,11,26,33,36,37

87160

Our ribs are not our best category but have averaged roughly around 10th place all year, and we liked them at this contest. At the end of the day the best team won, they also hit this table. I just thought it is textbook TOD example.

I'd love to say that was a TOD...

That is not a TOD. You ranked 5th on the table with those scores, if you ranked 1st on it, and got those scores THAT would be a TOD.

Don't hate, I'm just being honest...that just looks like a bad day.

TooSaucedToPork
11-05-2013, 03:40 PM
Outliers are on both sides. If you drop the low, you should drop the high as well. He is right IMO.

For true fairness and equality yes...you are correct

DUBBAGA
11-05-2013, 03:42 PM
Our ribs are not our best category but have averaged roughly around 10th place all year, and we liked them at this contest. At the end of the day the best team won, they also hit this table. I just thought it is textbook TOD example.

Although the results do show that Table 1 was a low scoring table, ultimately your ribs were still only the 5th best at the table.

My interest lies with Judge 5 at Table 6. This particular judge's average score was 29.6649. This score would normally place him/her at the TOD, correct? However, this same judge gave you 9-9-8 for your pork, for a total score of 34.8572.

The average of all 6 scores for the ribs at the TOD was 29.51, and Judge 5 at Table 6 averaged 29.6649, yet this judge still gave you a very good score on pork. What shall we do with this judge? He/she clearly is capable of assigning high scores where believed appropriate, so is this a JOD (judge of death)?

(having said that, I am worried for the safety of Judge 5 at Table 1... that one is a JOD !!!)

TooSaucedToPork
11-05-2013, 03:49 PM
Use this as an example (ribs). 2 decent scores, 4 bad scores, and the 4 bad scores all basically agreed which eachother. And for more perspective, here is how this particular table judged in all 4 categories:

Table scores out of 37 teams:
Chicken: 17,23,25,35,36,37
Ribs: 6,20,21,22,34,36,37
Pork: 27,28,31,32,33,36
Brisket: 8,11,26,33,36,37

87160

Our ribs are not our best category but have averaged roughly around 10th place all year, and we liked them at this contest. At the end of the day the best team won, they also hit this table. I just thought it is textbook TOD example.

Your chicken landed on a table that was scoring higher than the average...not to rain on the chicken parade. ;-)

ShencoSmoke
11-05-2013, 04:05 PM
My agenda here was to put an example out there of a table that scored poorly in all four categories and to show that in my situation dropping more than 1 score would not have made a difference. That contest is water under the bridge.

big matt
11-05-2013, 04:12 PM
I think most of us here that cook contests and have felt the sting of a TOD can understand one table scoring low on one meat..that happens all the time and I'm fine with that..it's when one table absolutely kills all 4 meats like what happened this weekend..also this same table scored 3 of the 4 meats DAL and were second to DAL on the other..and like what was posted above earlier this has been happening for a long time I think it's the fact we can now see it plain as day makes it hurt that much more.

DanaK6JQ
11-05-2013, 04:14 PM
My agenda here was to put an example out there of a table that scored poorly in all four categories and to show that in my situation dropping more than 1 score would not have made a difference. That contest is water under the bridge.

Not to rub salt into an old wound, but my question is - how can someone tell from this scoresheet whether you had a Table of Death (that unfairly gave your entry low scores) or your entry would have scored similarly at other tables?

That's the crux of the Table of Death question.

TooSaucedToPork
11-05-2013, 04:18 PM
Understood.

A table of death will have low average scores compared to the other judges at a contest.
If you see this:
28.4, 27.3, 26.6, 26.9, 30.7, 27.4

AND THE AVERAGE is a 29.7

You hit a TOD. You never had a chance

big matt
11-05-2013, 04:23 PM
Not to rub salt into an old wound, but my question is - how can someone tell from this scoresheet whether you had a Table of Death (that unfairly gave your entry low scores) or your entry would have scored similarly at other tables?

That's the crux of the Table of Death question.

Haha rubbing salt in an old wound?..they should call you Morton because your a master at keeping things "salty"..Dana why is this topic so important to you anyway?..you only cook a few comps a year and never chime in here but yet this thread has your full attention..Have you even hit a table of death?..my guess would be no or else you'd see the cooks side.

DanaK6JQ
11-05-2013, 04:27 PM
Understood.

A table of death will have low average scores compared to the other judges at a contest.
If you see this:
28.4, 27.3, 26.6, 26.9, 30.7, 27.4

AND THE AVERAGE is a 29.7

You hit a TOD. You never had a chance

The tables didn't get the same entries. You can't compare different tables.

ShencoSmoke
11-05-2013, 04:31 PM
Not to rub salt into an old wound, but my question is - how can someone tell from this scoresheet whether you had a Table of Death (that unfairly gave your entry low scores) or your entry would have scored similarly at other tables?

That's the crux of the Table of Death question.

1) the performance of the table in all 4 categories (which I posted)
2) the fact that I've cooked enough contests to know the difference between good ribs and DAL ribs

TooSaucedToPork
11-05-2013, 04:33 PM
The tables didn't get the same entries. You can't compare different tables.

WHAT?
Explain

big matt
11-05-2013, 04:36 PM
The tables didn't get the same entries. You can't compare different tables.

Again since your such the authority on scoring systems how about a solution..instead of denying a problem exists..I just wonder why a guy that hardly cooks comps is so concerned with this topic..have you even been affected by this change

DanaK6JQ
11-05-2013, 04:39 PM
Haha rubbing salt in an old wound?..they should call you Morton because your a master at keeping things "salty"..Dana why is this topic so important to you anyway?..you only cook a few comps a year and never chime in here but yet this thread has your full attention..Have you even hit a table of death?..my guess would be no or else you'd see the cooks side.

Well, sure, I did 6 contests this year. So what? You're saying I'm not a cook? Just because I'm not entirely agreeing with you, you start some sort of personal attack? Say that I don't see the "cooks side"? No thanks, Matt.

I'm sticking the objective facts here. Just because a table scores entries in the bottom half of the field doesn't tell us whether the judges were especially tough or whether the food wasn't especially great. Even great teams have bad cooks.

This topic interests me. I'm not half-bad applying facts, logic and mathematics.

Pappy Q
11-05-2013, 04:39 PM
Not to rub salt into an old wound, but my question is - how can someone tell from this scoresheet whether you had a Table of Death (that unfairly gave your entry low scores) or your entry would have scored similarly at other tables?

That's the crux of the Table of Death question.

I track the number of top 10 calls each table gets for the entire contest. When a table gets zero top 10 calls your significantly lower than the others, then I consider it a TOD. The top 3-5 teams never hit the TOD.

DanaK6JQ
11-05-2013, 04:42 PM
Again since your such the authority on scoring systems how about a solution..instead of denying a problem exists..I just wonder why a guy that hardly cooks comps is so concerned with this topic..have you even been affected by this change

It's logic, Matt. Perhaps you'd be kind enough step off the personal attack and let's think about it.

Table 1 gets 6 entries. The average score is 170.
Table 2 gets 6 different entries. The average score is 148.

What can you conclude from that?

DanaK6JQ
11-05-2013, 04:47 PM
WHAT?
Explain

It's possible I misunderstood what you wrote:

A table of death will have low average scores compared to the other judges at a contest.

I interpreted this to mean a ToD will have low averages scores compared to other judges at a contest, meaning, at other tables.

Did you mean to say a ToD will have judges that score the same entries at the same table with widely different scores?

big matt
11-05-2013, 04:47 PM
Well, sure, I did 6 contests this year. So what? You're saying I'm not a cook? Just because I'm not entirely agreeing with you, you start some sort of personal attack? Say that I don't see the "cooks side"? No thanks, Matt.

I'm sticking the objective facts here. Just because a table scores entries in the bottom half of the field doesn't tell us whether the judges were especially tough or whether the food wasn't especially great. Even great teams have bad cooks.

This topic interests me. I'm not half-bad applying facts, logic and mathematics.

Oh Dana here you go spinning it into an attack..come on man just admit you love to argue(debate)I never said your not a cook but it's obvious you've never hit a TOD..or you'd see my side

TooSaucedToPork
11-05-2013, 04:52 PM
It's logic, Matt. Perhaps you'd be kind enough step off the personal attack and let's think about it.

Table 1 gets 6 entries. The average score is 170.
Table 2 gets 6 different entries. The average score is 148.

What can you conclude from that?

Your logic is skewed. You are not looking at all the information.

Low average scores plus no calls from a table plus teams that normally walk getting thrashed on that table...

Equals...(que drumroll)

Table of Death...ba ba bum

big matt
11-05-2013, 04:52 PM
It's logic, Matt. Perhaps you'd be kind enough step off the personal attack and let's think about it.

Table 1 gets 6 entries. The average score is 170.
Table 2 gets 6 different entries. The average score is 148.

What can you conclude from that?

Again do you have a solution?..what's your theory?

DanaK6JQ
11-05-2013, 05:03 PM
Oh Dana here you go spinning it into an attack..come on man just admit you love to argue(debate)I never said your not a cook but it's obvious you've never hit a TOD..or you'd see my side

Rubstock '13, Brisket, Table 1/Judges 5&6 tended to score below the other 4 judges in every category - probably robbed us of RGC (1.1 points in the total). I get it, I totally do.

big matt
11-05-2013, 05:09 PM
Rubstock '13, Brisket, Table 1/Judges 5&6 tended to score below the other 4 judges in every category - probably robbed us of RGC (1.1 points in the total). I get it, I totally do.
Ok then why the argument about this contest?..it's plain as day if you hit table 10 in Laughlin you were done..those are hard facts and that's what I'm going by

Smitty250
11-05-2013, 05:40 PM
I just went back and looked at my score sheet from Laughlin...I'm glad I didn't hit Table 10! If you can't see that this was the TOD then you are blind!

DanaK6JQ
11-05-2013, 05:42 PM
Your logic is skewed. You are not looking at all the information.

Low average scores plus no calls from a table plus teams that normally walk getting thrashed on that table...

Equals...(que drumroll)

Table of Death...ba ba bum

This is where I suppose I'm diverging from "the cook's side"... I'm not willing to assume the judges were wrong just because teams that normally walk don't score well that day. I'm not saying there *aren't* tables of death - I've seen 'em myself.

In fact, I think a Table Of Death is more like "where 2 or 3 judges score below the others" when they're all looking at and eating the same food. 1 judge... low score is tossed, no harm. 2 judges scoring below the rest of the table, that's probably bad.

Using the example of Rubstock '13 Table 1:

32.0 32.0 35.44 35.44 26.8572 (ridiculously low score dropped)

OK, admittedly, not my best brisket ever. But let's average the 4 higher scores - we get 33.72. So the included low score is 6.86 points lower - we missed RGC by 1.05 points.

If all of the judges at a table are scoring the same entries lower, I just have to shrug and accept the judges doing a good job. But when there's a couple of judges that don't share the table consensus, those are the ones I wonder about.

DanaK6JQ
11-05-2013, 05:48 PM
Ok then why the argument about this contest?..it's plain as day if you hit table 10 in Laughlin you were done..those are hard facts and that's what I'm going by

If every judge at a table is scoring similarly, then maybe it's the food they got, or maybe it's the judges, or both.

If a couple of judges at a table are scoring far below the others... now that's a Table Of Death. They're all looking at and eating the same food and 2 judges disagree with the other 4? Of course the same could be said of where 4 judges consistently score entry low and 2 judges score it high. That's like a Table of Resurrection, I suppose.

big matt
11-05-2013, 05:53 PM
This is where I suppose I'm diverging from "the cook's side"... I'm not willing to assume the judges were wrong just because teams that normally walk don't score well that day. I'm not saying there *aren't* tables of death - I've seen 'em myself.

In fact, I think a Table Of Death is more like "where 2 or 3 judges score below the others" when they're all looking at and eating the same food. 1 judge... low score is tossed, no harm. 2 judges scoring below the rest of the table, that's probably bad.

Using the example of Rubstock '13 Table 1:

32.0 32.0 35.44 35.44 26.8572 (ridiculously low score dropped)

OK, admittedly, not my best brisket ever. But let's average the 4 higher scores - we get 33.72. So the included low score is 6.86 points lower - we missed RGC by 1.05 points.

If all of the judges at a table are scoring the same entries lower, I just have to shrug and accept the judges doing a good job. But when there's a couple of judges that don't share the table consensus, those are the ones I wonder about.
You give the example of one meat..they scored every meat low not just one.

DanaK6JQ
11-05-2013, 06:22 PM
You give the example of one meat..they scored every meat low not just one.

If I could get the raw data in CSV form... :-)

Smoke'n Ice
11-05-2013, 06:43 PM
I would propose the following is a possible reason for the scoring at this contest:
1. Location - How many CBJ's in the area
2. Number of teams - Exceeded number 1 by a large number
3. CBJ's and MBJ's tend to congregate together at a contest
4. Newly trained judges - Taught that 6 is average and score up or down
5. Celeberty, off the street judges - Listen to the CD and learn that 6 is averge and score up or down
6. Not enough CBJ's - CBJ's used as TC at a table of off the street and newly minted
7. Some tables of pure CBJ's - Good (not) news (they start at 9)
8. Some tables of new CBJ's - Bad news (they start at 6)
9. Some tables of off the street - really bad news (who knows where they start)

I really see the above as a real scenerio for a contest of this size in an area not normally used for KCBS events. You pays your money, you takes your chances.

DUBBAGA
11-05-2013, 06:51 PM
If I could get the raw data in CSV form... :-)
http://chroniclesofthenerds.com/wp-content/uploads/2010/10/1385fdd0e0dcde8b419758d81f614431.jpg

JD McGee
11-05-2013, 07:07 PM
We were lucky enough to dodge table 10...but they still hated our brisket...:hand::becky:

DawgPhan
11-05-2013, 07:22 PM
I like the 8 judge per table idea. drop the two bottom scores and the highest score.


there is your winner. Combine this with a good faith effort from reps and organizers and you would have really great judging each weekend.

bover
11-05-2013, 07:36 PM
Keep in mind that the inverse happens as well. At the Mission, KS contest this year, three of the four category winners and a boatload of top 10s all came from the same table. I started referring to it as the cheesecake table because after looking at the scores, it looked like the judges at that table were scoring everything that came across their plates like it was cheesecake. In the end it didn't end up mattering all that much because the two best teams still got GC and RGC, but there's no doubt that some deserving folks were robbed of some ribbons. Just like many others have said, this type of thing has always been happening but now that KCBS has the means to track and analyze the data I really hope some changes can be made to at least reduce some of the disparity. I'll refer back to my earlier post (http://www.bbq-brethren.com/forum/showpost.php?p=2682072&postcount=33) for suggestions on how to do just that.

goodsmokebbq
11-05-2013, 08:16 PM
Keep in mind that the inverse happens as well. At the Mission, KS contest this year, three of the four category winners and a boatload of top 10s all came from the same table. I started referring to it as the cheesecake table because after looking at the scores, it looked like the judges at that table were scoring everything that came across their plates like it was cheesecake. In the end it didn't end up mattering all that much because the two best teams still got GC and RGC, but there's no doubt that some deserving folks were robbed of some ribbons. Just like many others have said, this type of thing has always been happening but now that KCBS has the means to track and analyze the data I really hope some changes can be made to at least reduce some of the disparity. I'll refer back to my earlier post (http://www.bbq-brethren.com/forum/showpost.php?p=2682072&postcount=33) for suggestions on how to do just that.


The judge database can work, implementation wouldn't be to hard. A couple extra steps before the event. The problem here is the assumption that judges are consistent. Is a low judge always a low judge? Is a high? This would make a good starting point for KCBS to analyze the issue, but it probably means creating the infrastructure to track individual judges.

DanaK6JQ
11-05-2013, 08:24 PM
The judge database can work, implementation wouldn't be to hard. A couple extra steps before the event. The problem here is the assumption that judges are consistent. Is a low judge always a low judge? Is a high? This would make a good starting point for KCBS to analyze the issue, but it probably means creating the infrastructure to track individual judges.

As far as I understand it, the new KCBS scoring program does exactly this with CBJs. After a few years of data gathering, they may be able to tell us if there are judges correlated with low-scoring tables. I have a sneaking suspicion there will be very few such judges.

McEvoy AZ
11-05-2013, 08:44 PM
After doing a quick review of the score sheet it seems that on table 10 that 2 judges seem to be always scoring low. We were right next to the judges tent and we could hear that there was at least one judge that was sick as he was asked if he could continue judging. I wonder if that could have had the sick judge on it. It would be hard to judge BBQ if you were sick.I also head them arranging judges based on experience so one sick judge plus a new judge who does not know how to score can kill a table.

Big Poppa
11-05-2013, 09:55 PM
Dana did you cook Laughlin?

djqualls
11-05-2013, 09:55 PM
As I said earlier, I hit table 10 and I understand that the argument can be made that maybe they had bad food but this table didn't have 24 entries spread across 4 categories. And I didn't have a bad brisket cook!


Look At my past eight contests in a row and tell me if you can find the TOD?

Contest Score Place Table Rank
Fayetville 180.0000 1 2 1
Sam's Local 173.1200 2 4 1
Royal Open 164.0000 149 5 1
Sam's Regional 173.7372 7 4 2
Sam's National 176.5944 6 3 1
Fort Smith 173.1200 3 3 1
Fort Gibson 179.4400 1 1 1
Laughlin 157.1200 53 10 1


It is what it is. It most likely cost me 1st place TOY in Brisket But who knows....

Big Poppa
11-05-2013, 10:03 PM
david only one guy doesnt think table ten was the table of death.

djqualls
11-05-2013, 10:13 PM
As I said earlier, I hit table 10 and I understand that the argument can be made that maybe they had bad food but this table didn't have 24 entries spread across 4 categories. And I didn't have a bad brisket cook!


Look At my past eight contests in a row and tell me if you can find the TOD?

Contest Score Place Table Rank
Fayetville 180.0000 1 2 1
Sam's Local 173.1200 2 4 1
Royal Open 164.0000 149 5 1
Sam's Regional 173.7372 7 4 2
Sam's National 176.5944 6 3 1
Fort Smith 173.1200 3 3 1
Fort Gibson 179.4400 1 1 1
Laughlin 157.1200 53 10 1


It is what it is. It most likely cost me 1st place TOY in Brisket But who knows....

Sorry I couldn't get it to format correctly I hope readers can decipher the info.

Yes Poppa, I understand only one disagrees but, I wanted to prove my point that I think I know if I'm having a good cook.

Last year in our first year and early in this season, I would drive home and wonder what I did wrong and work on changing up. Well, one critical part of the equation has been explained with the new score sheet. Even if a team is new, if they are able to see a TOD then they keep some confidence that they may have had a good entry. If it wasn't, they will figure out that maybe they need to cook better when they get the same score on a table that had winners all across the categories.

IMHO, I think the judges score report is as helpful as the individual score for new teams. I'm interested in creating new teams and retaining existing teams from being discouraged that they can't get a fair shake unless they even the odds by cooking 20+ contests to even out the odds of the TOD's.

djqualls
11-05-2013, 10:15 PM
I also think The "West Coast Offense" in itself can convert a TOD to a Winning Table!!! :eusa_clap:clap2::bow:

DanaK6JQ
11-06-2013, 12:10 AM
Dana did you cook Laughlin?

No, I did not.

DanaK6JQ
11-06-2013, 12:14 AM
david only one guy doesnt think table ten was the table of death.

Who is that one guy?

PorkQBBQ
11-06-2013, 12:35 AM
I don't think Math can help in this equation... Pun Intended :) Who's to say that your food is good or bad other than the person tasting it... Even in every/any category? It all comes down to whatever people consider good or bad at the time. Maybe most were from Memphis or Carolina at that table, maybe they all ate at McDonalds before judging, etc..Maybe you have always ended up on the TOL (Table of Luck). Alot of Maybe's.... I know it's only 1 table I feel for you and the teams. But it's the way the cookie crumbles. Hope you win big next time... Kinda like playing the lotto.

Crash
11-06-2013, 04:49 AM
Good read and great discussion. Some quality responses in support of amending the KCBS rules.

On another note, we seem to hit the TOD, in most categories at every single contest.

TooSaucedToPork
11-06-2013, 07:33 AM
No one is complaining about "bad judges"
We are not complaining about our food not scoring well.
We are not saying that we are perfect everytime on turnins
We are not making excuses about the food...

All of you that say JOD's and TOD's don't exist are not using logic and reason. Let me explain why you are mistaken.

Lets say The “Yardbird Sucks Open” is a BBQ contest in Barbecue NC… There are 49 teams at this contest…

Lets take a (make believe) random sampling of judges…48 of them (8 tables of 6)
34 of these judges usually score along the same lines…a 28-31 average
9 of these judges score above a 31 average
5 of these judges score below a 28 average

Reps don’t know how the judges score BBQ.

They can only separate Spouses, seat MCBJ’s at different tables, and spread out CBJ’s amongst the 8 tables.

A= AVERAGE
H=HIGH
L = LOW

T1 A,A,A,H,H,A
T2 A,A,A,L,A,A
T3 H,A,A,A,A,A
T4 A,L,A,A,L,L
T5 A,A,A,A,A,A
T6 A,A,A,H,A,A
T7 H,A,H,A,A,H
T8 A,A,H,A,A,L

T1 – Scores will be higher as 2 highs and 4 average are at the table. 1 average score will be dropped

T2 – Scores will be balanced as 1 low and 5 average are at the table. 1 low score will be dropped.

T3 – Scores will be higher as 1 high and 5 average are at the table. 1 average score will be dropped

T4 – THIS IS YOUR TABLE OF DEATH AT MOST CONTESTS - Scores will be lower as 3 low and 3 average are at the table. 1 low score will be dropped leaving 2 judges that naturally score low to kill your chance of winning before your box is opened.

T5 – Scores will be “average” as 6 average are at the table. 1 average score will be dropped.

T6 – Scores will be higher as 1 high and 5 average are at the table. 1 average score will be dropped

T7 – THIS IS YOUR TABLE OF ANGELS AT MOST CONTESTS - Scores will be higher as 3 high and 3 average are at the table. 1 average score will be dropped leaving 3 judges that naturally score high to vastly improve your chance of winning before your box is opened.

T8 – Scores will be higher as 1 high, 1 low and 4 average are at the table. 1 low score will be dropped.

Now …those of you that say math can’t help this…

Apply a formula that divides the judges average score for all contests judged into high, average and low groups…

There are now 32 judges that, over all contests judged, normally score between 28.67 and 30.89
There are 8 judges that that normally score below 28.66
There are 8 judges that normally score above 30.90

Your tables now look like this….
T1 H,A,A,A,A,L
T2 H,A,A,A,A,L
T3 H,A,A,A,A,L
T4 H,A,A,A,A,L
T5 H,A,A,A,A,L
T6 H,A,A,A,A,L
T7 H,A,A,A,A,L
T8 H,A,A,A,A,L

Notice a difference?

Now please…tell me math can’t help this problem…

Better yet, tell me this problem doesn’t exist

TooSaucedToPork
11-06-2013, 07:43 AM
A lot of Maybe's.... I know it's only 1 table I feel for you and the teams. But it's the way the cookie crumbles. Hope you win big next time... Kinda like playing the lotto.

And you just proved our point.

Take out the luck. Aka the "lotto"

QN
11-06-2013, 07:49 AM
Another thing that impacts all of this is that there seem to be many judges now only using the numbers 9, 8, and 7 to score all entries instead of the entire scale as instructed. If you have a table of judges where 3 or 4 out of 6 are trying to use the entire scale and the others are 9, 8, 7 judges the scores from this table will be lower than a table where most of the judges are using only 9. 8. 7 to score. Every entry from every good team will never always score as well as the cook thinks it should. The personal taste of each judge will always come into play.

big matt
11-06-2013, 09:13 AM
I don't think Math can help in this equation... Pun Intended :) Who's to say that your food is good or bad other than the person tasting it... Even in every/any category? It all comes down to whatever people consider good or bad at the time. Maybe most were from Memphis or Carolina at that table, maybe they all ate at McDonalds before judging, etc..Maybe you have always ended up on the TOL (Table of Luck). Alot of Maybe's.... I know it's only 1 table I feel for you and the teams. But it's the way the cookie crumbles. Hope you win big next time... Kinda like playing the lotto.

This all sounds good if it was just one table..but it wasn't they hammered every category and 3 of the 4 DAL spots came off that table as well..so the bottom line is there IS a problem and it's staring us right in the face..it's how we go about fixing it that matters most

Funtimebbq
11-06-2013, 09:20 AM
When the new scoring system was announced, well before it was implemented, much of the talk was about how reps would be able to balance tables by using the CBJs average scores. Is there any rep or BOD reading this thread who can tell us if that information is actually available? If it is, when will that practice start?

Seating judges is the responsibility of the KCBS rep, not the organizer who is only responsible to recruit the reps. To use the judges scoring info (if it is actually available) a list of judges needs to be sent to the contest rep prior to a contest so the rep can enter the CBJ membership numbers into the program and come up with a seating chart that will average out all the tables.

Again, can anyone tell us if the program is available?

TooSaucedToPork
11-06-2013, 09:35 AM
Or KCBS just color codes the CBJ numbers in a database that keeps a running average of scores by that #, only accessible by reps.

3 colors for high, low, and average. This way last minute CBJ's can be added without throwing off the math and without needing to contact headquarters. Less stress for reps.

sdbbq1234
11-06-2013, 09:56 AM
A couple of things here I see:

1 - So, what makes up a TOD? A bunch of people that know each other and typically score low? I don't know.

2 - I say all six scores should be part of the scoring. If not, then why not throw out the highest score along with the lowest score? If there is a judge that scores high for everything, what is the difference of a judge that scores low on everything?

To question #1, it is hard to fix or better something until you know the "real" root cause.

#2, I still say a JOL (judge of life; I just made that up) is equally detrimental to the cause as a JOD.

wallace

TooSaucedToPork
11-06-2013, 10:49 AM
A couple of things here I see:

1 - So, what makes up a TOD? A bunch of people that know each other and typically score low? I don't know.

2 - I say all six scores should be part of the scoring. If not, then why not throw out the highest score along with the lowest score? If there is a judge that scores high for everything, what is the difference of a judge that scores low on everything?

To question #1, it is hard to fix or better something until you know the "real" root cause.

#2, I still say a JOL (judge of life; I just made that up) is equally detrimental to the cause as a JOD.

wallace

1. They don't have to know each other, A TOD is created when a cluster of low scoring judges (JOD, Judges of Death) are seated at one table. You can tell by their average scores given in ALL categories for the day, and the rankings of the entries coming off that table if a TOD exists. The scores will be significantly lower at that table than every other table at the contest. Are all the judges at the table JOD...No. It only takes a few to tank your chances. Are they collaborating to screw teams...No. They just score lower than the average CBJ and by luck, ended up at the same table, creating a TOD.

YES, high tables do just as much damage.

DanaK6JQ
11-06-2013, 11:59 AM
A couple of things here I see:

1 - So, what makes up a TOD? A bunch of people that know each other and typically score low? I don't know.

2 - I say all six scores should be part of the scoring. If not, then why not throw out the highest score along with the lowest score? If there is a judge that scores high for everything, what is the difference of a judge that scores low on everything?

To question #1, it is hard to fix or better something until you know the "real" root cause.

#2, I still say a JOL (judge of life; I just made that up) is equally detrimental to the cause as a JOD.

wallace

Agreed. While it may appear that I've argued that Tables of Death do not exist, that's not my intent at all. Instead, I'm observing that the information even in the enhanced KCBS scoring system probably is not adequate to objectively identify judges that are unfairly evaluating entries. Once the contest is over and the food is gone, there's no way to audit the process of judging. All we have left is teams saying "that was a good cook!" and some judges scoring otherwise.

Even if a table consistently scores lower than other tables, that's not something that can be objectively audited after the fact when the food is gone.

The new KCBS scoring system allows data to be collected on the scoring habits of CBJs - but not all contests are 100% CBJ. There will always be newbie judges in circulation, as well. Some portion of the judging pool will have no history.

Ultimately, the problem is that judging is inherently subjective. Judges will have different ideas of what is average, good or great. Maybe the KCBS can develop a detailed rubric that describes the characteristics of appearance, taste and tenderness and the corresponding score; that way both cooks and judges would know exactly what's being scored and how to cook.

But the problem with this is that all the teams would start cooking exactly that and all the boxes would look and taste the same. Not sure that's progress.

If there's a judge out there saying "I never hand out 9s to chicken breasts", that's a problem. But if there's a judge out there that just has a different idea of what is excellent, that's just the system.

landarc
11-06-2013, 12:17 PM
I think there are two issues here, one is not addressable, and one is.

If a judge us using personal bias, and is judging unfairly, based upon reasoning such as "I never score high for something that tastes like Blues Hog" or "I never score something high that I know used a commercial rub", that is something that cannot be addressed, unless someone knows about the bias.

If a person just naturally judges lower, that can be tracked, using the CBJ number. If the new software is tracking judges natural tendencies, then math is the answer, as you can easily adjust averages at a single table. Just assign handicaps to judges and seek a table average for every table.

The issue of not using the whole scale can be applied, but, that is just as random as people using only 7-8-9. Using a format similar to the GE Method for personnel evaluation, which is to say, that for every entry group of 6, there must be at least one 6 and one 9 and 4 that are not 6 or 9, that would change the judging, but, it would be just as artificial. So, there is bound to be randomness in the process.

bbq.tom
11-06-2013, 01:23 PM
I don't think Math can help in this equation... Pun Intended :) Who's to say that your food is good or bad other than the person tasting it... Even in every/any category? It all comes down to whatever people consider good or bad at the time. Maybe most were from Memphis or Carolina at that table, maybe they all ate at McDonalds before judging, etc..Maybe you have always ended up on the TOL (Table of Luck). Alot of Maybe's.... I know it's only 1 table I feel for you and the teams. But it's the way the cookie crumbles. Hope you win big next time... Kinda like playing the lotto.

WELL, Thank you VERY MUCH for considering judges from Memphis or Carolina as "BAD" judges!!!

TooSaucedToPork
11-06-2013, 01:30 PM
WELL, Thank you VERY MUCH for considering judges from Memphis or Carolina as "BAD" judges!!!

I believe the poster is saying that Memphis and Carolina has such amazing bbq that judges from there have no choice but to score BBQ laid in front of them low as nothing in the world compares to Memphis and Carolina BBQ.

:tape:

goodsmokebbq
11-06-2013, 01:45 PM
I am starting to think that the "table environment" may also be a factor in the TOD. What I mean is the teams on the same table as you. I think it is counter to what many people think. In Hudson Valley contest this year I had the luck of landing on the 3 of the four lowest tables at the event (10 total). Needless to say it was our worst comp in 4 years.

As I started looking at the sheets, win the chicken table but take 28 place (out of 51), win the pork table and finish 30... blah blah cry for me stuff, I noticed that the teams on my tables were almost exclusively new / local. The reps moved us around to different tables, but by and large we competed against the same teams in every category.

I think it is quite possible that the teams around you may set the bar for the table. A judges ruler is shifted by the average "goodness" of the table. I haven't looked at any numbers yet, just some personal experience and intuition.

Judges: Is it harder for a good entry to get a 9 when everything else has been a 6 or if everything has been an 8? I know you are not supposed to judge relatively, but I could definitely see it hard for me to give a 9 when everything on the table has been so bad.

Teams: Are any of the TOD's you are seeing contain mostly new / local teams?

This is pretty much against the school of thought to avoid the good teams at a contest. If the bad teams can hurt you the good ones can help. Anyone else seen this "anchor" / "buoy" effect?

I will say that I am glad I had the sheet with the new scoring or I would have really thought about changing something. I left the program alone and won my next contest :-)

DanaK6JQ
11-06-2013, 01:53 PM
If a person just naturally judges lower, that can be tracked, using the CBJ number. If the new software is tracking judges natural tendencies, then math is the answer, as you can easily adjust averages at a single table. Just assign handicaps to judges and seek a table average for every table.


One thing I noticed looking at a couple of (new) scoresheets is judges that were averaging lower scores, but were opposite the table consensus. In other words, of the 6 judges, 4 were scoring similarly (usually higher) and 2 were scoring similarly (usually lower, thus the lower average scores). The things that the 4 judges scored high, the 2 would score low, but the thing that 4 judges scored low, the 2 scored high.

Which makes the concept of a "handicap" a bit slippery. What do you do with opposite judges?

bignburlyman
11-06-2013, 02:02 PM
Judges: Is it harder for a good entry to get a 9 when everything else has been a 6 or if everything has been an 8? I know you are not supposed to judge relatively, but I could definitely see it hard for me to give a 9 when everything on the table has been so bad.

Teams: Are any of the TOD's you are seeing contain mostly new / local teams?

This is pretty much against the school of thought to avoid the good teams at a contest. If the bad teams can hurt you the good ones can help. Anyone else seen this "anchor" / "buoy" effect?

I will say that I am glad I had the sheet with the new scoring or I would have really thought about changing something. I left the program alone and won my next contest :-)

I don't have a problem giving a 9 even after scoring something a 5 that tasted bad. The biggest problem for me at least, and it was mentioned somtime earlier, is when (mostly chicken) the sauce used is so spicy it burns the lips and tongue. It is very difficult to cleanse and neutralize the pallette after an entry like this. It MAY affect the taste score of following entries, I TRY not to let it affect my score, but it is possible.

DanaK6JQ
11-06-2013, 02:05 PM
I think it is quite possible that the teams around you may set the bar for the table. A judges ruler is shifted by the average "goodness" of the table. I haven't looked at any numbers yet, just some personal experience and intuition.



Very good point. I judge a few contests a year, but I'm not in danger of Master CBJ status any time soon. I noticed pretty quickly that sometimes the mood of a table is influenced if the chicken boxes come in looking especially poor or especially nice. I remember specifically one time after a dull set of chicken boxes had been judged and the sheets turned in, a judge declared "well this isn't gonna be much fun today". Looking back, I don't think he influenced me, but it could be subtle. Maybe that table scored everything lower as a result.

big matt
11-06-2013, 02:11 PM
One thing I noticed looking at a couple of (new) scoresheets is judges that were averaging lower scores, but were opposite the table consensus. In other words, of the 6 judges, 4 were scoring similarly (usually higher) and 2 were scoring similarly (usually lower, thus the lower average scores). The things that the 4 judges scored high, the 2 would score low, but the thing that 4 judges scored low, the 2 scored high.

Which makes the concept of a "handicap" a bit slippery. What do you do with opposite judges?

Dana you spew all of this info out and question everyone..how about offering a solution instead of arguing with everyone..you did this daily on the cbbqa forum for months on end..now your on this site doing the same exact thing..and before you call this an attack it's not it's just straight up fact..this forum has always let me speak my mind unlike cbbqa..we are looking to solve the problem not argue about it..so hopefully for the 3rd time you'll give an answer..do you have a solution?

DanaK6JQ
11-06-2013, 02:28 PM
Dana you spew all of this info out and question everyone..how about offering a solution instead of arguing with everyone..you did this daily on the cbbqa forum for months on end..now your on this site doing the same exact thing..and before you call this an attack it's not it's just straight up fact..this forum has always let me speak my mind unlike cbbqa..we are looking to solve the problem not argue about it..so hopefully for the 3rd time you'll give an answer..do you have a solution?

Do I have a solution? No. I don't think we understand the "problem" enough to propose a solution. There is probably more than one component to the "problem", multiple things to consider. We've discussed multiple possible causes for Tables of Death, each one has different potential solutions.

There's a difference between discussion and arguing. It's not arguing to ask what a handicap system would with "opposite judges" - it's just trying to understand how that would work before it's launched and undesirable unintended consequences are the result.

TooSaucedToPork
11-06-2013, 02:50 PM
One thing I noticed looking at a couple of (new) scoresheets is judges that were averaging lower scores, but were opposite the table consensus. In other words, of the 6 judges, 4 were scoring similarly (usually higher) and 2 were scoring similarly (usually lower, thus the lower average scores). The things that the 4 judges scored high, the 2 would score low, but the thing that 4 judges scored low, the 2 scored high.

Not a TOD. 2 judges liked different things or a newbie was copying scores over the shoulder of another judge.

JOD's score EVERYTHING low.

Sawdustguy
11-06-2013, 02:56 PM
Put on your EE hat and think about that for a minute;) Hope all is well, Guy!

After thinking for a moment....not a great idea. Sorry Landarc. Everything is good, how about you?

Podge
11-06-2013, 03:14 PM
Maybe the list of judge’s names, numbers and the tables they were on can be given to the organizer only. If then the organizer hears of people complaining of the TOD, then he/she can then decide if they will let that judge come back. We all know some great Reps, and we know some bad reps. We can decide if we want to go to a contest or not, knowing of the quality of reps. Then the organizer will be forced to try to get better reps. I know myself will not go to certain contests if I do not care for the reps. I think organizers should have more decision power of their contest. An organizer can know who the problem judges are at their contest, and not allow them to come back. A contest can have a bad reputation that there’s a couple of judges who come back year after year, who give out 3’s and 4’s and not doing anything about it. Organizers should have the power to decide who will make their contest a success, Quality Reps AND Quality Judges.

So, you go to a contest in 2014. Bad judges.. If you go back in 2015 and there’s bad judges again, you don’t go back in 2016.

Organizers should have more control of their contest's success. They can't let a couple of judges leave a bad taste in the mouths of competitors as they are packing up to leave when they are scratching their heads looking at the score sheets.. The best electricity, great parking, kick a$$ goodie bag, etc.. all goes out the window and forgotten about when they know if a judge didn't give them a 665 they'd won GC and $2500.

TooSaucedToPork
11-06-2013, 03:30 PM
Maybe the list of judge’s names, numbers and the tables they were on can be given to the organizer only. If then the organizer hears of people complaining of the TOD, then he/she can then decide if they will let that judge come back. We all know some great Reps, and we know some bad reps. We can decide if we want to go to a contest or not, knowing of the quality of reps. Then the organizer will be forced to try to get better reps. I know myself will not go to certain contests if I do not care for the reps. I think organizers should have more decision power of their contest. An organizer can know who the problem judges are at their contest, and not allow them to come back. A contest can have a bad reputation that there’s a couple of judges who come back year after year, who give out 3’s and 4’s and not doing anything about it. Organizers should have the power to decide who will make their contest a success, Quality Reps AND Quality Judges.

So, you go to a contest in 2014. Bad judges.. If you go back in 2015 and there’s bad judges again, you don’t go back in 2016.

That doesn't solve anything, it is reactive, not proactive. It does nothing to prevent TOD's save a judge witchhunt. It could hurt a great judge that is sat at a bad table.

Plus I don't believe organizers should have access to any lists that target judges...only reps should have that knowledge. There are some shifty organizers out there that will try to stack the deck with "awesome high scoring judges" to attract teams.

dmprantz
11-06-2013, 03:46 PM
Some one mentioned a "table effect" a few posts ago, and my head started spinning thinking about the things that I could do with data. I could use Best Linnear Unbiassed Prediction to isolate the residual random effects of the scoring and indentify tables that are statistical outliers at competitions as well as judges that are statistical outliers across multiple competitions. Keep in mind that this would track high and low scoring tables/judges. I could then use some estimation to identify judges who are statistically likely to score very high and very low, taking the environment (table) effect into mind. This could be used to either exclude judges, or to select them based on the values for table placement. Estimating the random effects would take some time, but wouldn't it be cool? For the love of good data....

dmp

DanaK6JQ
11-06-2013, 03:49 PM
That doesn't solve anything, it is reactive, not proactive. It does nothing to prevent TOD's save a judge witchhunt. It could hurt a great judge that is sat at a bad table.

Plus I don't believe organizers should have access to any lists that target judges...only reps should have that knowledge. There are some shifty organizers out there that will try to stack the deck with "awesome high scoring judges" to attract teams.

I agree, this would inevitably turn judging into a popularity contest.

dmprantz
11-06-2013, 03:49 PM
Also, as my wife is getting the kids dressed to go to gymnastics, I'm wondering if there isn't a solution there: Do gymnastics meets have every one compete on the same floor mat or the same beam, or do they have multiple instances of each apparatus? If the former, maybe there's some magic to be learned from how they manage it. Something tells me that it could be like martial arts with small-ish divisions and/or bracket style competitions, ala final table.

dmp

The_Kapn
11-06-2013, 04:01 PM
Also, as my wife is getting the kids dressed to go to gymnastics, I'm wondering if there isn't a solution there: Do gymnastics meets have every one compete on the same floor mat or the same beam, or do they have multiple instances of each apparatus? If the former, maybe there's some magic to be learned from how they manage it. Something tells me that it could be like martial arts with small-ish divisions and/or bracket style competitions, ala final table.

dmp

I am a former USGF Girls Gymnastics Judge.

The scoring system starts with standards of performance and specified deducts for errors and omissions.

At the actual point of judging on the floor, all panel judges must be "in range" of the Head Judge. If not, scores are explained and compromised/adjusted as needed to reach an agreement within the specified range.

Works really well, but I can not see it ever working for BBQ.
First off, who in the hell would be the head judge on a table????

Also, we were "paid professionals" with expenses paid and a small stipend per event.
Not like volunteers at all.

TIM

goodsmokebbq
11-06-2013, 04:08 PM
Some one mentioned a "table effect" a few posts ago, and my head started spinning thinking about the things that I could do with data. I could use Best Linnear Unbiassed Prediction to isolate the residual random effects of the scoring and indentify tables that are statistical outliers at competitions as well as judges that are statistical outliers across multiple competitions. Keep in mind that this would track high and low scoring tables/judges. I could then use some estimation to identify judges who are statistically likely to score very high and very low, taking the environment (table) effect into mind. This could be used to either exclude judges, or to select them based on the values for table placement. Estimating the random effects would take some time, but wouldn't it be cool? For the love of good data....

dmp


Outliers are simply the judges or tables who's average is a couple sigmas (you decide) out in the distribution, no need for a fancy statistical test :-) but I like your thinking...

We would expect to have outlier tables just like we expect outlier judges. The tables will follow the same pattern as the judges, a couple low, a couple high and a bunch in the middle.


AGAIN assuming that judges are consistent (which I am not convinced they are).

dmprantz
11-06-2013, 04:20 PM
Outliers are simply the judges or tables who's average is a couple sigmas (you decide) out in the distribution, no need for a fancy statistical test :-) but I like your thinking...

What I'm talking about is a little more advanced than that. Not just looking at the raw values, but treating them as random effects, fitting them with a model that takes fixed effects such as the table into account, and estimating the residual variance. Same concept, but it looks a bit more at the overall population as well as the environment to determine where things should be. It's the sigmas of the random effect that I'd look at, not the raw score. The next step would be to take into account these values and actual variance to predict how likely the judge is to score high or low in the future. It works:)

I was thinking about this a bit more, and I'd like to fit sex and age in the model, but I can already hear the discrimination complaints from that even though there's mathematical reasoning behind it, and insurance companies already do that. I'm just exercising at this point:)

dmp

sdbbq1234
11-06-2013, 06:19 PM
I like some of the other ideas:

The judge picks a number from a hat upon arriving. She/he sits at the assigned table/seat.

Everyone changes table/seats after each category.

Logistics? Yea, maybe hell.

The best way to mix it up. At least there "hopefully" wouldn't be a TOD.

At the minimum, judges pick a number for the table/seat.

wallace

djqualls
11-06-2013, 08:31 PM
Maybe the list of judge’s names, numbers and the tables they were on can be given to the organizer only. If then the organizer hears of people complaining of the TOD, then he/she can then decide if they will let that judge come back. We all know some great Reps, and we know some bad reps. We can decide if we want to go to a contest or not, knowing of the quality of reps. Then the organizer will be forced to try to get better reps. I know myself will not go to certain contests if I do not care for the reps. I think organizers should have more decision power of their contest. An organizer can know who the problem judges are at their contest, and not allow them to come back. A contest can have a bad reputation that there’s a couple of judges who come back year after year, who give out 3’s and 4’s and not doing anything about it. Organizers should have the power to decide who will make their contest a success, Quality Reps AND Quality Judges.

So, you go to a contest in 2014. Bad judges.. If you go back in 2015 and there’s bad judges again, you don’t go back in 2016.

Organizers should have more control of their contest's success. They can't let a couple of judges leave a bad taste in the mouths of competitors as they are packing up to leave when they are scratching their heads looking at the score sheets.. The best electricity, great parking, kick a$$ goodie bag, etc.. all goes out the window and forgotten about when they know if a judge didn't give them a 665 they'd won GC and $2500.

I agree, this would inevitably turn judging into a popularity contest.

As an organizer, of which I am, I want to know and if its deemed a popularity contest by blacklisted judges then so be it!

I want my contest to give the teams the most level playing field available. And for a liability reason I'm going to require the organizer give me a list of the assignments for the judges now that I think of it.

I'm also a CBJ, and the required training for "CBJ's" is a joke. I took the class after I cooked 21 contests and they only thing emphasized was to find the negative in the entry not the positive. This is the root of the problem. Oh and placing six pieces of identically cooked chicken thighs on a CBJ class table with two boxes containing a banned garnish or another DQ item is not training........

jcog44
11-06-2013, 09:05 PM
The judge's class is to teach protocol and process. That way the new judge shows up at the contest with some idea of how things work. Then, it takes a lot of judging and sampling of both the good and the bad, along with many discussions among the judges after each sample, to get the scoring thing under control. I don't know how you would structure a class to teach someone what to like.

djqualls
11-07-2013, 12:34 AM
The judge's class is to teach protocol and process. That way the new judge shows up at the contest with some idea of how things work. Then, it takes a lot of judging and sampling of both the good and the bad, along with many discussions among the judges after each sample, to get the scoring thing under control. I don't know how you would structure a class to teach someone what to like.

The first thing you could do is give six different samples derived from six different flavor profiles. The vary the amounts of sauce and styles of meat in the boxes. i.e. thighs, legs, breast, St.louis, Baby Back, Sliced MM, Pulled, chunks, Brisket slices, and Burnt Ends.

Then do the Blot test on chicken to differentiate between juicy and bloody undercooked. Purposely overcook pork to mush on an entry. demonstrate a proper way to test tenderness on a brisket slice not just wiggle it and count until it breaks off.

There is a lot of things that can and should be done to prepare a person to judge. The taste function must be acquired. I understand that.

RumRunner_1492
11-07-2013, 07:24 AM
The first thing you could do is give six different samples derived from six different flavor profiles. The vary the amounts of sauce and styles of meat in the boxes. i.e. thighs, legs, breast, St.louis, Baby Back, Sliced MM, Pulled, chunks, Brisket slices, and Burnt Ends.

Then do the Blot test on chicken to differentiate between juicy and bloody undercooked. Purposely overcook pork to mush on an entry. demonstrate a proper way to test tenderness on a brisket slice not just wiggle it and count until it breaks off.

There is a lot of things that can and should be done to prepare a person to judge. The taste function must be acquired. I understand that.

I was in a judges class in March of this year. We did most of what you said. We had different sauces on different meats, over/under cooked, the blot test, the pull test and many other things. They showed how to tell if something was prepared correctly and what to look for. They were very clear to point out they can't tell you what tastes good to you and they can only tell you how it should be prepared but can't tell you what looks better than something else.

I think some of you need to understand these people are volunteers and it might be a month or more since they have been a judge. Also, you are talking about adding more judges and removing scores but keep in mind many contests don't have enough CBJ's to begin with. Would you rather they bring in 20 people off of the street, give them a 5 minute crash course and then have them judge? How accurate would that make things?

I judged 3 competition this year before competing. The first thing they did was have a rookie judges sit one at each table. Then they had the most experienced judges sit at each table. then newer (5-10 comps) and older then finally anyone left could fill the gaps. They tried to make tables even by experience and you couldn't be with a spouse. That seems like the fairest way to do things as well as the easiest. Will there be a low table? Possibly but there could be a high table as well. I'm sure for every TOD there is a Table of Victory where everyone votes high. If you land on that table you will move up in the ranks. I think it is the nature of the beast.

ModelMaker
11-07-2013, 07:33 AM
The first thing you could do is give six different samples derived from six different flavor profiles. The vary the amounts of sauce and styles of meat in the boxes. i.e. thighs, legs, breast, St.louis, Baby Back, Sliced MM, Pulled, chunks, Brisket slices, and Burnt Ends.

Then do the Blot test on chicken to differentiate between juicy and bloody undercooked. Purposely overcook pork to mush on an entry. demonstrate a proper way to test tenderness on a brisket slice not just wiggle it and count until it breaks off.

There is a lot of things that can and should be done to prepare a person to judge. The taste function must be acquired. I understand that.

The only purpose of KCBS judges training is to teach the rules and procedures used at a KCBS sanctioned BBQ contest. How to effectivly judge is learned through experience.
Meats supplied to KCBS judges training are by voulenteer cooks who are reimbursed for their meat purchases and spend part of their Fridays to help out the process. Can you imagine the additional meat costs, time, and lost family time needed to prepare all the different types of samples you propose?
It's an OK idea but not very reasonable.
Ed

dirtydingus
11-07-2013, 09:01 AM
As an organizer, of which I am, I want to know and if its deemed a popularity contest by blacklisted judges then so be it!

I want my contest to give the teams the most level playing field available. And for a liability reason I'm going to require the organizer give me a list of the assignments for the judges now that I think of it.

I'm also a CBJ, and the required training for "CBJ's" is a joke. I took the class after I cooked 21 contests and they only thing emphasized was to find the negative in the entry not the positive. This is the root of the problem. Oh and placing six pieces of identically cooked chicken thighs on a CBJ class table with two boxes containing a banned garnish or another DQ item is not training........
You're exactly right. Alot of the so called CBJ's heard thru the grapevine that they could go to a 2 hour class and become a CBJ. Does that make them a good judge? Hell no. It takes practice. So, what criteria do they use to judge,,,, they start comparing, just what they're taught not to do. The only bbq maybe they've ever had is the neighbor, the local cook shack, their spouse. They don't know good bbq from bad. They don't judge it according to KCBS criteria. Which needs to be emphasized at every contest. Yeah, they play the tape, but how many judges listen and take heed?
Judges eat with their eyes, just remove the part of the tape that has to do with the smoke ring.
And lastly, for you judges, watch the bird biters at yours and surrounding tables. One bird bite and the rest goes in the cooler. Is that a fair way to judge? IMHO no! And if there are judges out there that think "some" judges aren't there just to feed their family for the next week,,,,you're blind in one eye and can't see out of the other. Just watch the next time you judge.
As for the TOD problem. I'm a new competitor in 2013. Don't think I hit one. Only cooked 7 contests. The data base idea might work. Low, med, high classifications. then color code their KCBS card, which they are supposed to bring to every contest they judge. Then the reps could seat accordingly. Thanx for listening.

PS- My wife and I are MCBJ's and have never taken a cooler to a contest----EVER!! We judge for the cooks..... PERIOD. Okay, now I'm done.

bbqczar
11-07-2013, 09:15 AM
The bottom line here,there is NO current way to keep this from happening at any contest,and if you have competed for any length of time then inevitably it will,or has, happened to you.Sooooo,what to do ? You pack up,go home and enter the next contest and go on,not much else you can do,we've all been there,done that.

big matt
11-07-2013, 09:28 AM
The bottom line here,there is NO current way to keep this from happening at any contest,and if you have competed for any length of time then inevitably it will,or has, happened to you.Sooooo,what to do ? You pack up,go home and enter the next contest and go on,not much else you can do,we've all been there,done that.

That's true Tim..but now there is info out there proving the TOD..have you seen the new score sheets yet?..lots of info

TooSaucedToPork
11-07-2013, 12:41 PM
The bottom line here,there is NO current way to keep this from happening at any contest,and if you have competed for any length of time then inevitably it will,or has, happened to you.Sooooo,what to do ? You pack up,go home and enter the next contest and go on,not much else you can do,we've all been there,done that.

That is correct, there is no CURRENT way...so one needs to be created.

goodsmokebbq
11-07-2013, 01:05 PM
I like some of the other ideas:

The judge picks a number from a hat upon arriving. She/he sits at the assigned table/seat.

Everyone changes table/seats after each category.

Logistics? Yea, maybe hell.

The best way to mix it up. At least there "hopefully" wouldn't be a TOD.

At the minimum, judges pick a number for the table/seat.

wallace

Unfortunately you are left with just another random draw of the judge pool, which will have the same statistics and the same likelyhood of a TOD that exists now.

goodsmokebbq
11-07-2013, 01:13 PM
What I'm talking about is a little more advanced than that. Not just looking at the raw values, but treating them as random effects, fitting them with a model that takes fixed effects such as the table into account, and estimating the residual variance. Same concept, but it looks a bit more at the overall population as well as the environment to determine where things should be. It's the sigmas of the random effect that I'd look at, not the raw score. The next step would be to take into account these values and actual variance to predict how likely the judge is to score high or low in the future. It works:)

I was thinking about this a bit more, and I'd like to fit sex and age in the model, but I can already hear the discrimination complaints from that even though there's mathematical reasoning behind it, and insurance companies already do that. I'm just exercising at this point:)

dmp

I am not sure why you want to predict the future outcome of a judge with a complicated mixture model, when the best way to identify an outlier is how far out in the distribution they are using past performance.

I still am not convinced that an outlier is always an outlier. If judges are not consistent your model will have massive error bars and be useless.

goodsmokebbq
11-07-2013, 01:20 PM
The equation for Variance suggests two solutions:

1/n^2* the sum over i and j (1/2(xi - xj))

To minimize the equation you can:

1. increase n (add more judges to a table) which is pretty powerful as it is a squared term

2. minimize the difference between xi and xj , as you are summing those up (this means minimize the difference between judges... somehow...) :icon_smile_tongue:

RumRunner_1492
11-07-2013, 01:53 PM
That's true Tim..but now there is info out there proving the TOD..have you seen the new score sheets yet?..lots of info

Would you complain if your entries hit a table that rated everyone high?

I don't think there is anything that can be done that already isn't being done. They make it as even and as random as possible. There is always going to be a table that rates tougher than others and there will be a table that gives higher scores than others. Not everything people submit is a 9 or a 10.

Jorge
11-07-2013, 02:11 PM
Would you complain if your entries hit a table that rated everyone high?

I don't think there is anything that can be done that already isn't being done. They make it as even and as random as possible. There is always going to be a table that rates tougher than others and there will be a table that gives higher scores than others. Not everything people submit is a 9 or a 10.

I won't speak for Matt, but high scoring tables are an issue as well in my opinion. I KNOW I've benefitted from at least one of those in the past.

big matt
11-07-2013, 02:12 PM
Would you complain if your entries hit a table that rated everyone high?

I don't think there is anything that can be done that already isn't being done. They make it as even and as random as possible. There is always going to be a table that rates tougher than others and there will be a table that gives higher scores than others. Not everything people submit is a 9 or a 10.

First of all I'm not complaining..and your correct I have hit good tables and have won contests because of them I also didn't hit the dreaded TOD on those days the days we did we were out of it..so the problem remains that a table can be so bad it ruins your day no matter how good your food is or isn't..if you simply land on that table your day is done..yes I know it's been this way but we can now see it actually happen right on our score sheet..deny it doesn't exist but I've got the score sheet to prove it..and it CAN be fixed but I'm not the one..guys there is great info in this thread hopefully with all of these ideas being thrown out something good can come of this..that's all I really wanted by starting this thread..thanks for such great input

big matt
11-07-2013, 02:33 PM
And just to be clear I'm talking about a table that scores ALL categories in the toilet not just one meat.

dmprantz
11-07-2013, 02:35 PM
The reason why I see this as solvable by a mixed model equation is because I could effectively predict the likelihood that a judge would score high or low while taking into account the fixed effects of the model: environment, sex, MCBJ, etc. I would also fit either age of the judge, competitions judged, or years as a CBJ into the model as a covariate to see if judges normalize as they get older or judge longer. Wouldn't it be interesting to see if men or women tend to score differently as a group? Isn't it important to isolate the environment (competition or table) out of the equation so that you get a true prediction of future performance isolated from environmental effects such as table pressure, cold weather, or being in a barn? This looks to solve the question of is an outlier always an outlier? Instead of looking just at past results, it uses sound science to predict future outcomes. It answers the precise question of dealing with per competition variance by treating that as a fixed effect and solving for the competition and table as well as individual judge. I would also fit random effects for the judge to account for the discrete nature of the scores and get a continuous rating over a period of time, and for the overall residual variance of the data (VE).

The goal of this really is to do two things: 1) It isolates out things that we know (or think) would affect how a judge scores, and 2), taking those into account along with past results, it predicts the likelihood that the judge will score very low or very high in the future. You can certainly look at past data and make selections based on those, but trying to predict the future based on those past results has shown to have better results when possible. Together with other selection tools, this could be used to build even tables at a given competition. Rather than using all high or all low judges at an event, you select an even number of each and distribute those. It's about getting rid of high and low tables.

It's possible that I'm falling into the trap of being a carpenter and seeing everything as a nail, but at face value, this fits very nicely into the models that I already solve. I doubt any one else wants to read me geek out about mixed model statistics and using them to discriminate against judges, but if you'd like to discuss the math and science behind this further, you're welcomed to PM me. I just read discussion of environmental effects and thought "wouldn't it be nice." I would guess that this won't be happening any time soon, so there isn't much value in further discussing the best statistical model for preventing TODs in this thread.

dmp

Balls Casten
11-07-2013, 04:27 PM
How much of the problem is right in the rule book?

KCBS Rules and Regulations
"The scoring system is from 9 to 2, all whole numbers between two and nine may be used to score an entry. 9 excellent, 8 very good, 7 above average, 6 average, 5 below average, 4 poor, 3 bad, and 2 inedible. 5) A score of one (1) is a disqualification"

ITBFQ
11-07-2013, 05:20 PM
The equation for Variance suggests two solutions:

1/n^2* the sum over i and j (1/2(xi - xj))

To minimize the equation you can:

1. increase n (add more judges to a table) which is pretty powerful as it is a squared term

2. minimize the difference between xi and xj , as you are summing those up (this means minimize the difference between judges... somehow...) :icon_smile_tongue:

This guy is way smarter than me!

Pappy Q
11-07-2013, 05:47 PM
I predict KCBS will take one of the following actions to address the issue:
1. Nothing
2. Remove the table number from the scoresheet

Yellow Card BBQ
11-07-2013, 06:02 PM
I'm not denying the existence of a table of death. I'm wondering if people are denying the existence of a table of life.

I feel bad for the original poster, and I'm sorry it cost him, but as a judge (and rookie cook whose issues were his own, not the judges), this whole conversation has the feel of a blackjack table who gets mad at the guy at 3rd base for hitting and taking the dealers bust card when he should have stuck, but never seems to remember when the guy at 3rd base should have hit, didn't, and the dealer busted because of that strategy mistake.

The point I'm trying to make is, statistically speaking, for every table of death a team hits, shouldn't that team, over the course of the season, end up on a table of life where they scored higher than they probably should have because of generous judging? (With the generous judging probably not generating a 12 page thread). Obviously there's sample sizes at play with how many tournaments you enter, but I've got to believe this good luck/bad luck averages out over the long run.

goodsmokebbq
11-07-2013, 06:53 PM
The reason why I see this as solvable by a mixed model equation is because I could effectively predict the likelihood that a judge would score high or low while taking into account the fixed effects of the model: environment, sex, MCBJ, etc. I would also fit either age of the judge, competitions judged, or years as a CBJ into the model as a covariate to see if judges normalize as they get older or judge longer. Wouldn't it be interesting to see if men or women tend to score differently as a group? Isn't it important to isolate the environment (competition or table) out of the equation so that you get a true prediction of future performance isolated from environmental effects such as table pressure, cold weather, or being in a barn? This looks to solve the question of is an outlier always an outlier? Instead of looking just at past results, it uses sound science to predict future outcomes. It answers the precise question of dealing with per competition variance by treating that as a fixed effect and solving for the competition and table as well as individual judge. I would also fit random effects for the judge to account for the discrete nature of the scores and get a continuous rating over a period of time, and for the overall residual variance of the data (VE).

The goal of this really is to do two things: 1) It isolates out things that we know (or think) would affect how a judge scores, and 2), taking those into account along with past results, it predicts the likelihood that the judge will score very low or very high in the future. You can certainly look at past data and make selections based on those, but trying to predict the future based on those past results has shown to have better results when possible. Together with other selection tools, this could be used to build even tables at a given competition. Rather than using all high or all low judges at an event, you select an even number of each and distribute those. It's about getting rid of high and low tables.

It's possible that I'm falling into the trap of being a carpenter and seeing everything as a nail, but at face value, this fits very nicely into the models that I already solve. I doubt any one else wants to read me geek out about mixed model statistics and using them to discriminate against judges, but if you'd like to discuss the math and science behind this further, you're welcomed to PM me. I just read discussion of environmental effects and thought "wouldn't it be nice." I would guess that this won't be happening any time soon, so there isn't much value in further discussing the best statistical model for preventing TODs in this thread.

dmp


Way overkill if you ask me, but I get it... As you can see it is exciting for me also to use some of our formal training on something we are so passionate about :thumb:

goodsmokebbq
11-07-2013, 06:55 PM
This guy is way smarter than me!

More knowledgeable in statistics, maybe... smarter, I really doubt :becky:

Alexa RnQ
11-07-2013, 07:04 PM
Obviously there's sample sizes at play with how many tournaments you enter, but I've got to believe this good luck/bad luck averages out over the long run.
The only problem with saying "it'll all even out over time" is that time (i.e., the repetition of the expenses of coming to contest after contest) is something that many teams haven't got.

I've heard a lot of griping here over the years about the 6-to-10 contest per year crowd versus the 30+ contest per year crowd, and how the vast bulk of the competition population will never hit shows half the weekends out of the year. Yet now it's proposed that we should avoid methods to make each contest as balanced as possible, because it'll "average out over the long run"?

There are many teams that won't want to or can't afford to beat their heads against the brick wall of randomness on top of the difficulty of mastering their cook.

goodsmokebbq
11-07-2013, 07:39 PM
I was just getting over getting my butt kicked the summer at Hudson Valley, when this topic had to open up the wound again :shocked:

I did some analysis at the time and though in light of the current topic you guys might find it interesting, especially the folks the competed at Hudson Valley.

I calculated a table bias for each category and then calculated a bias factor for each team based on which tables they hit. I then reordered the contest based on the new score with the bias applied.

We had the second largest factor, moving us from 23 to 2...

How good is 3 EYZ! They finished 2 while hitting some lower scoring tables. After the bias factor it was a blowout!

Have fun!

http://goodsmokebbq.com/wp-content/uploads/2013/11/hudsonupdate.bmp

ModelMaker
11-07-2013, 07:46 PM
I don't know a lot of these teams, but the ones in the top ten I've heard of.....
Cream always rises to the top?
Perhaps the system ain't quite as broke as y'all think.
Ed

TooSaucedToPork
11-07-2013, 08:14 PM
I'm not denying the existence of a table of death. I'm wondering if people are denying the existence of a table of life.

I feel bad for the original poster, and I'm sorry it cost him, but as a judge (and rookie cook whose issues were his own, not the judges), this whole conversation has the feel of a blackjack table who gets mad at the guy at 3rd base for hitting and taking the dealers bust card when he should have stuck, but never seems to remember when the guy at 3rd base should have hit, didn't, and the dealer busted because of that strategy mistake.

The point I'm trying to make is, statistically speaking, for every table of death a team hits, shouldn't that team, over the course of the season, end up on a table of life where they scored higher than they probably should have because of generous judging? (With the generous judging probably not generating a 12 page thread). Obviously there's sample sizes at play with how many tournaments you enter, but I've got to believe this good luck/bad luck averages out over the long run.

Post 130 on page 9 of this thread...shows the existence of both Tables of Angels and Tables of Death.

Okie Sawbones
11-07-2013, 08:59 PM
I have enjoyed reading this thread. What is stated here could be transposed onto the wine judging circuit. I am a certified wine judge, and I did the circuit from CA, OR, WA, NY, and IN for several years. We had tracking software at many competitions that identified consistently low and high scoring judges. While nothing 'official' was ever done, those types of judges were simply not asked to judge at the competition the next year. The CA State Fair took it a step farther, and you could not judge there if you were not certified, and if you did not take their continuing education.

No one likes to hurt people's feelings, but the dead wood needs to be sorted out. Not everyone makes a good judge, be it a wine competition or a BBQ competition. Education and experience help, but even then, there are some judges that 'just don't get it'. And this doesn't take into account those folks who have their taste perception altered by medications, smoking, old age, allergies, upper respiratory infections, sugary drinks, etc.

I think I need to get into BBQ judging. 5s for everyone! :wink: :twisted:

Smoke'n Ice
11-07-2013, 09:27 PM
A majority of the discussion above makes the assumption that the TOD was populated by CBJ's. Looking at the KCBS results page, there is no indication about the percentage for this contest.

Given the location, time of year, size of contest and other factors, there is a good possibility that there were a lot of tables with employees on lunch break and no real guidance. It happens more than you would believe.

Yellow Card BBQ
11-07-2013, 09:34 PM
Yet now it's proposed that we should avoid methods to make each contest as balanced as possible, because it'll "average out over the long run"?

There are many teams that won't want to or can't afford to beat their heads against the brick wall of randomness on top of the difficulty of mastering their cook.

I'm not advocating not trying. I'm just advocating for more threads saying "Thank goodness for table 3, because we got 1st when we probably should've gotten 7th"

Well, like the wine judge said, so long as the competition is judged, you're going to run into debates about judges. Happens in wine, figure skating, boxing...

Chenernator
11-07-2013, 09:58 PM
They are all very experienced and they spent quite a bit of time sorting out the judges to try to balance the tables with new, experienced, master judges, etc.

Well, it's true that they spent a lot of time, but don't confuse activity with achievement. They did ensure there was a MCBJ at each table. They also checked for noobs; there weren't any. I would say, however, that over 85% of judges stayed at the same table they sat down at originally. That increases the likelihood of friends judging together, who MAY have similar tastes, experiences, and/or scoring tendencies. Other KCBS reps who sort everyone by experience (aka the Southwest Airlines sort) achieve a much more balanced distribution of judges. These reps have crunched the numbers and noted that their method results in more consistency in judging as well. It's also faster.


I would propose the following is a possible reason for the scoring at this contest:
1. Location - How many CBJ's in the area
2. Number of teams - Exceeded number 1 by a large number
3. CBJ's and MBJ's tend to congregate together at a contest
4. Newly trained judges - Taught that 6 is average and score up or down
5. Celeberty, off the street judges - Listen to the CD and learn that 6 is averge and score up or down
6. Not enough CBJ's - CBJ's used as TC at a table of off the street and newly minted
7. Some tables of pure CBJ's - Good (not) news (they start at 9)
8. Some tables of new CBJ's - Bad news (they start at 6)
9. Some tables of off the street - really bad news (who knows where they start)


I am not certain, but I believe the percentage of CBJ's was 100%. Definitely no celebrity judges. Everyone had at least one comp under their belt. The high number of judges led to greater geographic diversity than usual. Some judges did do double duty, but they had all done so previously.

One other note, When one or two judges score low, that generally merits a visit or discussion with the rep. If the TOD was consistent across the board, it wouldn't have set off any alarms with the TC or the reps when the scorecards were turned in.

I'd go with a better sort of the judges, and see if that takes care of the issue.

Haastyle
11-07-2013, 11:07 PM
Well I am new to the competition scene and only did 6 events this year. We were never in contention to win anything and knew that we would have to take our licks for awhile before we get to that point. But even I recognizied who hit and avoided the glariingly obvious table of death when the new score sheets came out. It became a game for me to figure out who got screwed.

We were getting better and just had taken a bbq class, and planned for several more classes before next season. This entire thing makes me question if I want to do this. Reaffirms some of our original hesitation in the matter. I was considering on really making this a hobby. Now I am not so sure. This isn't the only time I have read about problems in the comp world. It is a lot of fun and I have met a bunch of good people, but I can do the same thing for much cheaper elsewhere.

I don't know exactly what the fix is but I would think that logic says to try something, even if it turns out to be wrong you can always change it again. This isn't baseball with a storied set of proven rules over a century old. The problem it seems stemmed from the massive explosion in the game. Not a lot of effort being made to adjust for it I think. And again, you can't get everyone to agree on one thing. But I think if you could get a heavy majority to try somethign, it is an avenue worth pursuing. Why not? Leaders should lead, that is why the are there... So lead. But I am not holding my breath for any changes.

That's my two cents, I'll go back to staying quiet and reading again.

Pappy Q
11-08-2013, 03:41 AM
Teams need to keep pushing the Organizers to address it as that is who KCBS gets their money from. The Organizers are the true "customer" to KCBS so they will listen to them more. We cooks are mere "employees" in the grand scheme. If the Organizers demand fair and balanced judging, KCBS will do something about it.

esselle
11-08-2013, 04:27 AM
Wasn't this exactly what the new scoring system was brought in for? To eliminate high or low scoring judges/tables?

big matt
11-08-2013, 07:44 AM
Well I am new to the competition scene and only did 6 events this year. We were never in contention to win anything and knew that we would have to take our licks for awhile before we get to that point. But even I recognizied who hit and avoided the glariingly obvious table of death when the new score sheets came out. It became a game for me to figure out who got screwed.

We were getting better and just had taken a bbq class, and planned for several more classes before next season. This entire thing makes me question if I want to do this. Reaffirms some of our original hesitation in the matter. I was considering on really making this a hobby. Now I am not so sure. This isn't the only time I have read about problems in the comp world. It is a lot of fun and I have met a bunch of good people, but I can do the same thing for much cheaper elsewhere.

I don't know exactly what the fix is but I would think that logic says to try something, even if it turns out to be wrong you can always change it again. This isn't baseball with a storied set of proven rules over a century old. The problem it seems stemmed from the massive explosion in the game. Not a lot of effort being made to adjust for it I think. And again, you can't get everyone to agree on one thing. But I think if you could get a heavy majority to try somethign, it is an avenue worth pursuing. Why not? Leaders should lead, that is why the are there... So lead. But I am not holding my breath for any changes.

That's my two cents, I'll go back to staying quiet and reading again.
This right here is exactly what we don't want to happen..new people going away because they feel like they might get screwed right out of the gate..anybody else feel this way?

HarleyGirl14226
11-08-2013, 08:28 AM
The equation for Variance suggests two solutions:

1/n^2* the sum over i and j (1/2(xi - xj))

To minimize the equation you can:

1. increase n (add more judges to a table) which is pretty powerful as it is a squared term

2. minimize the difference between xi and xj , as you are summing those up (this means minimize the difference between judges... somehow...) :icon_smile_tongue:

Brian, You are like an Evil Math Genius. :becky:

cpw
11-08-2013, 09:04 AM
This right here is exactly what we don't want to happen..new people going away because they feel like they might get screwed right out of the gate..anybody else feel this way?

We're a new team (KCBS anyways), and I'll compete regardless because I like to do it. It would however make me feel a lot better knowing that there was less "luck" involved and more "skill" if I win or lose a contest.

What I can't understand about this entire thread is why there is so many people who are against doing anything with all of this new data that we have access to. It can't really be that hard to average out and place 6 judges on specified tables without hurting their feelings. I don't care that there are statistical anomalies and outliers and all that. An average is an average, and it will only improve and become more precise over time. If a judge has only scored 2 contests with low scores, that means on average they're going to score low. So put them on a table with a judge who scores high on average. It can only help with the TOD and the TOL.

Alexa RnQ
11-08-2013, 09:06 AM
Teams need to keep pushing the Organizers to address it as that is who KCBS gets their money from.
And here we are getting closer to the truth.
I have heard it said that the bulk of KCBS membership cash flow comes from judges, and that nothing will be done that will risk that cash flow. I don't know how accurate either statement is.

It's true that action can be taken at the level of organizers and cooks, even before KCBS formulates policy on what to do with the data it's collecting. It doesn't take much effort for an organizer to be aware of the judges they're signing up; and cooks already by and large know which reps they can count on to sort as much as possible and reduce wild variations in tables.

However, I take heart from the introduction of the new information on the score sheets. It provides SO much -- back in the day as a new team, if a meat tanked when we thought it was good we would spend months "fixing" something that didn't need to be fixed. Now we can see if it truly was just the table, although the onus is still on the team to realistically evaluate their product/performance.

This new information has been out for about five minutes, relatively speaking. What comes next from it will be very interesting. And while the results may be bumpy at first, I think the ultimate effect will be of benefit to all.

Can we say, this is all that's needed? No.
Can we say, we can never eliminate variation so we shouldn't do more? No.

I can say that I am happy to see the corner of the veil lifted a bit. We have always said that cooks are fully accountable with names attached to their results, and that a balancing accountability should take place on the other side of the table. This is an important first step, but equally important is to take the next step to provide a level playing field across all the tables.

TooSaucedToPork
11-08-2013, 10:29 AM
we have always said that cooks are fully accountable with names attached to their results, and that a balancing accountability should take place on the other side of the table. This is an important first step, but equally important is to take the next step to provide a level playing field across all the tables.

amen

Podge
11-08-2013, 10:31 AM
My wife (who is smarter than I am) has an idea. Give all judges score sheets too. If a judge gives half a $hit on what they do, they can take these scores and analize them too, and learn from that and try to improve. If the best chicken cook in the country hit their table, and the judge realizes they gave them 5's and 6's, but the rest of the table gave them 8's and 9's, then they can realize their issues themselves, and hopefully try to become a better judge.

carlyle
11-08-2013, 10:58 AM
I am all for giving judges and organizers the score sheets. But blind the teams to keep

the double blind in effect. Just because it is a famous team, does not necessarily mean

that entry was outstanding.

The need to judge what is presented is paramount and should not be influenced by reputation, either before the entry is judged or after.

Judges don't need to know the team name, just how they are doing relative to the rest of the table.

Just look at this thread- the TOD and the " hot table" are big problems. We now have the info to make the process better. Let's use the new scoring info wisely.

dhuffjr
11-08-2013, 11:22 AM
Judges don't need to know the team name, just how they are doing relative to the rest of the table.

That is not a bad idea. Provide the judges with their scores relative to their table. Show them their avg score compared to their table. Maybe show them the avg score of judges at that contest.

Podge
11-08-2013, 12:15 PM
That is not a bad idea. Provide the judges with their scores relative to their table. Show them their avg score compared to their table. Maybe show them the avg score of judges at that contest.

I agree, but since it's after the fact, it ain't going to hurt for a judge to know who they judged.

Brew-B-Q
11-08-2013, 01:51 PM
Then we'll have an Entries of Death discussion among judges that are upset that they didn't get food from any of the top teams at a contest. There will be mention of a conspiracy to direct the so-called top teams' food to others tables because of tie-ins with the organizer.

carlyle
11-08-2013, 03:28 PM
Letting a judge know who they judge is a problem, even after the judging is over.
At least a couple of problems that I can think of right off the bat.
Brew B Q is on the right track- Entry of Death, yeah, I like that - think I judged one of those at the Royal this year. Talk about bad brisket!
Also if a sharp judge is able to tie a taste together with a name and they encounter that taste again at another contest, it can lead the judge to think he or she may know whose entry it is and effect the judges score either up or down. Don't want that either.

Yellow Card BBQ
11-08-2013, 03:43 PM
What I can't understand about this entire thread is why there is so many people who are against doing anything with all of this new data that we have access to. It can't really be that hard to average out and place 6 judges on specified tables without hurting their feelings. I don't care that there are statistical anomalies and outliers and all that. An average is an average, and it will only improve and become more precise over time. If a judge has only scored 2 contests with low scores, that means on average they're going to score low. So put them on a table with a judge who scores high on average. It can only help with the TOD and the TOL.

I think you kind of answered it yourself. I'm not against doing something with the new data. I'm against doing something with the new data now, just because there isn't enough of it. Once you have enough judges with say, 5 contests in the database, then you can start using that data to balance out tables. But I wouldn't want to start making kneejerk reactions and sorting judges based on 1 or 2 competitions.

Meat Man
11-08-2013, 03:52 PM
It's the KCBS reps job to put together the tables. Difficult to do at any contest, let alone one with 85 teams. One thought might be to have KCBS reps that work in certain regions, who might be more familiar with the majority of the judging pool work these contest.

bbq.tom
11-08-2013, 04:13 PM
It's the KCBS reps job to put together the tables. Difficult to do at any contest, let alone one with 85 teams. One thought might be to have KCBS reps that work in certain regions, who might be more familiar with the majority of the judging pool work these contest.

If the organizers could confirm judges sooner, the REPs would have more time to find out who is coming and could do some pre-placement of judges at tables. Unfortunately, it is RARE for the REPs to have a clue on who will be there until they get there, and at that point the REPs have their hands full with other matters.

McEvoy AZ
11-08-2013, 04:14 PM
Maybe since the table captain should be the most experienced judge at a given table, after the scores have been given he should shuffle the scores and read back the scores given by all the judges at the table. I would think this would at least help train the judges how to score as it would force conversation on how to score correctly. Or the table could give a consensus score that would not count but would be more guidance to the team. The consensus score would be after the individual scores have been turned in based on the conversation and agreed between judges at a given table. Anything that would help a low scoring or a high scoring judge to understand how to judge. My bet is that TOD judges didn't even realize they scored lower then maybe they should have. I think forcing conversation between the judges is the key to having consistent trained judges.

bignburlyman
11-08-2013, 04:50 PM
Maybe since the table captain should be the most experienced judge at a given table, after the scores have been given he should shuffle the scores and read back the scores given by all the judges at the table. I would think this would at least help train the judges how to score as it would force conversation on how to score correctly. Or the table could give a consensus score that would not count but would be more guidance to the team. The consensus score would be after the individual scores have been turned in based on the conversation and agreed between judges at a given table. Anything that would help a low scoring or a high scoring judge to understand how to judge. My bet is that TOD judges didn't even realize they scored lower then maybe they should have. I think forcing conversation between the judges is the key to having consistent trained judges.

In my experience a lot of the time the table captain is just a volunteer who is helping at the contest. At the last contest I judged a 1st time CBJ volunteered to table captain. He did a fine job captaining, I don't know about his judging, lol.

dr pearl
11-08-2013, 05:29 PM
If comment cards were brought back and made MANDATORY, this would at least make the judge accountable with details regarding your food. Still might not help with the TOD, but then again, maybe the food at your table just sucked, including your entry. I am a fairly new cook who struggled with my first competitions. I almost bailed on competing, thinking the judging was rigged somehow. I worked hard and my scores this year improved drastically, not because I finally got on a "GOOD" table, but because my food improved, my techniques improved, my attention to detail improved, my program improved. Of course it's bothersome when you see 6's or 7's without knowing the why. Comment cards justifying the score would at least give me some idea as to what went wrong, instead of coming up with calculations, formulas, excuses and the such to determine Death Tables or Angel Tables. That's is just bad sportsmanship. We all play by the same rules. Cook good food consistently and your scores will show it as such. You will not win them all and there will be bad days. To me, this is just ruining it all. Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.

peterz
11-08-2013, 11:10 PM
If comment cards were brought back and made MANDATORY, this would at least make the judge accountable with details regarding your food. Still might not help with the TOD, but then again, maybe the food at your table just sucked, including your entry. I am a fairly new cook who struggled with my first competitions. I almost bailed on competing, thinking the judging was rigged somehow. I worked hard and my scores this year improved drastically, not because I finally got on a "GOOD" table, but because my food improved, my techniques improved, my attention to detail improved, my program improved. Of course it's bothersome when you see 6's or 7's without knowing the why. Comment cards justifying the score would at least give me some idea as to what went wrong, instead of coming up with calculations, formulas, excuses and the such to determine Death Tables or Angel Tables. That's is just bad sportsmanship. We all play by the same rules. Cook good food consistently and your scores will show it as such. You will not win them all and there will be bad days. To me, this is just ruining it all. Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.


And there you have it !!!!:clap2:

bbqczar
11-09-2013, 10:49 AM
If comment cards were brought back and made MANDATORY, this would at least make the judge accountable with details regarding your food. Still might not help with the TOD, but then again, maybe the food at your table just sucked, including your entry. I am a fairly new cook who struggled with my first competitions. I almost bailed on competing, thinking the judging was rigged somehow. I worked hard and my scores this year improved drastically, not because I finally got on a "GOOD" table, but because my food improved, my techniques improved, my attention to detail improved, my program improved. Of course it's bothersome when you see 6's or 7's without knowing the why. Comment cards justifying the score would at least give me some idea as to what went wrong, instead of coming up with calculations, formulas, excuses and the such to determine Death Tables or Angel Tables. That's is just bad sportsmanship. We all play by the same rules. Cook good food consistently and your scores will show it as such. You will not win them all and there will be bad days. To me, this is just ruining it all. Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.

EXACTLY! And like I said,pack up,and move on to the next contest ! Things sure were easier and more fun 8 years ago when we started,it gets less and less about BBQ and fun every year.

TooSaucedToPork
11-09-2013, 11:23 AM
If comment cards were brought back and made MANDATORY, this would at least make the judge accountable with details regarding your food. Still might not help with the TOD, but then again, maybe the food at your table just sucked, including your entry. I am a fairly new cook who struggled with my first competitions. I almost bailed on competing, thinking the judging was rigged somehow. I worked hard and my scores this year improved drastically, not because I finally got on a "GOOD" table, but because my food improved, my techniques improved, my attention to detail improved, my program improved. Of course it's bothersome when you see 6's or 7's without knowing the why. Comment cards justifying the score would at least give me some idea as to what went wrong, instead of coming up with calculations, formulas, excuses and the such to determine Death Tables or Angel Tables. That's is just bad sportsmanship. We all play by the same rules. Cook good food consistently and your scores will show it as such. You will not win them all and there will be bad days. To me, this is just ruining it all. Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.

It's not bad sportsmanship to want tables balanced...it's logic.

Comment cards are great, but you can't force judges to write exactly what is wrong.

Many will come up with stock answers to write down.

"Entry salty", "bland", "flavor imbalanced"
We have all seen the generic cards, and we try to figure out what they mean...


You put to much faith into humanity. The only truth in this world is mathematics...they do not lie.

Math is in everything, it is in all of your recipes...they can be broken down into equations.
Math is in your cook times, they are formulas.

Mathematics proves there are tables of death, and tables of angels.
Mathematics is the only way to fix those.

I love comment cards and believe they need to be mandatory...but comment cards, in the way used above are a bandaid on a gunshot wound...you accomplish nothing but covering the wound as the underlying problem persists

Podge
11-09-2013, 04:15 PM
I remember LPQ back in 2007. It was the highest payout of a contest at the time. We had to turn in Turkey as the first category. Then, we had an hour until the big 4 started. When the turkey was turned in, the reps then had an opportunity to get rid of the inconsistent judges/tables.

big matt
11-09-2013, 04:50 PM
EXACTLY! And like I said,pack up,and move on to the next contest ! Things sure were easier and more fun 8 years ago when we started,it gets less and less about BBQ and fun every year.

This is what's wrong with America not just BBQ..too many people throwing their hands up and say oh we'll move on to the next one..how about fixing the problem staring us in the face..nah just move on it's easier.

Bubba
11-10-2013, 05:35 AM
If comment cards were brought back and made MANDATORY, this would at least make the judge accountable with details regarding your food. Still might not help with the TOD, but then again, maybe the food at your table just sucked, including your entry. I am a fairly new cook who struggled with my first competitions. I almost bailed on competing, thinking the judging was rigged somehow. I worked hard and my scores this year improved drastically, not because I finally got on a "GOOD" table, but because my food improved, my techniques improved, my attention to detail improved, my program improved. Of course it's bothersome when you see 6's or 7's without knowing the why. Comment cards justifying the score would at least give me some idea as to what went wrong, instead of coming up with calculations, formulas, excuses and the such to determine Death Tables or Angel Tables. That's is just bad sportsmanship. We all play by the same rules. Cook good food consistently and your scores will show it as such. You will not win them all and there will be bad days. To me, this is just ruining it all. Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.

Dude I won the Royal and if you don't think LUCK was a factor in it I don't know what to tell you. The best food does not win at contests if any body thinks it does your sadly mistaken

dr pearl
11-11-2013, 10:05 PM
OK. So just to get this straight. It's luck when you win and math when you don't. Got it.

TooSaucedToPork
11-12-2013, 08:50 AM
OK. So just to get this straight. It's luck when you win and math when you don't. Got it.

Everyone that competes knows that luck is part of this sport. The 2 decades of comp BBQ I have under my belt has taught me many things.

Math is everything.
That perfect recipe is a balanced equation.
That perfect judging table is a balanced equation

But bad luck (variables you can't control) TOD, TOA, Weather, too much booze, Equipment malfunctions, etc. Will effect the balanced outcome of your contest entry.

When you change the variables (process) of your recipe it becomes an unbalanced equation.
That horrible judging table is an unbalanced equation, and so is a high scoring table.

That being said, not every contest has TOD and TOA. They are statistical outliers and don't occur at every contest...but the fact they do occur is a correction that can be made. This eliminates a bit of (luck...aka a variable) from the equation, bringing it closer to balance (aka fair across the board)

You said you are new to the game...you will learn with time just how big a role luck currently plays in Comp BBQ.

Dan - 3eyzbbq
11-12-2013, 09:22 AM
Ask Tuffy how he managed to win The Royal AND The Jack this year. He won't tell you he got lucky and was at the GOOD ANGEL table.

:doh: You obviously don't know Tuffy.

Alexa RnQ
11-12-2013, 09:26 AM
Anyone who has won more than a few GCs can point to contests they lost because of one table, or contests they won that were a gift.

big matt
11-12-2013, 09:53 AM
Anyone who has won more than a few GCs can point to contests they lost because of one table, or contests they won that were a gift.

Your exactly right..been on both ends this year more than once..that's the problem with this thread..lots of people chiming in that have never even won a contest..for those of us that cook comps on a regular basis know what really needs to happen to win it..cook your best and hope you don't hit "that table".

big matt
11-12-2013, 09:56 AM
:doh: You obviously don't know Tuffy.

And if I remember right it was one of the first things he said..we got lucky

dr pearl
11-12-2013, 11:45 AM
Your exactly right..been on both ends this year more than once..that's the problem with this thread..lots of people chiming in that have never even won a contest..for those of us that cook comps on a regular basis know what really needs to happen to win it..cook your best and hope you don't hit "that table".

Must feel great to be such an elitist. I forgot, those of us that compete a few times a year don't count.

Podge
11-12-2013, 11:55 AM
It only takes 2 judges to make a bad table.

TooSaucedToPork
11-12-2013, 12:06 PM
Must feel great to be such an elitist. I forgot, those of us that compete a few times a year don't count.

He wasn't being elitist...he was responding to:

"Anyone who has won more than a few GCs can point to contests they lost because of one table, or contests they won that were a gift."

The teams that only compete a few times a year are the bread and butter of KCBS. Most teams cook less than 5 contests a year.

What I took from the comment is that many of the people against the TOD or that don't agree it is a problem, are newer additions to the sport, with less experience, and no history in their memory of how luck gave them a gc or how it was missed bc of a bad table.

Experience = knowledge...

You and teams like yours are the future of BBQ.

big matt
11-12-2013, 12:14 PM
Must feel great to be such an elitist. I forgot, those of us that compete a few times a year don't count.

Where did I ever state that teams that don't cook more than a few don't count?..actually those teams make up the better part of competitions..what I'm pointing out is your lack of experience and misunderstanding of the point of this thread..have you actually lost a GC because of a table of death or vise versa?..my guess from from your responses would be no.

TooSaucedToPork
11-12-2013, 12:44 PM
what I'm pointing out is your lack of experience and misunderstanding of the point of this thread..have you actually lost a GC because of a table of death or vise versa?..my guess from from your responses would be no.

On the other hand...that was elitist ;-)

big matt
11-12-2013, 12:59 PM
On the other hand...that was elitist ;-)

Ok maybe just a bit:)..the point I was trying to make is that unless you have been directly affected by a TOD you might just not understand the feeling you get when you see you hit one..it's worked both ways for us and I'd be a fool to think luck wasn't a part of winning contests..I just feel like this one factor can be changed and fixed.

Podge
11-12-2013, 01:19 PM
Ok maybe just a bit:)..the point I was trying to make is that unless you have been directly affected by a TOD you might just not understand the feeling you get when you see you hit one..it's worked both ways for us and I'd be a fool to think luck wasn't a part of winning contests..I just feel like this one factor can be changed and fixed.

I fit into the category of not cooking many contests per year.. 4 to 6.. just happened I cooked 8 this year.. 59 contests total. Until recently, I've not really noticed a TOD. When one is present, you'll know it!.. I've seen just not a TOD but an evil Table of Total Eternal Damnation in Hendersonville, Tn. this year!

big matt
11-12-2013, 01:23 PM
I fit into the category of not cooking many contests per year.. 4 to 6.. just happened I cooked 8 this year.. 59 contests total. Until recently, I've not really noticed a TOD. When one is present, you'll know it!.. I've seen just not a TOD but an evil Table of Total Eternal Damnation in Hendersonville, Tn. this year!

Hahaha like the old saying goes..some pople get it some don't..Chad you deffinetly get it

Balls Casten
11-12-2013, 02:53 PM
Hahaha like the old saying goes..some pople get it some don't..Chad you deffinetly get it

Its not that people dont "get it". Its that they dont agree with the solutions that have been presented or the severity of the problem.

Scottie
11-12-2013, 03:15 PM
All it takes is to be in that position of losing a GC because of a bad table anf then you would get it...Then again. I never give back trophies when I am on a favorable table.... :-P

big matt
11-12-2013, 04:07 PM
Doug and Scottie I get both of your points..Doug maybe it's not a severe as it's being made out to be..but the fact remains it does happen and probably always has but does that mean we should just keep doing the same thing when we have score sheets proving it..guys we all know BBQ is subjective and not everyone has the same taste so there will always be variance in scores and I get that..I just think this element can be taken out of it.

Kit R
11-12-2013, 05:11 PM
All it takes is to be in that position of losing a GC because of a bad table anf then you would get it...Then again. I never give back trophies when I am on a favorable table.... :-P

I'd have to give back the chainsaw carved Chalice of Victory from that comp we did together in 2012 if we had to give back trophies after landing on favorable tables. Much butt hurting was had that day by the not so happy few. But we don't and I didn't.

dr pearl
11-12-2013, 05:13 PM
Its not that people dont "get it". Its that they dont agree with the solutions that have been presented or the severity of the problem.

Of course luck is involved. I never said it doesn't play a factor. Heck, luck is involved everyday I wake up and walk out the door. Of course math is involved. I know that too, but I also know math can give false results if not given the correct input. Yes, it sucks when a bad table or bad judge(s) ruins your day of glory. We are all very good at what we do, that is why we are able to compete. Some do it for fun with aspirations to become great, or at least get better and maybe hear our names. Some of you are already at the top of your game and I sincerely congratulate all of you on your successes and continued success.

JD McGee
11-12-2013, 07:05 PM
All it takes is to be in that position of losing a GC because of a bad table anf then you would get it...Then again. I never give back trophies when I am on a favorable table.... :-P

Lol...

The way I see it is this...there are two things I can control...how the meat looks (appearance)...and how it is cooked (texture). The flavor thing (taste) is up to six different sets of tastebuds per table per meat. We take a "do not offend any judge" approach with our bbq...that being a balance of smoke, sweet, and heat. The ONLY thing that really ticks me off is an unbalanced table of judges as far as their experience level is concerned. If the head judge is not assigning tables for balance...I have a problem with that. If the tables are balanced and one gives high...or low scores...oh well...not a whole lot I can bitch about. Well...I can...but it wouldn't do any good...that's just competition bbq. Moving along now...:becky:

North Is Up
11-12-2013, 11:56 PM
In the latest Board meeting (11/11/2013) posted on the KCBS site, a motion was made by Candy Weaver, "I move that KCBS develop a score sheet for judges so they can evaluate their scores."

I'm a 1st year judge and I know that there's room to improve. This would be a helpful tool. It would be valuable to me to take notes and compare it to the sheet at the end of the day. The notes would be there to jog my memory. Also, bring back comment cards.

I do remember a time this year that I had a very tasty destined-to-be a 9-9-9 sample of chicken, but while munching on it, I almost choked on the cut up bone and gristle that was left behind. It totally went against the appearance and flavor profile of 9s by leaving these in there and maybe the cook was just not careful about this one particular sample. There were at least four cut up bony/gristly pieces in there. The problem is that I had no way of letting the team know about it to support my low score for texture. I’m sure it had the team wondering what was wrong with me. Everything was excellent except for the that, but that was a biggie to me.

Also, considering flavor, I do not care if the seasoning is hot, mild, rich, salty, Asian flavored or whatever. Is it good? Does it make sense or is it a hodge-podge of mismatched flavors? I have a very open mind about BBQ and I welcome new flavors whatever the base seasoning is. The score sheet will let me know if I'm too liberal and need to reign in my taste buds. I don't think so. I like the old standbys and I like originality. I treat each entry as an individual art form that a team worked hard to produce. Crazy, and maybe I'm off base. Let me see where I stand.

ModelMaker
11-13-2013, 07:48 AM
In the latest Board meeting (11/11/2013) posted on the KCBS site, a motion was made by Candy Weaver, "I move that KCBS develop a score sheet for judges so they can evaluate their scores."

I'm a 1st year judge and I know that there's room to improve. This would be a helpful tool. It would be valuable to me to take notes and compare it to the sheet at the end of the day. The notes would be there to jog my memory. Also, bring back comment cards.

I do remember a time this year that I had a very tasty destined-to-be a 9-9-9 sample of chicken, but while munching on it, I almost choked on the cut up bone and gristle that was left behind. It totally went against the appearance and flavor profile of 9s by leaving these in there and maybe the cook was just not careful about this one particular sample. There were at least four cut up bony/gristly pieces in there. The problem is that I had no way of letting the team know about it to support my low score for texture. I’m sure it had the team wondering what was wrong with me. Everything was excellent except for the that, but that was a biggie to me.

Also, considering flavor, I do not care if the seasoning is hot, mild, rich, salty, Asian flavored or whatever. Is it good? Does it make sense or is it a hodge-podge of mismatched flavors? I have a very open mind about BBQ and I welcome new flavors whatever the base seasoning is. The score sheet will let me know if I'm too liberal and need to reign in my taste buds. I don't think so. I like the old standbys and I like originality. I treat each entry as an individual art form that a team worked hard to produce. Crazy, and maybe I'm off base. Let me see where I stand.

My biggest fear of judges having table score sheets is the comparison between judges. Do we want all judges to be alike in every aspect?
Isn't comparison of entries and thus scoring comparison still a bad idea?
Should you "reign your taste buds" to match the majority of the table?
I say absolutley not.
KCBS's goal should be properly trained judges that judge only whats in the box.
Educated individualism is a good thing (Martha Stewart mod)
Ed

Scottie
11-13-2013, 09:20 AM
Part of BBQ is hitting the table. You do have to have good food, but there is also a ton of luck involved. Anyone who says that they weren't lucky winning a GC or category can't say they were lucky. Dies it mean they had the best? No. They had the best for that table. It sucks, but I like it better than just using 9, 8, 7 in scoring .

The Great Darren 'Clone' Warth told me early in my career some words of wisdom. He was only a badass then, now he is supreme being with Tuffy... :-P But Clone said something that I will never forget. To be a champion, you have to learn to lose first.

bbqczar
11-13-2013, 09:52 AM
And the late,great Brent Walton of QN4U fame told me something similar a few years back.he said,"To be a winner you have to LEARN from losing",still miss Brent,what a great guy he was.

bbq.tom
11-13-2013, 08:53 PM
My biggest fear of judges having table score sheets is the comparison between judges. Do we want all judges to be alike in every aspect?
Isn't comparison of entries and thus scoring comparison still a bad idea?
Should you "reign your taste buds" to match the majority of the table?
I say absolutley not.
KCBS's goal should be properly trained judges that judge only whats in the box.
Educated individualism is a good thing (Martha Stewart mod)
Ed

I believe that you are missing the point here Ed, NOT trying to make all judges alike in every aspect, but getting them all in the same ballpark would be nice.

If Taste was the only score that had significant variance it probably wouldn't be that much of an issue, but 9s and 6s in Appearance on the same entry or on Tenderness of the same brisket is out of line.

Personally, I want to see how my scores compare with other judges, as I truly believe we can all learn from each other. Apparently you do as well, as you post Appearance scores on BBQCritic.com.

Kit R
11-19-2013, 06:50 AM
The Great Darren 'Clone' Warth told me early in my career some words of wisdom. He was only a badass then, now he is supreme being with Tuffy... :-P But Clone said something that I will never forget. To be a champion, you have to learn to lose first.

And the quote that comforts me (I forget who said it but it's gold): "You ain't s**t until you been DQ'ed". Really happy I got that out of the way early on. And I even have a picture to document the violation.

big matt
11-19-2013, 09:47 AM
Part of BBQ is hitting the table. You do have to have good food, but there is also a ton of luck involved. Anyone who says that they weren't lucky winning a GC or category can't say they were lucky. Dies it mean they had the best? No. They had the best for that table. It sucks, but I like it better than just using 9, 8, 7 in scoring .

The Great Darren 'Clone' Warth told me early in my career some words of wisdom. He was only a badass then, now he is supreme being with Tuffy... :-P But Clone said something that I will never forget. To be a champion, you have to learn to lose first.
I've heard him say that more than once Scottie..actually he said these score sheets just give people more to bitch about..and he's right..again:)!

Sawdustguy
11-19-2013, 10:54 AM
I do remember a time this year that I had a very tasty destined-to-be a 9-9-9 sample of chicken, but while munching on it, I almost choked on the cut up bone and gristle that was left behind. It totally went against the appearance and flavor profile of 9s by leaving these in there and maybe the cook was just not careful about this one particular sample. There were at least four cut up bony/gristly pieces in there. The problem is that I had no way of letting the team know about it to support my low score for texture. I’m sure it had the team wondering what was wrong with me. Everything was excellent except for the that, but that was a biggie to me.

Were there feedback cards for you to fill out?

North Is Up
11-20-2013, 12:32 AM
No, it was a fairly recent contest, so no comment card. I would have certainly written a "mouthful"! :becky: