Table of Death

[TooSaucedToPork]

Is this really fixable? We are talking humans after all. That awesome judge that is never part of a table of death can get sick and not taste sh*t. Suddenly that judge that knows BBQ can't taste it. Should he pull out of judging, sure, but would he if its say, the Jack?

I just pulled one variable out as an example, there are many. There will always be hot and cold tables IMO.

Like I said...

It is all about balancing the tables and reducing “tables of death”. But there is NO perfect system…there will still be tables that become mathematical outliers no matter how hard we try. This will reduce drastically but NEVER eliminate bad tables… the only way to ensure that bad tables are forevermore eliminated would be to have 1 table of six judges judge EVERY entry.

It will just bring balance to the force.

 

[landarc]

I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

 

[goodsmokebbq]

The underlying problem is that the number of judges at the table (6) does not reasonably ensure a representative sample of judges. You want a distribution of judges that represents the population at large, a normal distribution, maybe one high, one low and a few average. You know, the good old bell shaped curve. With a large enough sample size you can be reasonably assured that you have a good representative sample. That number depends on the population distribution, but a good rule of thumb is usually around 12 or so. Pretty hard to implement, but it would almost guarantee equality of tables.

We can after the fact “pretend” that the judges are from the same population and adjust the scores for individual table / judge bias. This is pretty standard in the data analysis world, when we are compensating for some bias, particularly when dealing with humans. Not sure how teams would respond to having their scores changed after the fact, but it would accomplish the goal of normalizing the tables / judges.

 

[QN]

There were four reps present, not sure why the website did not reflect that.

I would think it would be entirely possible for the current data being gathered to be applied to establishing a "handicap" or relevance score for judges, and for the judges' seating to be presorted accordingly. It would take some diligence in sign-ups (never a bad thing), and would still be subject to the occasional no-show, but as a cook I'd be willing to give it a spin and see how things shake out. It would provide much-needed consistency in the application of judge sorting across contests and regions.

There were actually a total of six KCBS contest reps working this contest; Randy and Carol Bilgler, Wayne and Maria Lohman, and Bill and Debbie Gage. They are all very experienced and they spent quite a bit of time sorting out the judges to try to balance the tables with new, experienced, master judges, etc.

 

[Jorge]

I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

That would actually have the effect of potentially creating more of those tables, or more tables that score well above the mean. If boxes are shuffled as they are supposed to be you increase the chance of a team or teams to actually land on those tables more often. Everybody should have the same opportunity, via chance, to land on the same good or bad table. If you change the makeup of the table that chance is gone.

 

[Sawdustguy]

I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

I like that idea!!

 

[boogiesnap]

that is a good idea.

 

[goodsmokebbq]

I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

You could just as easily create a new TOD for each category

 

[TooSaucedToPork]

I wonder how much of this comes down to the phenomenal growth of Competition BBQ and that judging is an easier and lower threshold of entry than cooking. A good way to be a part of a new passion, without the $1000's of dollars it takes to play. That would lead to the potential for more Tables of Death. Given the sheer numbers, there will always be a possibility of this happening. In that situation, and despite the chaos, maybe just shuffling judges after each category, would eliminate the TOD issue, and assure that there is no single bad table for an entire comp. And then there is no math.

Problem...ensuring you don't create chaos. Both on the scoring, the odds, and lastly in the judges tent.

Half hour between entries, and you have to assign seats 4 times, ensuring each time that the same people do not sit together more than once.

 

[Alexa RnQ]

And somebody's gonna complain that they have to move their cooler.

*runs like the wind*

 

[dhuffjr]

Who wants to set the O/U on how long it takes cooks to bitch and/or judges if the board actually does it?

Someone will complain. Especially if they don't do as well as they think they should or typically do. It is in our nature :becky:.

 

[goodsmokebbq]

Problem...ensuring you don't create chaos. Both on the scoring, the odds, and lastly in the judges tent.

Half hour between entries, and you have to assign seats 4 times, ensuring each time that the same people do not sit together more than once.

The implementation would be very difficult.

What if my team hits the TOD for chicken, then you reshuffle and then I hit the TOD for ribs, reshuffle and I hit a TOD for pork... At least the reps can move the teams to different tables now to avoid hitting it twice.

 

[TooSaucedToPork]

The implementation would be very difficult.

What if my team hits the TOD for chicken, then you reshuffle and then I hit the TOD for ribs, reshuffle and I hit a TOD for pork... At least the reps can move the teams to different tables now to avoid hitting it twice.

Exactly...moving judges is not a good solution. It creates more variables than we currently have in the judging process.

 

[Jorge]

I like that idea!!

Put on your EE hat and think about that for a minute Hope all is well, Guy!

 

[DanaK6JQ]

Let's look at the numbers Matt gave us a little closer. First of all, entries are given a point score, they're not ranked. So someone doesn't score 20th place - they score 164.000, for example. Let's look at how Table 10 scored the best entries they had.

According to Matt, the highest scoring entries at Laughlin Table 10 were:

Meat Overall Meat Score
Chicken 47th 164.0000
Ribs 48th 162.8228
Pork 41st 164.5600
Brisket 53rd 157.6800

Those *scores* don't look really insane; with the exception of brisket, they average above all 8s and the brisket score is mostly 8s. The best meats at table 10 were scored 8s or higher.

Let's take it another step: without mentioning team names, let's see how those teams did in other meats:

Team Score/Place (chicken/ribs/pork/brisket)
"47th Chicken" 47,164.000 45,163.4172 69,155.9428 79,142.24
"48th Ribs" 68,159.4284 48,162.8228 59,158.8344 22,167.4172
"41st Pork" 35,167.9656 43,163.9772 41,164.5600 27,165.7028
"53rd Brisk" 21,170.8800 33,165.6800 3,177.1772, 53,157.6800

This is a lot of concentrated data and maybe a little hard to read, but it says a lot. That 47th chicken team had lower _scores_ in every other meat and chicken was their second-best placement.

The 48th ribs team had lower scores in two other meats and ribs was their 2nd-best placement.

The 41st pork team had very consistent scores in the mid-160s and place lower in chicken.

The 53rd brisket team scored better in every other meat; brisket was their lowest score. Maybe Table 10 hates brisket. Maybe the team had a bad cook.

In other words, while it's easy to say that Table 10 was a Table Of Death and want to talk to those judges, it looks like all the other tables, for the most part, were scoring the other entries from these teams pretty similarly.

When you have 85 entries, placing near the middle of a category is gonna put you in 45th place or so.

Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.

 

[Podge]

How about no contests over 24 teams. Everybody hits the bad table. Increase the entry to $400-$500. That way, total take in for entries would still equal a 48 team contest for the organizer..

 

[big matt]

Let's look at the numbers Matt gave us a little closer. First of all, entries are given a point score, they're not ranked. So someone doesn't score 20th place - they score 164.000, for example. Let's look at how Table 10 scored the best entries they had.

According to Matt, the highest scoring entries at Laughlin Table 10 were:

Meat Overall Meat Score
Chicken 47th 164.0000
Ribs 48th 162.8228
Pork 41st 164.5600
Brisket 53rd 157.6800

Those *scores* don't look really insane; with the exception of brisket, they average above all 8s and the brisket score is mostly 8s. The best meats at table 10 were scored 8s or higher.

Let's take it another step: without mentioning team names, let's see how those teams did in other meats:

Team Score/Place (chicken/ribs/pork/brisket)
"47th Chicken" 47,164.000 45,163.4172 69,155.9428 79,142.24
"48th Ribs" 68,159.4284 48,162.8228 59,158.8344 22,167.4172
"41st Pork" 35,167.9656 43,163.9772 41,164.5600 27,165.7028
"53rd Brisk" 21,170.8800 33,165.6800 3,177.1772, 53,157.6800

This is a lot of concentrated data and maybe a little hard to read, but it says a lot. That 47th chicken team had lower _scores_ in every other meat and chicken was their second-best placement.

The 48th ribs team had lower scores in two other meats and ribs was their 2nd-best placement.

The 41st pork team had very consistent scores in the mid-160s and place lower in chicken.

The 53rd brisket team scored better in every other meat; brisket was their lowest score. Maybe Table 10 hates brisket. Maybe the team had a bad cook.

In other words, while it's easy to say that Table 10 was a Table Of Death and want to talk to those judges, it looks like all the other tables, for the most part, were scoring the other entries from these teams pretty similarly.

When you have 85 entries, placing near the middle of a category is gonna put you in 45th place or so.

Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.

your missing the point..anybody that hit table 10 was done regardless of where there other meats finished..it was the worst table of death I've seen since the new data was released..and again I'm not bashing ANYBODY!..there is a problem that is staring us right in the face.its about fixing it not slamming anyone..and never once did I mention "bad" judges.

 

[TooSaucedToPork]

Maybe that's why the KCBS isn't doing much with the per-judge scoring data - maybe they'll find out that most judges are doing their honest best and there are very few "bad" judges.

It's not "bad" judges that are the main problem...

It's judges that score too high, and judges that score too low...they are not bad, they are just not average. Most judges are doing their honest best, even the ones that score high and low. By balancing the tables according to a judges scoring trend you take away a chunk of the TOD risk.

Equations have to balance...by setting all tables up with the same makeup of judges, truly excellent entries will prevail and truly average or bad entries will not rise to the top.

Assigning tables by scoring takes away a bit of the luck and replaces it with consistency.And as everyone knows we cooks all strive for consistency.

 

[goodsmokebbq]

Procedure for normalizing the tables after the fact:

Calculate the overall mean score for a category for all tables (for example 160)

Calculate the mean score for each category on each table (table 1 170, table 2 150)

Create a table bias factor for each category and table (table 1 bias = -10, table 2 bias= +10)

Adjust a team’s score plus or minus based on which tables they hit. A team the scored 140 on table 2 would be biased up 10 points because table 2 was “on average” 10 points lower.

The only problem is that the average for the table is not very accurate, again because there are only 6 data points for the category. Taking a mean across all the categories for each table would help IF the table is consistent across categories (correlated). But the couple cases I have looked at suggest that there isn’t a large correlation between the categories for a given table. Being low on chicken doesn’t necessarily mean that the table will be low on the other categories (exceptions exist).

 

[big matt]

Someone will complain. Especially if they don't do as well as they think they should or typically do. It is in our nature :becky:.

Have you hit a table of death while in competition?