• working on DNS.. links may break temporarily.

Food for thought

Smoke'n Ice

is one Smokin' Farker
Joined
May 8, 2009
Messages
939
Reaction score
405
Points
0
Location
Plano, TX
Theory:
The current method of selecting grand and reserve is flawed; it does not reflect the best team overall but is biased with the judges’ preference for the meats they are judging.

Observations:
· Judges, in general, tend to like ribs, chicken, brisket and pork in that order
· Scores, in general, tend to reflect this in that the higher scores tend to be in the same order
· We select our grand champion and reserve grand champion based on a cumulative score of all four meats

Supposition:
If we concentrate on improving our scores in the order of the judge’s preference of meats, we can improve our odds of winning the grand as the KCBS method of selection grand does not take overall performance into account. The current method adds all of the scores and compares. Two meats in the high 170’s coupled with two in the 160’s normally wins.

Solution:
· Award each team in each meat the inverse of their place versus the number of teams: ie. 48 teams – 1st=48 points, 2nd =47 points, 3rd =46 point, etc. This removes judge bias
· Add all meat points together for each team and highest score wins. This weighs each meat equally regardless of the individual meat scores.

I ran a spread sheet on a random selection of cook offs this year and the grand/reserve did not change, but 3rd through 10th did on the three I checked. I also ran it on last year’s Jack and oops, I probably should not have. The Jack winners order was as follows:

Overall
1 4 LEGS UP BBQ
2 4 MILE SMOKIN' CREW
3 COOL SMOKE
4 BUTCHER BBQ
5 LUNCHMEAT
6 SMOKERS WILD
7 PARROTHEAD SMOKERS
8 DIZZY PIG
9 BUBBA & JEFF'S BBQ
10 CARCASS COOKERS

Applying the proposed solution based on place the following became the order
Overall
1 4 MILE SMOKIN' CREW
2 COOL SMOKE
3 4 LEGS UP BBQ
4 PARROTHEAD SMOKERS
5 SMOKERS WILD
6 LUNCHMEAT
7 DIZZY PIG
8 ALL DAY SMOKE
9 BUTCHER BBQ
10 CARCASS COOKERS

Question:
Do you think that this is a valid theory, supposition?
Should we do anything about it?
Does anyone care?
 
NO Dumb idea

Are the observations incorrect and Dumb? Do you feel that there is not a bias? I have looked at scores and the trend is average scores are higher for ribs then chicken, then brisket and last, but not least pork. To remove this bias in some method should be a goal. What would others suggest?
Mack
 
I don't mean to sound obnoxious but you really must have a lot of time on your hands! You based this entire idea on the results of three contests or am I misunderstanding what you did? Honestly, to say judges prefer ribs over all else is a stretch. While I don't judge often, I prefer pork and you have that last on the list. I would stop far short of calling this dumb but in different parts of the country, different categories rule so to do this would kinda be pointless. Besides, how can someone from Texas rank brisket as the third most popular category with judges? I'd guess in Texas, it's number one.
 
I don't know ... each team has to cook all 4 categories thus each team has the same advantage in ribs and disadvantage in pork.
 
Last edited:
I went back and looked at a few comps where I had all the scores and while the sample is not huge, I didn't see a pattern of consistently higher average rib scores, quite the opposite with the high averages ranked Pork; Chicken; Ribs; Brisket. Again, on my small sample the range of average scores across the board was 5 points.

There are a lot of factors that affect the final score, there is some anecdotal evidence that scores tend to decline a little on each category because the judges are getting full; but as long as the whole judging pool is affected by these, it really doesn't affect the overall outcome.

I can see how leveling the categories as Smoke'n Ice suggests might remove some of the variance for teams between contests where for example a team is very strong in ribs but less successful in brisket and they run into a judge pool that favors brisket in one event and ribs in another, but again as long as the same judges are evaluating the same cooks at the same contest the results of that contest are unlikely to change.
 
I don't mean to sound obnoxious but you really must have a lot of time on your hands! You based this entire idea on the results of three contests or am I misunderstanding what you did? Honestly, to say judges prefer ribs over all else is a stretch. While I don't judge often, I prefer pork and you have that last on the list. I would stop far short of calling this dumb but in different parts of the country, different categories rule so to do this would kinda be pointless. Besides, how can someone from Texas rank brisket as the third most popular category with judges? I'd guess in Texas, it's number one.


the three that I spread sheeted were from the KCBS results pages checking if a different method of selecting GC & RGC would make any difference. Since the scores are not posted, I persused the sheets of the 12 to 14 cooks that I did this year in OK, AR, and NM and noticed the trend. I only did one in TX and it was the first KCBS in Bedford.

The point was, these areas have a bias toward ribs & chicken and other areas can be different. The only time I cooked in the East was several years ago in NC and my pork scores and those of the competitors were higher indicating a possible bias in that area.

Just trying to be part of the solution by thinking about ways to improve the judging methods to prevent unconsious bias from being a part of the equation.

We all agree there are issues and complain but what do we do?

Mack
 
If you ask me, this all comes down to weighting and bias, not toward the meat, but rather the magnatude of the difference. Whether #1 finishes 2 points ahead of #2 or 10 points, #1 is still going to get more points toward GC regardless of the GC algorithm used. The difference is that if #1 finishes 10 points ahead, it will help them overcome another category where the lost to #2 by 5 points. Does that make sense? Using point ratings across the board helps a team far and away better in one category defeat a team marginally better in another. Using just placing points allows for more ties and doesn't take into effect by how much each team wins.

I'm not saying the idea is a bad one, I'm just examining what is being compared: Absolute pointing rather than relative pointing. I don't think meats have any input, but the magnitude portion does....

Further thoughts?

dmp
 
Actually, whether your observations and theory are correct or not, it's not a terrible
way of scoring. Sometimes simplicity is best. Oh, dont try to explain math or numerical and statistical variance or anything else 'round here; most dont have the capacity for it... JMHO.

-The Dawg
 
It would not be a good idea at all. The system on scoring is good the way it is..

The scores which lead to our GC and RGC is all bases on not only high scored but also on what we all know is the leading reason for WINS is consitancy..

It is also the best way... just because a blind squirrel finds a nut and get 1st place in say ribs.. 1st in chicken but last in pork and last in brisket doesnt mean he deserves a GC..

You must hold consistency and that is the best part of the scoring system.. if you place consistantly in the top 10 of all categories you are def almost locked into a GC or RGC unless someone else does the same in all 4 cats and over powers your scores! But if you can do that you DESERVE IT!!

my thoughts and IMHO!
 
Solution:
· Award each team in each meat the inverse of their place versus the number of teams: ie. 48 teams – 1st=48 points, 2nd =47 points, 3rd =46 point, etc. This removes judge bias
· Add all meat points together for each team and highest score wins. This weighs each meat equally regardless of the individual meat scores.


All your are saying is to take an average of the finishing position for all four categories. Treat each rank for each category equally.

Doesn't sound to fair to me.

And get ready for lots of coin flips...
 
Last edited:
Observations:
· Judges, in general, tend to like ribs, chicken, brisket and pork in that order
Honestly, to say judges prefer ribs over all else is a stretch.
I would not agree with the postulated bias toward ribs. We've had one GC where we tanked in ribs, and four more GCs where our ribs were not in the top five. If the bias toward ribs were that heavy, we'd have been sunk.

we all know is the leading reason for WINS is consitancy..
Our experience would more agree with this. Turning out consistently good product across four categories, time after time, is more predictive of success.

We all agree there are issues and complain
Um, no. Instead of trying to rebuild a system that we're working well in, we just go out and try to cook to a higher standard of consistency and excellence. We don't always get there, but we keep trying.
 
Anyone also think that brisket does hold a higher point value then anything else in the field?!? Hmmmmmm..
 
Back
Top