First (and probably last) time judging

[PimpSmoke]

It is not so much that they are being suggested on what to score... it is that you are trying to set the same scale. 5 mm vs. 5 ft. which scale is your 5 on.... The TCs and reps should be looking across the scores and address any variance that is large to ensure that the judge is in the same scale as the rest of the judges. If the food sucks, it sucks. score it that way.... but the grey area is where the seasoned folks are trying to help.

The flaw is... that the scale does not have a defined starting point. Start at "6", which is average.... well average is different to different folks. and is average edible.... or does the average guy over smoke his meat.... so you are looking for the average champion's food.... etc...

Since there is not a defined scale, and there is a wide range of judging experience.... Those folks are just trying to help normalize the scale, not keep people from judging how they feel.

Well then based on what your saying Andy, I would venture a guess that the BOD needs to re-evaluate the judging procedure. Something isn't working here.

 

[PatioDaddio]

Here's a seemingly stupid and simple question...

Isn't the KCBS rule to drop the lowest score expressly designed to limit the damage of a "rogue" judge? In other words, how much overall score damage can one judge really do, intentionally or otherwise?

I apologize if this was addressed earlier in the thread.

John

 

[Podge]

Here's a seemingly stupid and simple question...

Isn't the KCBS rule to drop the lowest score expressly designed to limit the damage of a "rogue" judge? In other words, how much overall score damage can one judge really do, intentionally or otherwise?

I apologize if this was addressed earlier in the thread.

John

I agree with that statement, and would also like to see the highest score dropped too. just go with the 4 judges left. that would be a lot more interesting. A rogue judge could be just as easy as a 999 as a 577, if the rest were a combination of 8's and 9's. Both of these judges have a potential to put you in the top 10, or knock you out of the top 20.

No one complains about the rogue high scoring judge. I've had entries make it to the top 10 that i didn't think they should have, and vise-versa. But look at it this way, if a high scoring rogue judge on table #1 gave one entry a 999 that wasn't good, and you're on table #2 and had an exellent entry and got a 567. what's fair ?

 

[PatioDaddio]

I agree with that statement, and would also like to see the highest score dropped too. just go with the 4 judges left. that would be a lot more interesting. A rogue judge could be just as easy as a 999 as a 577, if the rest were a combination of 8's and 9's. Both of these judges have a potential to put you in the top 10, or knock you out of the top 20.

And I'd agree with that, Podge. What we're really talking about is statistical variance. Outliers on either end of the spectrum need to be flattened or eliminated. Absent some complex statistical calculations (like perhaps standard deviation), dropping the high and low is a quick and dirty way of removing the "noise" from the scores.

I'd vote for moving to a 7-judge table (simply let the captain judge -- first) and throw out the high and low scores. I think these two simple steps would go a long way toward getting truer scores. Many teams already include a portion for the captain anyway, and s/he judging first would perhaps help establish the "baseline" that has been mentioned/alluded to frequently in this thread.

John

 

[JamesTX]

Why don't you just norm all the judges scores so that each judge's average is a 6? That way, if a judge is to lenient, their scores will be adjusted down, and if a judge is too harsh, their scores will be adjusted up.

Isn't the relative standing from one entry to the next (within a single contest) that is important? I mean, you don't try and compare scores from one contest to those from a different contest do you?

 

[Podge]

Why don't you just norm all the judges scores so that each judge's average is a 6? That way, if a judge is to lenient, their scores will be adjusted down, and if a judge is too harsh, their scores will be adjusted up.

Isn't the relative standing from one entry to the next (within a single contest) that is important? I mean, you don't try and compare scores from one contest to those from a different contest do you?

I personally do not compare one contest's scores to another. I've won a couple of grands with scores in the 630's and in the 670's. Comparing contests together is, i think, useless. I do like to compare what I have turned in to the scores i got in a contest. Sometimes i feel they are in line, sometimes i don't. I do think a lower scoring contests seem to have more newbie judges, (therefore, scores can be all over the place, and add up lower at the end) and the higher scores have more estabished judges.I could be wrong, but that's my theory and my experience.

 

[baughman]

I have always had problems with any sport that doesnt have a clear cut finish line so to speak. But I realise in sports like BarbQ, skateboarding, diving, and figure skating.

There has to be a way to judge. The problem is it has to be by a person. Unless they are going to invent a device to test the 3 cats. But then it would be pure science and not art, which is what Barbq leans taword in my mind.

I mean how would a computer test it? Would the meat closest to a set moisture point get tenderness? Then a template would be used to make sure slices are perfect for looks? Not sure yet how a computer could do taste tho hehe.

All in all. I am amazed at how well contests are judged in Barbq for the most part. I mean there are tons and tons of comps out there each week, and most KCBS contest seem to be close to the same.

Most other sports with judging. have the same judges travel around to each comp. since there is only an event or 2 each week or month.

REally i think KCBS jsut needs to come out and say, everyone use sweet sauce. At least then you know.

AA I am prob way off being a noob here. But thats my 2 cents.

All in all. The more I look into competing the more I am amazed at the friendlyness, and devotion to the sport there is.

I post in a few other sports forums. And by far and away this is the most easy going. This thread did tend to slide a bit. But everyone tried to be nice. Cant say that for pretty much any other sport.

 

[HoDeDo]

Well then based on what your saying Andy, I would venture a guess that the BOD needs to re-evaluate the judging procedure. Something isn't working here.

Pimp - I think you are correct.... but having said that -- some folks must be getting that "special flavor" or touch into thier cooking...

Because if it really was a crap shoot -- how would someone like say, Pellet Envy, be able to get 7 Grand Championships all over the country. It isn't just mastering mediocrity as someone mentioned earlier.... It is cooking good quality que that is a notch above the rest. Even with the judging as it is... somehow the cream still rises to the top, and isnt that the goal? If I hear the Slabs are going to be at a contest, I know they are going to be up there in the hunt, 4 Men and Pig, Ribs 4 U, you name it.... there are teams you see/hear week in and week out. Those names get called in Sugar Creek MO, Burlington KS, and everywhere in between... It is because they produce excellent Que, and have great flavor profiles that a large percentage of the judges like.

 

[PimpSmoke]

Well, I think that is a very good point. I have seen it, but what about those teams that place in the top three that no one has ever heard of?


It happens at every KCBS comp, someone out there scores big.


Is it good Q? Or have they touched on mediocrity?

 

[PatioDaddio]

You know, come to think about it, there really are two fundamental problems -- statistical (score) variance, which is founded in group-think, while the other is more objective.

The masses seem to feel that middle-of-the-road sweet is "the holy grail", while the the independent-minded judges seem to want to judge the product as it's presented.

I think this probably lends credibility to the position that both the high and low scores should be thrown out. How else (minus major mental gymnastics) do you flatten the curve?

John

P.S. Yes, I realize that all BBQ judging is subjective from the outset, but it seems the goal is to remove "rogue" biases.

 

[HoDeDo]

Well, I think that is a very good point. I have seen it, but what about those teams that place in the top three that no one has ever heard of?


It happens at every KCBS comp, someone out there scores big.


Is it good Q? Or have they touched on mediocrity?

Personally, I think it is good Que... and just a team that does not cook alot of contests....

learning to cook the good stuff, in any situation... (hot/cold, rain/shine, etc) that is what makes the champions we see year in and year out. Consistency is the next level. Win one, and that's great. But there are only 24 teams in the entire country that have won 2 or more grands this year. and of those, a handful definately win thier share....

 

[PimpSmoke]

Consistency is HUGE!! I agree, But at the expense of expanding Q??

Being able to turn out good Q every time out is awesome. that is a championship team.

I just wish the judging was a bit more.........open, don't know what else to say.

 

[Big George's BBQ]

I would love to reach the point of consistency. As I continue to cook my Que does improve. I think one of the problems with judging is that like those that cook it takes time and experience to reasch the area of consistency. That is why I thing that there should be mentoring at a contest for the less experience judges. I have only judged two events a back yard and a desert. Even at those events I was nervous that I would be the one inconsistent score that would ruin a persons placement. I was for the most part in line with the rest of the table. Mentoring would help the learning process.

 

[Double D's BBQ]

That's an interesting point George. I would like to see Rookie judges be an addition to the table. Maybe they could judge for a year while learning from the more experienced judges. I don't think there is a perfect solution but I do like the idea of something like an apprentice program for judges.

 

[HoDeDo]

Yea, Pimp I know what you mean.... I think however, that having tasted some of that uber champ BBQ over the last few years -- they are expanding BBQ -- the flavors pop, above the rest. I mean brisket that explodes in your mouth, chicken that is moist and tender. Time and again, you can look at folks that have gone to a class like Rod's class, and have incorporated some of his techniques and/or flavor profiles into thier own cooking - and BAM! I don't know anyone that carbon copies things from the class stuff; but the ones that have married it with the best of thier own, crank out even better que.

 

[JamesTX]

Or have they touched on mediocrity?

If I ever need a slogan, that will be it - "Touching on Mediocrity"!

Good one.

 

[Podge]

Yea, Pimp I know what you mean.... I think however, that having tasted some of that uber champ BBQ over the last few years -- they are expanding BBQ -- the flavors pop, above the rest. I mean brisket that explodes in your mouth, chicken that is moist and tender. Time and again, you can look at folks that have gone to a class like Rod's class, and have incorporated some of his techniques and/or flavor profiles into thier own cooking - and BAM! I don't know anyone that carbon copies things from the class stuff; but the ones that have married it with the best of thier own, crank out even better que.

Bingo !.. and add in the fact that a lot of teams take competition BBQ serious enough to practice every weekend they get the chance, even in the winter.