Judging contests--My First Experience

Outnumbered · Jul 23, 2012

I judged my first contest this weekend and I was flabbergasted by the lack of knowledge and consistency from the judges at my table. This is certainly something the KCBS BOD needs to address, and quickly IMO.

I understand there is no way to get away from subjectivity, but the way judges score needs to be addressed. I was lectured by the table captain and nearly every judge at my table about the need to be liberal because of the time and money the teams put into competing. I agreed to that and explained that I am a competition cook. They shut up immediately and I got the feeling they were intimidated because I may actually know what I'm talking about. (I didn't set them straight, mind you. Truth is, I haven't a clue.

)

Four of the judges at my table all took the liberal approach and bragged that they never give a score of less than a 7 and that's only if they see something wrong with it. There was no consensus, however, on what constitutes a 9.

The other judge at my table said her base score is a 6. I told her that was fine and it is actually the way it's supposed to be, but if every other judge gives 8s for average, and she gives us a 6, how are we supposed to take anything away from that? Not blaming her, mind you, since she's technically following rules. But what happens if someone cooks the best BBQ on the planet and the table judges on the 6 scale? Meanwhile mine may be OK, but the table I get uses 8 as average, and I get 9s do I deserve to win?

Furthermore, my table captain looked at me like I had two heads when I started filling out comment cards. In one case, the rib was just plain bad and had a chemical taste to it. I wanted to let the cook know. In the other case I scored down a little on presentation and wanted to let the cook know what kept me from giving them a perfect score across the board.

I was the only one who filled out a card and I was made the feel guilty for doing it. My perception is that the judges (at my table anyway) don't feel confident enough in their BBQ knowledge to fill out comment cards. And if that's the case, why are they allowed to judge?

If we can't require judges to use a consistent scale, can we at least require them to fill out comment cards on each entry so we have an understanding of why the scores are the way they are? And maybe it may make the judges put a little more thought into it, if they have to defend their scores?

Zig · Jul 23, 2012

I agree, there is no consistancy in the judges. As a competator, how am I to improve if I can't trust the judges opnion, if you get one. I get the impression they think they are at the sample line at Sams Club. (Just bring me food) you get the score you get, just feed me.
I am a judge also, I thought it would help with competition, guess what?they need to get everyone on the same page. This isn' a buffet.

speers90 · Jul 23, 2012

I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

I guess I don't necessarily agree that everything should start out as a 6, why shouldn't it start out as a 9 and then come down from there???

JimmyDAL · Jul 23, 2012

I agree with the comment cards being mandatory.

Smoke Ring · Jul 23, 2012

I've been reassured by a board member that nothing is wrong with the KCBS judging procedures and the board shouldn't waste any time on it. There is no such thing as inconsistent judging, only inconsistent cooking.

big matt · Jul 23, 2012

speers90 said:
I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

I guess I don't necessarily agree that everything should start out as a 6, why shouldn't it start out as a 9 and then come down from there???

My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?

TooSaucedToPork · Jul 23, 2012

Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

Leatherheadiowa · Jul 23, 2012

big matt said:
My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?

I have tasted both of their bbq and I can tell you that it is great! As a cook and a judge, more often a cook the fact that the score ranges at any given table show discrepancies weekly. When judges still talk about the garnish, sauce on clam shell lids, and compare entries we have a problem. Realizing that in theory that an entrants bbq shouldn't hit the same table twice there shouldn't be issues with scores ranging from 6 to 9. Unfortunately, it happens and when teams get their printed scores you can see the variations.
Now, while easy to complain about how do we take the human inperfections out of competition bbq? We don't so we recruit, retain, and support the leadership that improves the quality of competition bbq. While it is easy to paint a negative picture, please take a moment and email or call the people that work hard to maintain the integrity and thank them!

speers90 · Jul 23, 2012

Smoke Ring said:
I've been reassured by a board member that nothing is wrong with the KCBS judging procedures and the board shouldn't waste any time on it. There is no such thing as inconsistent judging, only inconsistent cooking.

Isn't there always going to be a certain level of inconsistency in a competition where subjectivity is part of the scoring method?

With that being said, I would love to see a more defined criteria for the subjective nature of judging:

Maybe something like this:
9 - unbelievable, can't wait to take another bite
8 - great, would eat more
7 - good, one bite was enough
6 - edible, but no more for me

Even with a gauge like this, it is still possible for two different judges to have scores that vary widely on taste, because taste is so subjective.

For example, I can barely stomach the taste of a sauce that has vinegar as it's primary ingredient. I have such a sensitivity to it that I honestly can't taste anything else if it's present. I can't imagine there would be any possible way for me to judge any higher than 6 or 7 if presented with such a sample. I know this about myself so I would not even consider judging a contect in a region where entries are likely to be turned in with a vinegar based sauces.

First thing I would take a look at is: am I doing something that is turning some judges off? Even if it tastes great to you, it may be turning one or two judges off. I personally will be tweaking my ribs a little bit even though I've had two decent showings because they definately have a strong flavor that may be off putting to some judges even though I think it is the best thing on the planet.

The point I am getting at here is: if some teams have figured out how to consistently finish in the top 5 then it can be done. We have come to the conclusion that it is more likely an issue with something we are doing than with the judging.

I can say from my own experiences I feel like the judging has been in line with our turn ins for our first three comps. I just keep pressing on knowing that if there are teams out there that can consistently finish in the top 5 then we can too. Right now we need to work on our consistency, we have had success in each category but not all at the same competition.

Maybe it's possible that my experiences, thus far, are unique because my region has above average judging?

big matt · Jul 23, 2012

TooSaucedToPork said:
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to

speers90 · Jul 23, 2012

big matt said:
My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?

Exactly! That's is what I am getting at. These two teams, among others, have figured out not only how to cook great bbq but in the process have also figured out a flavor profile that isn't pissing anyone off either.

Therefore, I know it can be done and I will continue to work to get there.

"Holy Smokes" · Jul 23, 2012

As a first time competitor this past weekend, that started out judging...I will fill out comment cards from here on out, no question!!! 7 or less on appearance will get a comment card and 6 or less on taste/tenderness will get a card also. It should be in the rules and governing documents for all the judges to abide by.

I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.

big matt · Jul 23, 2012

TooSaucedToPork said:
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

I'm gonna kind of blow that theory out of the water..this last New Year Joe flew in here (Ca) cooked on borrowed equipment and pulled RGC the first day and GC the next..and he didn't cook for any class then..like I said great Q is great Q..same can be said about Scott Nelson he's done the same stuff.

TooSaucedToPork · Jul 23, 2012

big matt said:
I'm gonna kind f blow that theory out of the water..this last New Year Joe flew in here (Ca) cooked on borrowed equipment and pulled RGC the first day and GC the next..and he didn't cook for any class then..like I said great Q is great Q..same can be said about Scott Nelson he's done the same stuff.

Like I said, Devil's Advocate...In the years that I have been BBQing I have seen a few teams that were top ten, go to first consistantly after always volunteering to cook the class for the judges. Nothing against anyone...notice I did acknowledge that those guys can cook, and cook well...

speers90 · Jul 23, 2012

TooSaucedToPork said:
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

I don't know whether they have ever cooked for a judging class or not, but my own personal experience and others from whom I have spoken with is that no cooks are preparing competion quality bbq for those judging classes.

I have never tasted either of their foods but I don't have a doubt that they can both flat out cook bbq and that is why they are in the running every single week.

I guess from my perspective I sort of agree with the KCBS on this; because, if the same teams are repeatedly winning or in the running to win then is there really an issue with judging?

It sort of reminds me of throwing the shot & discus in high school, every year at the state meet you would see different athletes from the same schools. The only logical conclusion is that the coaches at those schools know something that the others didn't. Because while the athletes come and go, the same coach with the same school would be there year after year.

Thus, my reasoning, if the judges change weekly, and the same teams are still finishing in the top 5 then I have to believe that the teams know something that I don't, yet!

Outnumbered · Jul 23, 2012

speers90 said:
I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

Haven't had their BBQ, but my guess is that it's pretty darn good. That's the first reason. Second reason I have is only a theory. The majority of the judges I met were judging a contest last weekend. Missouri had two last week, and most were at one contest or the other. That would tell me that we have the same judges, judging the same cooks.

To help support my theory, let me use the example A Boy & His BBQ. Shad, too, is a pretty good cook. He cooked at Weston two weeks ago, as did I. Many of the judges at the comp I judged this weekend also judged at Weston. Shad got 34th of 50 teams. For those who don't know, this is the guy who won the 2nd season of BBQ Pitmasters. I would say he's probably consistently closer to GC than he is 34th of 50.

Anyway, he won the Sam's Club round in Minnesota this past weekend. I would imagine he maybe adjusted a little bit from one week to the next, but I can't imagine he made adjustments to get that much better.

Maybe he had a better cook this weekend than the week before. Could be and I'm sure some on here will say he did. I honestly doubt, however, that it was that much better of a cook than what he did the week prior.

Outnumbered · Jul 23, 2012

My best guess is that it's a combination of these two things, but nothing will ever trump cooking good Q.

That doesn't change my belief that the issue of consistency needs to be addressed, however. If you use the same 5-6 teams in the running all the time as the reason to justify there is no problem with consistency, you're fooling yourself. They get it too. If not, you may as well limit comps to those same 5-6 teams, and the group of locals who pull a team together once a year...cause that's what you're going to end up with if the 5-10 comps/year teams feel like they're beating their heads against a wall.

Shotgun · Jul 23, 2012

I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.[/QUOTE]

This could be a little dangerous too, some may just score up as not to fill out any cards. I wont knock the judges for giving their opinion and altough it might sound strange, I think the problem lies in the Reps not having full control of the judging area. Every judge should be prepared to justify a score weather its a 3 or a 9 and think that reps or TC should pull random cards for just that. There is no accountablity.

speers90 · Jul 23, 2012

Outnumbered said:
My best guess is that it's a combination of these two things, but nothing will ever trump cooking good Q.

That doesn't change my belief that the issue of consistency needs to be addressed, however. If you use the same 5-6 teams in the running all the time as the reason to justify there is no problem with consistency, you're fooling yourself. They get it too. If not, you may as well limit comps to those same 5-6 teams, and the group of locals who pull a team together once a year...cause that's what you're going to end up with if the 5-10 comps/year teams feel like they're beating their heads against a wall.

I guess we may disagree on which end the consistency needs to be addressed. My view is that the top teams are consistently turning in good to great bbq and are conistently finishing in the running. As pointed out by top cooks/teams that travel outside of their "home" region and have equally good results.

It is not unheard of for a team that is doing well to have an off week every once in a while, but if asked I wonder if they would say it was the judges or maybe something happened out of the ordinary for that particular comp???

My point is that from my perspective what seperates our team from Big T'z, Tippy, Spitfire, etc... is not the judging but our own inconsistent performing.

In the 3 contests that we have cooked in we have had: chicken 5th & 10th, ribs 10th, pork 6th and brisket 5th. We came close to putting it all together at our last event with calls in 3 out of 4 categories, but I didn't get the brisket where it needed to be. After some time to cool off and time to honestly reflect on the brisket that I turned in, it wasn't the judges fault that I didn't get a higher score, it was all me. I screwed it up and that is my fault.

At any one contest, there is a possibility for judging to be inconsistent, but if over the course of the year, the same teams are winning or in the running then I still can't believe that the judging is the issue.

I am sure the top teams understand that there is likely some inconsistency in judging, as there always will be because the majority of the scoring is subjective. What they do better than the rest of us is put together a flavor profile that is less offensive to the pool of judges as a whole then we do.

I feel like if someone is a better (insert anything) than me then there is a reason, but if I want it bad enough then I can learn/change to get to that point as well. That is the way I was brought up, that is the way I was coached through athletics and that is what I will pass on to my boys. There may be legitimate reasons why that person is better than me, but if I want it bad enough I can get there too!

speers90 · Jul 23, 2012

Shotgun said:
I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.

This could be a little dangerous too, some may just score up as not to fill out any cards. I wont knock the judges for giving their opinion and altough it might sound strange, I think the problem lies in the Reps not having full control of the judging area. Every judge should be prepared to justify a score weather its a 3 or a 9 and think that reps or TC should pull random cards for just that. There is no accountablity.[/QUOTE]

That may be one of the best idea's I have heard yet on how to provide honest feedback without compromising the score that would be given by a judge.

I am not saying that there are not ways to improve what is currently happening but lets not kid ourselves into thinking that we can "regulate" our way into consistent scoring when the very nature of judging bbq is subjective.

I also would be in favor of judging one box at a time.

Judging contests--My First Experience

is One Chatty Farker

Knows what a fatty is.

is Blowin Smoke!

is one Smokin' Farker

Smoke Ring

Guest

is one Smokin' Farker

is one Smokin' Farker

Full Fledged Farker

is Blowin Smoke!

is one Smokin' Farker

is Blowin Smoke!

Is lookin for wood to cook with.

is one Smokin' Farker

is one Smokin' Farker

is Blowin Smoke!

is One Chatty Farker

is One Chatty Farker

Full Fledged Farker

is Blowin Smoke!

is Blowin Smoke!

Similar threads