PDA

View Full Version : Judging contests--My First Experience


Outnumbered
07-23-2012, 10:41 AM
I judged my first contest this weekend and I was flabbergasted by the lack of knowledge and consistency from the judges at my table. This is certainly something the KCBS BOD needs to address, and quickly IMO.

I understand there is no way to get away from subjectivity, but the way judges score needs to be addressed. I was lectured by the table captain and nearly every judge at my table about the need to be liberal because of the time and money the teams put into competing. I agreed to that and explained that I am a competition cook. They shut up immediately and I got the feeling they were intimidated because I may actually know what I'm talking about. (I didn't set them straight, mind you. Truth is, I haven't a clue. :-P)

Four of the judges at my table all took the liberal approach and bragged that they never give a score of less than a 7 and that's only if they see something wrong with it. There was no consensus, however, on what constitutes a 9.

The other judge at my table said her base score is a 6. I told her that was fine and it is actually the way it's supposed to be, but if every other judge gives 8s for average, and she gives us a 6, how are we supposed to take anything away from that? Not blaming her, mind you, since she's technically following rules. But what happens if someone cooks the best BBQ on the planet and the table judges on the 6 scale? Meanwhile mine may be OK, but the table I get uses 8 as average, and I get 9s do I deserve to win?

Furthermore, my table captain looked at me like I had two heads when I started filling out comment cards. In one case, the rib was just plain bad and had a chemical taste to it. I wanted to let the cook know. In the other case I scored down a little on presentation and wanted to let the cook know what kept me from giving them a perfect score across the board.

I was the only one who filled out a card and I was made the feel guilty for doing it. My perception is that the judges (at my table anyway) don't feel confident enough in their BBQ knowledge to fill out comment cards. And if that's the case, why are they allowed to judge?

If we can't require judges to use a consistent scale, can we at least require them to fill out comment cards on each entry so we have an understanding of why the scores are the way they are? And maybe it may make the judges put a little more thought into it, if they have to defend their scores?

Zig
07-23-2012, 11:25 AM
I agree, there is no consistancy in the judges. As a competator, how am I to improve if I can't trust the judges opnion, if you get one. I get the impression they think they are at the sample line at Sams Club. (Just bring me food) you get the score you get, just feed me.
I am a judge also, I thought it would help with competition, guess what?they need to get everyone on the same page. This isn' a buffet.

speers90
07-23-2012, 11:32 AM
I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

I guess I don't necessarily agree that everything should start out as a 6, why shouldn't it start out as a 9 and then come down from there???

JimmyDAL
07-23-2012, 11:46 AM
I agree with the comment cards being mandatory.

Smoke Ring
07-23-2012, 12:28 PM
I've been reassured by a board member that nothing is wrong with the KCBS judging procedures and the board shouldn't waste any time on it. There is no such thing as inconsistent judging, only inconsistent cooking.

big matt
07-23-2012, 12:48 PM
I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

I guess I don't necessarily agree that everything should start out as a 6, why shouldn't it start out as a 9 and then come down from there???

My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?

TooSaucedToPork
07-23-2012, 12:58 PM
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

Leatherheadiowa
07-23-2012, 01:06 PM
My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?
I have tasted both of their bbq and I can tell you that it is great! As a cook and a judge, more often a cook the fact that the score ranges at any given table show discrepancies weekly. When judges still talk about the garnish, sauce on clam shell lids, and compare entries we have a problem. Realizing that in theory that an entrants bbq shouldn't hit the same table twice there shouldn't be issues with scores ranging from 6 to 9. Unfortunately, it happens and when teams get their printed scores you can see the variations.
Now, while easy to complain about how do we take the human inperfections out of competition bbq? We don't so we recruit, retain, and support the leadership that improves the quality of competition bbq. While it is easy to paint a negative picture, please take a moment and email or call the people that work hard to maintain the integrity and thank them!

speers90
07-23-2012, 01:10 PM
I've been reassured by a board member that nothing is wrong with the KCBS judging procedures and the board shouldn't waste any time on it. There is no such thing as inconsistent judging, only inconsistent cooking.

Isn't there always going to be a certain level of inconsistency in a competition where subjectivity is part of the scoring method?

With that being said, I would love to see a more defined criteria for the subjective nature of judging:

Maybe something like this:
9 - unbelievable, can't wait to take another bite
8 - great, would eat more
7 - good, one bite was enough
6 - edible, but no more for me

Even with a gauge like this, it is still possible for two different judges to have scores that vary widely on taste, because taste is so subjective.

For example, I can barely stomach the taste of a sauce that has vinegar as it's primary ingredient. I have such a sensitivity to it that I honestly can't taste anything else if it's present. I can't imagine there would be any possible way for me to judge any higher than 6 or 7 if presented with such a sample. I know this about myself so I would not even consider judging a contect in a region where entries are likely to be turned in with a vinegar based sauces.

First thing I would take a look at is: am I doing something that is turning some judges off? Even if it tastes great to you, it may be turning one or two judges off. I personally will be tweaking my ribs a little bit even though I've had two decent showings because they definately have a strong flavor that may be off putting to some judges even though I think it is the best thing on the planet.

The point I am getting at here is: if some teams have figured out how to consistently finish in the top 5 then it can be done. We have come to the conclusion that it is more likely an issue with something we are doing than with the judging.

I can say from my own experiences I feel like the judging has been in line with our turn ins for our first three comps. I just keep pressing on knowing that if there are teams out there that can consistently finish in the top 5 then we can too. Right now we need to work on our consistency, we have had success in each category but not all at the same competition.

Maybe it's possible that my experiences, thus far, are unique because my region has above average judging?

big matt
07-23-2012, 01:12 PM
[quote=TooSaucedToPork;2146809]Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to

speers90
07-23-2012, 01:14 PM
My opinion only here..great BBQ is great BBQ no matter where they go..while I haven't tasted Ryan or Joe's food my guess is that it's just kick ass..everytime..and they probably cook to the middle?

Exactly! That's is what I am getting at. These two teams, among others, have figured out not only how to cook great bbq but in the process have also figured out a flavor profile that isn't pissing anyone off either.

Therefore, I know it can be done and I will continue to work to get there.

"Holy Smokes"
07-23-2012, 01:16 PM
As a first time competitor this past weekend, that started out judging...I will fill out comment cards from here on out, no question!!! 7 or less on appearance will get a comment card and 6 or less on taste/tenderness will get a card also. It should be in the rules and governing documents for all the judges to abide by.

I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.

big matt
07-23-2012, 01:18 PM
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

I'm gonna kind of blow that theory out of the water..this last New Year Joe flew in here (Ca) cooked on borrowed equipment and pulled RGC the first day and GC the next..and he didn't cook for any class then..like I said great Q is great Q..same can be said about Scott Nelson he's done the same stuff.

TooSaucedToPork
07-23-2012, 01:19 PM
I'm gonna kind f blow that theory out of the water..this last New Year Joe flew in here (Ca) cooked on borrowed equipment and pulled RGC the first day and GC the next..and he didn't cook for any class then..like I said great Q is great Q..same can be said about Scott Nelson he's done the same stuff.

Like I said, Devil's Advocate...In the years that I have been BBQing I have seen a few teams that were top ten, go to first consistantly after always volunteering to cook the class for the judges. Nothing against anyone...notice I did acknowledge that those guys can cook, and cook well...

speers90
07-23-2012, 01:25 PM
Did Big T'z & Tippy Canoe cook for the judging classes? Their BBQ could be what all other BBQ in that region is getting compared to. Not saying that this is what happened, those guys can cook, and cook well...just playing devil's advocate.

My opinion is there will be no proactive fix on this issue. It will be allowed to fester and it will eventually be a huge sore spot to many of us that cook in KCBS. Then after it blows up, the powers that be will try to address it and it will be too late. Welcome to politics...

I don't know whether they have ever cooked for a judging class or not, but my own personal experience and others from whom I have spoken with is that no cooks are preparing competion quality bbq for those judging classes.

I have never tasted either of their foods but I don't have a doubt that they can both flat out cook bbq and that is why they are in the running every single week.

I guess from my perspective I sort of agree with the KCBS on this; because, if the same teams are repeatedly winning or in the running to win then is there really an issue with judging?

It sort of reminds me of throwing the shot & discus in high school, every year at the state meet you would see different athletes from the same schools. The only logical conclusion is that the coaches at those schools know something that the others didn't. Because while the athletes come and go, the same coach with the same school would be there year after year.

Thus, my reasoning, if the judges change weekly, and the same teams are still finishing in the top 5 then I have to believe that the teams know something that I don't, yet!

Outnumbered
07-23-2012, 01:30 PM
I don't disagree with what you are saying or with what you experienced, but explain to me how, in my region, Big T'z & Tippy Canoe can consistently finish in the top 5 (at dang near every contest) if the judges are all over the board.

Haven't had their BBQ, but my guess is that it's pretty darn good. That's the first reason. Second reason I have is only a theory. The majority of the judges I met were judging a contest last weekend. Missouri had two last week, and most were at one contest or the other. That would tell me that we have the same judges, judging the same cooks.

To help support my theory, let me use the example A Boy & His BBQ. Shad, too, is a pretty good cook. He cooked at Weston two weeks ago, as did I. Many of the judges at the comp I judged this weekend also judged at Weston. Shad got 34th of 50 teams. For those who don't know, this is the guy who won the 2nd season of BBQ Pitmasters. I would say he's probably consistently closer to GC than he is 34th of 50.

Anyway, he won the Sam's Club round in Minnesota this past weekend. I would imagine he maybe adjusted a little bit from one week to the next, but I can't imagine he made adjustments to get that much better.

Maybe he had a better cook this weekend than the week before. Could be and I'm sure some on here will say he did. I honestly doubt, however, that it was that much better of a cook than what he did the week prior.

Outnumbered
07-23-2012, 01:40 PM
My best guess is that it's a combination of these two things, but nothing will ever trump cooking good Q.

That doesn't change my belief that the issue of consistency needs to be addressed, however. If you use the same 5-6 teams in the running all the time as the reason to justify there is no problem with consistency, you're fooling yourself. They get it too. If not, you may as well limit comps to those same 5-6 teams, and the group of locals who pull a team together once a year...cause that's what you're going to end up with if the 5-10 comps/year teams feel like they're beating their heads against a wall.

Shotgun
07-23-2012, 02:20 PM
I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.[/QUOTE]

This could be a little dangerous too, some may just score up as not to fill out any cards. I wont knock the judges for giving their opinion and altough it might sound strange, I think the problem lies in the Reps not having full control of the judging area. Every judge should be prepared to justify a score weather its a 3 or a 9 and think that reps or TC should pull random cards for just that. There is no accountablity.

speers90
07-23-2012, 02:23 PM
My best guess is that it's a combination of these two things, but nothing will ever trump cooking good Q.

That doesn't change my belief that the issue of consistency needs to be addressed, however. If you use the same 5-6 teams in the running all the time as the reason to justify there is no problem with consistency, you're fooling yourself. They get it too. If not, you may as well limit comps to those same 5-6 teams, and the group of locals who pull a team together once a year...cause that's what you're going to end up with if the 5-10 comps/year teams feel like they're beating their heads against a wall.

I guess we may disagree on which end the consistency needs to be addressed. My view is that the top teams are consistently turning in good to great bbq and are conistently finishing in the running. As pointed out by top cooks/teams that travel outside of their "home" region and have equally good results.

It is not unheard of for a team that is doing well to have an off week every once in a while, but if asked I wonder if they would say it was the judges or maybe something happened out of the ordinary for that particular comp???

My point is that from my perspective what seperates our team from Big T'z, Tippy, Spitfire, etc... is not the judging but our own inconsistent performing.

In the 3 contests that we have cooked in we have had: chicken 5th & 10th, ribs 10th, pork 6th and brisket 5th. We came close to putting it all together at our last event with calls in 3 out of 4 categories, but I didn't get the brisket where it needed to be. After some time to cool off and time to honestly reflect on the brisket that I turned in, it wasn't the judges fault that I didn't get a higher score, it was all me. I screwed it up and that is my fault.

At any one contest, there is a possibility for judging to be inconsistent, but if over the course of the year, the same teams are winning or in the running then I still can't believe that the judging is the issue.

I am sure the top teams understand that there is likely some inconsistency in judging, as there always will be because the majority of the scoring is subjective. What they do better than the rest of us is put together a flavor profile that is less offensive to the pool of judges as a whole then we do.

I feel like if someone is a better (insert anything) than me then there is a reason, but if I want it bad enough then I can learn/change to get to that point as well. That is the way I was brought up, that is the way I was coached through athletics and that is what I will pass on to my boys. There may be legitimate reasons why that person is better than me, but if I want it bad enough I can get there too!

speers90
07-23-2012, 02:29 PM
I don't think that the organizers push the judges to fill out comment cards, but they should and this would help alleviate a lot of issues. When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.

This could be a little dangerous too, some may just score up as not to fill out any cards. I wont knock the judges for giving their opinion and altough it might sound strange, I think the problem lies in the Reps not having full control of the judging area. Every judge should be prepared to justify a score weather its a 3 or a 9 and think that reps or TC should pull random cards for just that. There is no accountablity.[/QUOTE]

That may be one of the best idea's I have heard yet on how to provide honest feedback without compromising the score that would be given by a judge.

I am not saying that there are not ways to improve what is currently happening but lets not kid ourselves into thinking that we can "regulate" our way into consistent scoring when the very nature of judging bbq is subjective.

I also would be in favor of judging one box at a time.

Outnumbered
07-23-2012, 02:30 PM
Speers, I would agree with virtually everything you said. I think we've gotten away from the original point I was trying make with this, however. It is that there is no consistency to setting a basis for what the judges should view as average...is it a 6 or 8? That is defined, but it is not reinforced.

I like the version of scoring explanations that you proposed earlier. That would make a lot of sense to me.

speers90
07-23-2012, 02:38 PM
Speers, I would agree with virtually everything you said. I think we've gotten away from the original point I was trying make with this, however. It is that there is no consistency to setting a basis for what the judges should view as average...is it a 6 or 8? That is defined, but it is not reinforced.

I like the version of scoring explanations that you proposed earlier. That would make a lot of sense to me.

Sorry, sometimes I wonder off on a little tangent :blah:

My personal feeling is that a judge at a bbq competition ought to be expecting a 9 and then to adjust from there. As a cook, I understand that there is and alwasy will be subjectivity in this hobby that I am growing to love, so I will not sweat any score that is either an 8 or 9.

big matt
07-23-2012, 03:06 PM
Like I said, Devil's Advocate...In the years that I have been BBQing I have seen a few teams that were top ten, go to first consistantly after always volunteering to cook the class for the judges. Nothing against anyone...notice I did acknowledge that those guys can cook, and cook well...
I see your point here but cooking for a class and cooking for a comp are two different things..even if they did cook for a class it wouldn't even close to what they turn in..there is just too much that goes into comp q for it to be replicated on a mass level(IMO)..furthermore they win in more than one state so could there be that many judges that know their flavor profile?..bottom-line is when you get to their level people will question why they are always up there..my point is they just cook kick ass Q..and I know your saying that too I just don't think they are winning because the cooked a class or not.

TooSaucedToPork
07-23-2012, 03:42 PM
So the age old argument comes up again…

You have KCBS teams that are doing very well, they have great product, they are getting great scores from the judges, and they are winning contests. They don’t NEED comment cards or judges input.

THEN

You have the other 98-99% of KCBS teams that are doing ok, they have good product, they might get a call or two each contest, maybe top 10 or so in the contests they attend…They just need information on what is keeping them from a couple 9’s to get them that first place win that they have been after for soooo long. Wow, if only there were something that could help these teams understand what they are doing wrong at turn-in…something that could help them understand what the judges are seeing…

Oh yeah, comment cards that aren’t used…

Plus 80% of KCBS judges skedaddle right after judging… 80% of MBN/MIM judges will come back and visit if you ask, they tell you their opinions and exactly what they thought was wrong with your product…if this were in writing, it would be a comment card.

Everyone always argues inconsistency in cooking…well...how do the cooks know what is inconsistent or wrong in the judges eyes without feedback?

Y’all gotta remember - The rules we’re written when there weren’t near as many teams competing. Look how many contests and teams were around 7-10 years ago compared to now. BBQ has exploded and there are now many new teams grasping at straws, trying to figure out what they are doing wrong. This sport is about friendship and spreading the BBQ Love as well as Competition.

So once again I plead!

Judges – Please start filling out comment cards with your opinion for all entries – I do it when I judge…it ain’t hard…You will gain the respect and admiration of many cooks. You might even be the change necessary to get a team their first RGC or GC. Wouldn’t that make you feel all warm and fuzzy inside?

And I promise you that if you do this consistently, you will see better and better product come across your placemat. Its win/win y’all!!!

speers90
07-23-2012, 04:02 PM
So the age old argument comes up again…

You have KCBS teams that are doing very well, they have great product, they are getting great scores from the judges, and they are winning contests. They don’t NEED comment cards or judges input.

THEN

You have the other 98-99% of KCBS teams that are doing ok, they have good product, they might get a call or two each contest, maybe top 10 or so in the contests they attend…They just need information on what is keeping them from a couple 9’s to get them that first place win that they have been after for soooo long. Wow, if only there were something that could help these teams understand what they are doing wrong at turn-in…something that could help them understand what the judges are seeing…

Oh yeah, comment cards that aren’t used…

Plus 80% of KCBS judges skedaddle right after judging… 80% of MBN/MIM judges will come back and visit if you ask, they tell you their opinions and exactly what they thought was wrong with your product…if this were in writing, it would be a comment card.

Everyone always argues inconsistency in cooking…well...how do the cooks know what is inconsistent or wrong in the judges eyes without feedback?

Y’all gotta remember - The rules we’re written when there weren’t near as many teams competing. Look how many contests and teams were around 7-10 years ago compared to now. BBQ has exploded and there are now many new teams grasping at straws, trying to figure out what they are doing wrong. This sport is about friendship and spreading the BBQ Love as well as Competition.

So once again I plead!

Judges – Please start filling out comment cards with your opinion for all entries – I do it when I judge…it ain’t hard…You will gain the respect and admiration of many cooks. You might even be the change necessary to get a team their first RGC or GC. Wouldn’t that make you feel all warm and fuzzy inside?

And I promise you that if you do this consistently, you will see better and better product come across your placemat. Its win/win y’all!!!

I would love to get comment cards, but for us it has been relatively easy to figure out what they liked and what they didn't like from the scores.

JeffR
07-23-2012, 10:13 PM
Having just completed judging class, I must say I was expecting to walk away with a better understanding of what the target benchmark is.

I did walk away with hearing plenty of "judge the meat" :-D
Scores were all over the place from the class. I do suspect that as judges judge more comps, their level of expectation will have to increase as the will start to get so good Q.

They did a pretty good job of defining rib and brisket tenderness, but not for the other two. And flavor profile really wasn't covered well, except for things should be in balance.

I agree that comment cards should be filled out. Perhaps for every score.

Tack
07-23-2012, 10:51 PM
Inconsitancy in judging??? I guess getting a 999 from one judge and a 765 from another on the same box qualifies???

Outnumbered
07-23-2012, 10:57 PM
Inconsitancy in judging??? I guess getting a 999 from one judge and a 765 from another on the same box qualifies???

Wow. :twitch::frusty:

speers90
07-23-2012, 11:03 PM
Inconsitancy in judging??? I guess getting a 999 from one judge and a 765 from another on the same box qualifies???

What were the other 4?

All those two numbers tell me is that one judge liked what you did and one did not, in order to get a better understanding we need to look at all the scores?

speers90
07-23-2012, 11:09 PM
I think what I might be trying to say is don't get so focused in on what one judge is scoring you on, look at the whole and figure out what that is telling you.

Anyone who works with statistics knows that any one data point can be way off, but as more and more information becomes available you are able to make some fairly educated conclusions about the "whole" of the scenario.

I also believe that is why in the scoring system the bottom scores are dropped. The system tries to overcome some of the inconsistency by removing the outlying data.

TooSaucedToPork
07-24-2012, 07:21 AM
I would love to get comment cards, but for us it has been relatively easy to figure out what they liked and what they didn't like from the scores.

Okay…this is not me being a DB, I'm not attacking you in any way. I’m just trying to understand how to pull exact judges views out of plain numbers.

Here are my rib scores from the last contest we did, please tell me exactly what we did wrong last contest, and exactly how we can improve our product without anything but these numbers.

9/8/8 9/8/9 9/9/8 9/8/8 7/7/7 7/8/9

Again, not being an a$$, just trying to understand.

:becky:Once agian...thwarted by that dang judge #5:becky:

Disconnect
07-24-2012, 07:22 AM
We've been lucky to get good reps who asked the crazy judge wtf ("didn't look appatizing".. but at least they asked.)

Oops. Thought I had the score sheet available, but in any case it was 8s, 9s and a 4... No card. Oh, and he was on par with everyone else for taste/tenderness so his score stood.

Sent from my SGH-T999 using Tapatalk 2

speers90
07-24-2012, 08:25 AM
Okay…this is not me being a DB, I'm not attacking you in any way. I’m just trying to understand how to pull exact judges views out of plain numbers.

Here are my rib scores from the last contest we did, please tell me exactly what we did wrong last contest, and exactly how we can improve our product without anything but these numbers.

9/8/8 9/8/9 9/9/8 9/8/8 7/7/7 7/8/9

Again, not being an a$$, just trying to understand.

:becky:Once agian...thwarted by that dang judge #5:becky:

Here is the tricky part, it is going to be hard for me to evaluate your scores because I didn't taste your product or see the box. That is where the honest self evaluation comes in with the aid of the scores.

I don't put much weight into judge 5's scores, because he/she is the statistical outlier.

For your scores, I would say that there was obviously something that judge 5 & 6 saw that negatively effected your appearance score. Again I don't worry to much about judge 5's opinion but in this case, judge 6 (who seems to be a reasonable judge) confirmed what judge 5 saw. Hard for me to say without seeing the box. Which I guess this is where you are saying a comment card would be nice :grin:

As far as taste, at this particular contest it looks like you accomplished a flavor profile that didn't tick anyone off but your not wowing them either, if your getting similar resulsts each and every competition then it may be time to tinker with the flavor profile to work on getting more of the judges to think "wow". I don't think comment cards here are going to help you though, unless you are scoring 6's and 7's. I would be willing to bet that most people and judge are people, don't know what wow's them until they taste it.

And for tenderness, looks like you had a good contest.

Like I said before, because the judging for our sport is highly subjective I don't sweat 8's at any one competition. Meaning I won't make changes to my technique or recipe because of 8's I get at any one competition. I want to see what the scores are over a larger sample, thus in my case I am making a small tweak to my rib recipe based on 3 competitions worth of results.

But yes, I would love to get comment cards. Even though I see posts on this forum from time to time that show comments from judges and they are even more puzzling than the scores. So would they really help? who knows!

Not sure if that helps at all, but that is sort of my thinking process after I have cooled down and had time to reflect. The funny part about our last contest was that the statistical outlier for my brisket, which sucked, was high! I received mostly 8's and 7's, and here is one judge that was 9/8/9 or something like that. In this case I had to disregard he/she as the statistical outlier.

Outnumbered
07-24-2012, 08:33 AM
Statistics? You're working a very small sample size to let that anomaly happen. And maybe the anomaly is on the high side? Either way, something's wrong.

I will go back to the original post and say that something needs to be addressed by the BOD at KCBS.

Outnumbered
07-24-2012, 08:37 AM
And Speers, I'm glad to see there is somebody on here that has a similar line of obsessiveness with their scores. :) Do you have a spread sheet?

I do. I have all the scores, the total possible, how many we missed for each category, what the percentage of each category's points we received, etc. Then I combine all the scores for each category in a separate sheet and compare from contest to contest to see where we're improving and falling off.

My wife makes fun of me, but she did a damn spreadsheet for each day when we went to Disney World. This is far more important. :wink:

speers90
07-24-2012, 10:27 AM
And Speers, I'm glad to see there is somebody on here that has a similar line of obsessiveness with their scores. :) Do you have a spread sheet?

I do. I have all the scores, the total possible, how many we missed for each category, what the percentage of each category's points we received, etc. Then I combine all the scores for each category in a separate sheet and compare from contest to contest to see where we're improving and falling off.

My wife makes fun of me, but she did a damn spreadsheet for each day when we went to Disney World. This is far more important. :wink:

No I don't keep spread sheets, that is Trent's (TTNuge) deal. He scans our results and then emails them to me, I go back and take a look from time to time.

I guess it would help if I also mentioned that I feel like the subjectivity is high enough that I shoot for a base score and if I hit that score then there really isn't much else I could do, for any one given contest. Here again though, there is no way to take the subjectivity out of our sport, so we as cook have to learn to deal with it. Could KCBS help judges a little more with how to score their subjective opinions, absolutely! Give them a simple chart:

9 - can't wait to get another bit
8 - would eat more
7 - good but no more for me
6 - edible but get it away from me

And no, my base score is not 180! If over the course of time, I continue to hit that base score (developing consistency) then it will be time to tinker and try to increase. That is why after 3 events I am going to tinker with my rib recipe a little bit. So far, the judges have consistently told me the ribs are really good but not great, yet!

As I continue to hit that score and make tweaks I expect my average base score to rise and that will continually put us in contention. FYI, we have a ways to go at this point, especially with the consistency across all 4 categories.

And don't even get me started on Disney World, we are surprising our boys Christmas morning and heading down, she has been obsessing over it like it is more important than BBQ :wacko:

Tack
07-24-2012, 10:35 AM
All scores for the box are 888,765,877,999,776,987. Now I have no problem if the guy just didn't like it but damn man tell me why

speers90
07-24-2012, 10:38 AM
Statistics? You're working a very small sample size to let that anomaly happen. And maybe the anomaly is on the high side? Either way, something's wrong.

I will go back to the original post and say that something needs to be addressed by the BOD at KCBS.

Whether or not the sample size is large enough is unimportant in our situation because the scoring system takes into account that there will be an anomaly so as cooks we should take that into account as well when reflecting on our scores.

Luckily for us, the scoring system gives us the benefit of the doubt and throws out the bottom score.

I don't agree that there is something wrong, the scoring is subjective. Even with the score definitions that I posted a couple of times, it is still completely likely that out of 6 judges, you will get one that doesn't care for your product.

I haven't judged, but honestly if you told me I had to fill out a comment card for anything scored 6 or lower, I'd probably make 7 my lowest number. As I see it, the only way to keep from changing scores is to make comment cards mandatory or random. I like either option; although, what are the odds that you will get a random comment card from the judge that scores you the lowest.

speers90
07-24-2012, 10:53 AM
All scores for the box are 888,765,877,999,776,987. Now I have no problem if the guy just didn't like it but damn man tell me why

Tack,

Not trying to be a jerk because those scores look similar to my brisket box last weekend. I would honestly say that in your situation I think the 999 might be the outlier. I would evaluate your scores by ignoring the 999.

So I am going to assume that the 999 is the outlier and ignore it.

For appearance we have: 8,7,8,7,9. Do you have a picture of the box you turned it? It sounds like you are close and a couple of tweaks will have you with all 8's and 9's. Look around for high quality looking boxes and try to replicate those.

For taste we have: 8,6,7,7,8. There is some work to do here, but I can't taste your product so no clue on where to go with that. A comment card from a judge may be helpful in this situation, but you have to remember that is only one judge and they could be the oddball that expects their mouth to be on fire. Also, remember that taste and tenderness will effect each other. So it is possible that your taste scores get higher with better tenderness scores.

For tenderness we have: 8,5,7,6,7. Obviously when looking at it like this, there is a tenderness issue, get that fixed and I bet the taste scores come up a little bit.

Again, no offense intended, just trying to let you in on the process I go through when I am honestly accessing my scores.

mobow
07-24-2012, 12:18 PM
Speers 90. That is a outstanding evaluation. I try the same process for myself. I too am tweaking some this week due to a pattern I am seeing. Great post. keith

speers90
07-24-2012, 01:29 PM
Okay…this is not me being a DB, I'm not attacking you in any way. I’m just trying to understand how to pull exact judges views out of plain numbers.

Here are my rib scores from the last contest we did, please tell me exactly what we did wrong last contest, and exactly how we can improve our product without anything but these numbers.

9/8/8 9/8/9 9/9/8 9/8/8 7/7/7 7/8/9

Again, not being an a$$, just trying to understand.

:becky:Once agian...thwarted by that dang judge #5:becky:

As long as I understand how the judging works then judge #5's score would have been tossed so you were really thwarted by judge #6 and the 7 for appearance, correct?

I went back and re-read the rules on the KCBS site, my understanding is that judge 5's scores would have been dropped.

Tack
07-24-2012, 02:16 PM
No offense taken just would have like to get a comment card on the 5 score. was it too tonder or not tender enough for em??? I personally thought it was slightly to tender and knew it wasn't right. But it seems the other judges weren't too put off by it.

speers90
07-24-2012, 02:53 PM
No offense taken just would have like to get a comment card on the 5 score. was it too tonder or not tender enough for em??? I personally thought it was slightly to tender and knew it wasn't right. But it seems the other judges weren't too put off by it.

This is just speaking for myself and the expectations that I have as a cook for myself. I see those tenderness scores and it immediately pops in my mind that the judges obviously didn't like it, so I don't necessarily agree that "the other judges weren't too put off by it." In fact, I think when looking at the 5 tenderness scores all together, the judges are in fact telling you they didn't like the tenderness at all.

Then it is up to you to determine what happened. Was it not tender enough, to tender, was it perfect when you put in it in the box, did the tenderness change in the 10-20 minutes between cutting, boxing and judging? All of those are possible.

I happen to think the judges gave you a lot of information to help improve your cooking.

The great news is that you can quickly fix appearance and tenderness issues.

Again, this is my opinion, and in my world a 7 is failing.

Smoke Ring
07-24-2012, 02:58 PM
Okay…this is not me being a DB, I'm not attacking you in any way. I’m just trying to understand how to pull exact judges views out of plain numbers.

Here are my rib scores from the last contest we did, please tell me exactly what we did wrong last contest, and exactly how we can improve our product without anything but these numbers.

9/8/8 9/8/9 9/9/8 9/8/8 7/7/7 7/8/9

Again, not being an a$$, just trying to understand.

:becky:Once agian...thwarted by that dang judge #5:becky:

About the only thing you can do is get lucky and have your entry land on a better table next time. Luck is a big part of KCBS contests. Yes you have to know how to cook but you also need to get lucky. I would say that a judging system that includes luck has a serious problem.

speers90
07-24-2012, 03:28 PM
About the only thing you can do is get lucky and have your entry land on a better table next time. Luck is a big part of KCBS contests. Yes you have to know how to cook but you also need to get lucky. I would say that a judging system that includes luck has a serious problem.

No doubt that there is some luck involved, aka subjectivity, but I don't think luck has all that much to do with it. Are you telling me that Big T'z, Tippy, Spitfire, Trigg, etc... are all just lucky? If luck were the prevailing reason then there would be fewer repeat winners.

Maybe instead of luck it is the fact that these repeat winners have perfected their craft so much so that they can cook better competition bbq in their sleep then us mortals.

Maybe instead of luck it is cooking 200+ briskets over the course of a couple years. I know I expect to be dang good brisket cook when I have that much experience and I will be as long as I keep getting better because of honest self-evaluation that I can perform after receiving my scores.

Tack
07-24-2012, 04:23 PM
Okay I guess the real question is which way do you go with the info the judges gave ??? More tender, less? More spice or less?? Without a comment card there is no indication of a direction to go.Comment cards should be madatory for scores of 6 or less

Lake Dogs
07-24-2012, 04:35 PM
... Luck is a big part of KCBS contests. Yes you have to know how to cook but you also need to get lucky. I would say that a judging system that includes luck has a serious problem.

First, luck is a factor in every cooking contest I've ever seen, regardless of sanctioning body. Luck, like which table your product lands on and which product landed on your table with your entry, and which product (aka. BBQ) landed on someone else's table. Then you must get a little lucky that 6 judges agree. Some like it a little salty, others HATE sodium, some like some fat in the pork/beef and others HATE it. Yours a tad salty and get 6 judges that dont mind it, you're getting good. 3 and 3 at your table; you're toast. Just luck this way. Also, some of the luck has to do with which other entries are at your table. I've seen where great product judged immediately after a particularly spicy, hot, or salty product gets scored DOWN while that flavor apparently lingers in everyones mouth. Reverse the order and the score changes. <we've had a lot of fun at chili contests testing these theories>

Yeah, there is some luck; luck of the draw, not a flawed judging system.

speers90
07-24-2012, 04:48 PM
Okay I guess the real question is which way do you go with the info the judges gave ??? More tender, less? More spice or less?? Without a comment card there is no indication of a direction to go.Comment cards should be madatory for scores of 6 or less

To me this is part of the self analysis that we do after each turn in and then we compare what we thought with what the judges scored and then make adjustment if necessary.

For us it has been obvious what the issues were when receiving the 7's and few 6's that we have received thus far.

Outnumbered
07-24-2012, 05:37 PM
It's interesting that the conversation has taken this turn, again. The original point is that there is no consistency in judging, not that people are getting scored too low. In the situation I outlined in the original post, it is likely that people are getting scored too high more often than they are too low.

If you are in this to cook good BBQ, get better, get to a point where you can learn from competitions and do consistently better, THERE NEEDS TO BE A WAY TO ESTABLISH A CONSISTENT BASE OF SCORING.

If one judge uses a 6 as a base and the rest use an 8, how do we take away from that if there is no other feedback? KCBS BOD? Comments?

Tack
07-24-2012, 05:45 PM
trust me I self analyse my scores bigtime I am definitely my biggest critic. but when you turn in what you think is a good product and get the wide range of scores without getting feedback from the judges how do you know what to change?

Outnumbered
07-24-2012, 06:08 PM
trust me I self analyse my scores bigtime I am definitely my biggest critic. but when you turn in what you think is a good product and get the wide range of scores without getting feedback from the judges how do you know what to change?

Ding, ding, ding...We have a winner.

Exactly, Tack.

mobow
07-24-2012, 06:26 PM
trust me I self analyse my scores bigtime I am definitely my biggest critic. but when you turn in what you think is a good product and get the wide range of scores without getting feedback from the judges how do you know what to change?

I do not care about feedback from this table. They can't agree among themselves what to score. This the table that needs help on cosistency. And my experience has been that the ones that need the most work are just as likely to be high scoring judges as low scorin judges. Keith

Outnumbered
07-24-2012, 06:37 PM
Quote:
Originally Posted by Tack
trust me I self analyse my scores bigtime I am definitely my biggest critic. but when you turn in what you think is a good product and get the wide range of scores without getting feedback from the judges how do you know what to change?

I do not care about feedback from this table. They can't agree among themselves what to score. This the table that needs help on cosistency. And my experience has been that the ones that need the most work are just as likely to be high scoring judges as low scorin judges. Keith

I believe you guys are saying essentially the same thing. How do you know that a 7 is just average or it's repulsive to the judge? You don't.

jbrink01
07-24-2012, 08:52 PM
I was one of those that got a comment card from Outnumbered this week and it was helpful. I also think there should be some sort of standardized judging handbook issued. We as cooks spend way to much money for KCBS not to standardize on something.

YankeeBBQ
07-24-2012, 09:07 PM
In theory the BOD has taken some action to improve judging. The new scoring system will allow judges to log in and see how their scores compare to the rest of the table. Hopefully this will help with the consistency issue.

I have heard the argument that if comment cards are mandatory for scores of 6 or below then judges will just score a 7. Is this true? I really have no clue. In my experience the biggest obstacle to comment cards are the reps that don't want to be bothered with dealing with them so they don't encourage them. Currently there is no easy way to continue comment cards when the new scoring system comes out and maintain the blind system (or so I'm told).

The reason I started this comment with "In theory" is because I have not been able to get a look at the new scoring program. I can only tell you what I've been told and not what I've viewed with my own two eyes.

I would like to add one small caveat to this conversation. When I started competing in KCBS competitions there was no such thing as a comment card. I was still able to improve my cooking over time and win many grand championships. Is it really the job of the judges to educate you on how to become a better cook ? That's a serious question not just a throw away line. I think cooks today have a lot more shortcuts available to them then we did back in the day. There are forums, tutorials and classes galore. I don't think that's a bad thing but I think people expect results allot faster now. They expect to go out after three competitions and win a grand. It's been done but I don't think it's the norm. Patience grasshopper.




It's interesting that the conversation has taken this turn, again. The original point is that there is no consistency in judging, not that people are getting scored too low. In the situation I outlined in the original post, it is likely that people are getting scored too high more often than they are too low.

If you are in this to cook good BBQ, get better, get to a point where you can learn from competitions and do consistently better, THERE NEEDS TO BE A WAY TO ESTABLISH A CONSISTENT BASE OF SCORING.

If one judge uses a 6 as a base and the rest use an 8, how do we take away from that if there is no other feedback? KCBS BOD? Comments?

Tack
07-24-2012, 10:00 PM
when is this new scoring system going to debut?

Outnumbered
07-24-2012, 10:16 PM
I have heard the argument that if comment cards are mandatory for scores of 6 or below then judges will just score a 7. Is this true?

I would suggest just making them mandatory, or, as someone suggested, having someone (TC, Rep, etc.) randomly call out a judge each round to defend why they scored the way they did...good or bad.

Is it really the job of the judges to educate you on how to become a better cook ? That's a serious question not just a throw away line. I think cooks today have a lot more shortcuts available to them then we did back in the day. There are forums, tutorials and classes galore. I don't think that's a bad thing but I think people expect results allot faster now. They expect to go out after three competitions and win a grand. It's been done but I don't think it's the norm. Patience grasshopper.

All due respect, I think that is a throw away line. Maybe the question isn't, but the comments following make it so, particularly the comments about how you to had to walk uphill on your way to school back in the good ol' days when you were a kid. Yes, we have it so much easier today and we're softer for it.

To answer your question, however, no, it is not the job of the judges to make me a better cook. It is the job of the judges to judge the product put in front of them and judge it consistently. The very fact that you mentioned the BOD is addressing this somewhat is a clear indication that I'm not alone in this. Giving everyone at least a 7 is not the answer. That's the same as a participant ribbon in Little League. If 6 is average, encourage judges to use 6 as an average. Don't justify being nice because we spend so much money. Just judge it fairly and on the scale the BOD has instituted.

Asking the judging to use a consistent basis for what constitutes what score in their mind so the cook can take something away from the judging is not too much to ask. See what Speers90 suggested. Maybe that's a good start.

I also don't believe I or anybody else on here has suggested that the judging is the reason we are not winning comps. And I'm certainly not suggesting that I should have won any comp, category or even scored better than I have.

I just want to have some sort of rationale behind the things that have been outlined here.

Bunny
07-24-2012, 10:33 PM
In theory the BOD has taken some action to improve judging. The new scoring system will allow judges to log in and see how their scores compare to the rest of the table. Hopefully this will help with the consistency issue.

I have heard the argument that if comment cards are mandatory for scores of 6 or below then judges will just score a 7. Is this true? I really have no clue. In my experience the biggest obstacle to comment cards are the reps that don't want to be bothered with dealing with them so they don't encourage them. Currently there is no easy way to continue comment cards when the new scoring system comes out and maintain the blind system (or so I'm told).

The reason I started this comment with "In theory" is because I have not been able to get a look at the new scoring program. I can only tell you what I've been told and not what I've viewed with my own two eyes.

I would like to add one small caveat to this conversation. When I started competing in KCBS competitions there was no such thing as a comment card. I was still able to improve my cooking over time and win many grand championships. Is it really the job of the judges to educate you on how to become a better cook ? That's a serious question not just a throw away line. I think cooks today have a lot more shortcuts available to them then we did back in the day. There are forums, tutorials and classes galore. I don't think that's a bad thing but I think people expect results allot faster now. They expect to go out after three competitions and win a grand. It's been done but I don't think it's the norm. Patience grasshopper.

Very well said, Steve!

Red Valley BBQ
07-24-2012, 10:38 PM
When you get 9s from 5 judges on appearance and then one 7, I want to know what that one judge is looking at...not in a bad way but just to have the insight. Maybe he likes the burnt ends in the front compared to the back, who knows.

A "7" is considered "above average" on the KCBS scale, why should a judge be required to fill out a comment card when he gave an above average score? As far as liking the burnt ends in the front of the box, obviously the other five judges liked them in the rear of the box. Why should one persons subjective opinion have an affect on how you cook or present your BBQ when its obvious that five people like it as it was presented?

Here are my rib scores from the last contest we did, please tell me exactly what we did wrong last contest, and exactly how we can improve our product without anything but these numbers.[/SIZE]

9/8/8 9/8/9 9/9/8 9/8/8 7/7/7 7/8/9

One thing I have learned is to not take the scores from one contest to heart. I analyze my scores from three or four contests to get the bigger picture. Over those contests if I am averaging 7's or 8's in any certain criteria, then I know it is something I have to look at and make some adjustments. Only looking at one contests results may cause you to change some things that may not need to be done especially since you may have just hit a bad table in that one comp.

Obviously it would benefit the cooks and the KCBS as a whole if all the judges were on the same page. By instructioning the judges as to a starting point in the judging process would be a good first step to get everyone on the same page. Having one judge start his judging at 9 and deducting from that for things he deems wrong and having another judge start at "average" (6) is not having everyone on the same page. The KCBS should pick a starting point and instruct the judges to do so accordingly. Its not a cure all for the judging problems, but its a step in the right direction towards consistency.

ModelMaker
07-25-2012, 09:26 AM
There is no "starting point" in KCBS scoring. Your box gets a fresh look on a blank page. It gets judged on it's own merits and a score is given according to that judges decision on appearence,taste, and tenderness.
The problem, if one exists, is my 8 is someone elses 7 or 9. I think a bigger problem is over scoring. I have over the years settled in to a pattern of a 9 is gimme that now, and a 6 is average. I have no problem giving a 7 telling the cook that his product is
"above average".
I am truley suprised at the number of 9,9,9 I encounter in the tent.
There ain't that much excellent per table per contest.
Those of you getting 7's and complaining about them should just as easily complain about the exsessive amount of undeserving 9's per contest.
Ed

TTNuge
07-25-2012, 10:23 AM
There is no "starting point" in KCBS scoring. Your box gets a fresh look on a blank page. It gets judged on it's own merits and a score is given according to that judges decision on appearence,taste, and tenderness.


That may be the way YOU were instructed and the way you judge but that's the problem, it isn't that way consistently. Some are taught to start from a 6 and judge up or down accordingly, others are taught to start at a 9 and judge down if needed. Some are pressured into thinking that the teams put in so much time and effort that you should be kind to them and not give them horrible scores. Others feel pressure to fill out comment cards for anything 6 or below so the lowest they'll go is a 7.

Consistency. That is the key but it is also the hardest thing to get. We're asking volunteers, A LOT of volunteers, to be our judges and hold our fate in their hands. I would imagine just like the teams competing, there are plenty who take it much more serious than others.

I do think the KCBS is on the right path with the continued education and more emphasis on the judging but I also know it isn't going to be fixed overnight.

ModelMaker
07-25-2012, 10:58 AM
That may be the way YOU were instructed and the way you judge but that's the problem, it isn't that way consistently.

I took my class in '03 with Mike Lake and at that time he stressed there is no starting point but did mention "in the past" the starting point was taught as a 6. So in the last decade at least all entries had no "starting point"
Looking around the judges tent in my area, when seating judges by number of contests judged I would say the majority of judges do not even know of the "start at 6" teachings.
I think the "start at 9" is from another BBQ santioning assoc. (MIM?)
Ed

"Holy Smokes"
07-25-2012, 11:16 AM
What is interesting about this discussion is if we exchange BBQ scoring with military performance reports, it is the same issue. This scoring process is the exact same, imho. The inherent problems surface in both, inflated scores in order not to justify. We require explanations for mark downs, always...so people inflate. Also, no one wants to hurt their subordinates chances for promotion. Only way to fix any of this is to have everyone on the same page and that is impossible.

paydabill
07-25-2012, 11:54 AM
This is a great post - I am glad you had the gumption to start the conversation.

When I judge - I do not tell people I am a cook - because it changes the way people talk around you.

The only time that the table knew I was a cook is when KCBobby was sittign across from me and we started to talk about events we were doing. The rest of table shut up and just stared at us.

mobow
07-25-2012, 12:31 PM
This is a great post - I am glad you had the gumption to start the conversation.

When I judge - I do not tell people I am a cook - because it changes the way people talk around you.

The only time that the table knew I was a cook is when KCBobby was sittign across from me and we started to talk about events we were doing. The rest of table shut up and just stared at us.
My limited experience is that they couldn't get a word in once you start talking. lol keith

Outnumbered
07-25-2012, 01:25 PM
This is a great post - I am glad you had the gumption to start the conversation.

Does gumption = stupidity? :icon_blush:

Crash
07-26-2012, 04:46 AM
In theory the BOD has taken some action to improve judging. The new scoring system will allow judges to log in and see how their scores compare to the rest of the table. Hopefully this will help with the consistency issue.

I have heard the argument that if comment cards are mandatory for scores of 6 or below then judges will just score a 7. Is this true? I really have no clue. In my experience the biggest obstacle to comment cards are the reps that don't want to be bothered with dealing with them so they don't encourage them. Currently there is no easy way to continue comment cards when the new scoring system comes out and maintain the blind system (or so I'm told).

The reason I started this comment with "In theory" is because I have not been able to get a look at the new scoring program. I can only tell you what I've been told and not what I've viewed with my own two eyes.

I would like to add one small caveat to this conversation. When I started competing in KCBS competitions there was no such thing as a comment card. I was still able to improve my cooking over time and win many grand championships. Is it really the job of the judges to educate you on how to become a better cook ? That's a serious question not just a throw away line. I think cooks today have a lot more shortcuts available to them then we did back in the day. There are forums, tutorials and classes galore. I don't think that's a bad thing but I think people expect results allot faster now. They expect to go out after three competitions and win a grand. It's been done but I don't think it's the norm. Patience grasshopper.

100% agreed. Thanks for the insightful comments.