PDA

View Full Version : Judging (moved from KCBS NAshville thread)


SaucyWench
11-29-2008, 01:48 PM
Curious, do any of you cookers plan on going to the CBJ/Cooks Roundtable on Friday? And would you really want to hear judges' opinions?

If any of yall would like to know the judge's side, we would go, but if it's just a cook vs judge "what do you know cause you don't cook competitively" free-for-all, we'll stay home.

drbbq
12-14-2008, 11:16 AM
Curious, do any of you cookers plan on going to the CBJ/Cooks Roundtable on Friday? And would you really want to hear judges' opinions?

If any of yall would like to know the judge's side, we would go, but if it's just a cook vs judge "what do you know cause you don't cook competitively" free-for-all, we'll stay home.

That's a great attitude. I'm sure the cooks will be very receptive. This should be a good one. WTF will moderate this!

I think I'll attend as a judge.

Master Judge #738

Rolling Smoke
12-14-2008, 11:31 AM
Curious, do any of you cookers plan on going to the CBJ/Cooks Roundtable on Friday? And would you really want to hear judges' opinions?

If any of yall would like to know the judge's side, we would go, but if it's just a cook vs judge "what do you know cause you don't cook competitively" free-for-all, we'll stay home.

I have to agree with Ray regarding the attitude. I for one would welcome hearing the judges opinions and I don't think I've ever been guilty of having a "what do you know cause you don't cook competitively" attitude either.

I think a good topic for discussion would be what ideas the judges have to make scoring more consistant.

Jorge
12-14-2008, 11:50 AM
That's a great attitude. I'm sure the cooks will be very receptive. This should be a good one. WTF will moderate this!

I think I'll attend as a judge.

Master Judge #738

I might have to attend......if you moderate that discussion:eek::mrgreen:

Ray for President!

drbbq
12-14-2008, 03:42 PM
I'll moderate but we may need security.

Jeff_in_KC
12-14-2008, 08:40 PM
Oh man, all of the sudden it's looking more interesting. But I'm not going to play in Troy Black's backyard party.

Plowboy
12-14-2008, 08:52 PM
Oh man, all of the sudden it's looking more interesting. But I'm not going to play in Troy Black's backyard party.

That'll show him! :roll:

Strangely, I think Audrey wants to go. She keeps bringing it up.

SaucyWench
12-15-2008, 08:46 AM
That's a great attitude. I'm sure the cooks will be very receptive. This should be a good one. WTF will moderate this!

I think I'll attend as a judge.

Master Judge #738


I don't mean to have a bad attitude, but there have been times when we've been asked for advice on what judges look for, told what we've learned over the years, and then basically been told that the judges must be the problem, not their methods, recipes, whatever.

Most cooks are very receptive, but, sadly, some definitely see us as adversaries, or think we're only there for free food. When you are serious about your judging, it's discouraging to hear things like that. Just makes me a little leery. Sorry if I've offended anyone.

Looks like we wouldn't be able to go anyway, work deadlines interfere.

Podge
12-15-2008, 09:41 AM
I don't mean to have a bad attitude, but there have been times when we've been asked for advice on what judges look for, told what we've learned over the years, and then basically been told that the judges must be the problem, not their methods, recipes, whatever.

Most cooks are very receptive, but, sadly, some definitely see us as adversaries, or think we're only there for free food. When you are serious about your judging, it's discouraging to hear things like that. Just makes me a little leery. Sorry if I've offended anyone.

Looks like we wouldn't be able to go anyway, work deadlines interfere.

I for one, value judges' opinions, but I don't get to get them. I can understand the opinions by assumptions of mine, of the 5 judges that give me, 7's, 8's and 9's, but I'd really like to know the opinion of the 1 judge that'd give me 4's and 5's at the same table. That is the judge we all have problems with !!!!

Merl
12-15-2008, 10:09 AM
I for one, value judges' opinions, but I don't get to get them. I can understand the opinions by assumptions of mine, of the 5 judges that give me, 7's, 8's and 9's, but I'd really like to know the opinion of the 1 judge that'd give me 4's and 5's at the same table. That is the judge we all have problems with !!!!

Great observations. There are two issues:
1st is feedback. I worked hard to bring about the comment card program. Unfortunately it was ended. I do not believe those who were against it, truly understood the value of the project. I would love to bring it back.

And yes sometime there were comments which made little sense. And yes sometimes the comments were off base. But the majority helped with useful information.

2nd. There judge who is inconsistent is to be talked to by the Contest Rep. He/she should be questioned if they understand the scoring system. The should be shown the cards on the table to show how they are outside of the table. If this does not work, the Rep does have the authority to remove the judge. (This is not going to happen often) But should be writing a letter the the CBJ chairman (currently Ed Roith) and make a report.

We must begin a program for continuing cbj education. The rules change, and cooking changes and we owe it to the judges and the cooks to provide on going education.

Just a few thoughts.
Yours in Que
Merl Whitebook

Podge
12-15-2008, 12:10 PM
Great observations. There are two issues:
1st is feedback. I worked hard to bring about the comment card program. Unfortunately it was ended. I do not believe those who were against it, truly understood the value of the project. I would love to bring it back.

And yes sometime there were comments which made little sense. And yes sometimes the comments were off base. But the majority helped with useful information.

2nd. There judge who is inconsistent is to be talked to by the Contest Rep. He/she should be questioned if they understand the scoring system. The should be shown the cards on the table to show how they are outside of the table. If this does not work, the Rep does have the authority to remove the judge. (This is not going to happen often) But should be writing a letter the the CBJ chairman (currently Ed Roith) and make a report.

We must begin a program for continuing cbj education. The rules change, and cooking changes and we owe it to the judges and the cooks to provide on going education.

Just a few thoughts.
Yours in Que
Merl Whitebook

I'd like to see judges be able to talk to each other while they're judging. Someone's entry will win the table anyway, might as well talk about it. That way, when judges are in line, and the one judge that isn't can be convinced otherwise, or he/she can at least give their opinion to the other judges why they think what they think. I really don't think it's a bad thing for judges to talk to each other.

KC_Bobby
12-15-2008, 03:17 PM
Podge, are you suggesting they talk about it while they are judging (as in between viewing/tasting) or after each catagory? I know the handful of times I've judged the majority of the judges at each table seem to compare thoughts on the entries - what they liked and/or disliked about each (or favorite and least favorite). I enjoy sharing feedback after each catagory is judged and cards are handed in.

Podge
12-15-2008, 03:49 PM
Podge, are you suggesting they talk about it while they are judging (as in between viewing/tasting) or after each catagory? I know the handful of times I've judged the majority of the judges at each table seem to compare thoughts on the entries - what they liked and/or disliked about each (or favorite and least favorite). I enjoy sharing feedback after each catagory is judged and cards are handed in.

yes, that's what I'm getting at.. talk about it right then before scores get turned in. I bet scores would be a bit more consistent then, or judges will start arguing, or....who knows... but it'd be interesting to say the least !!!.. I'd go to a contest that'd try that ou !

SaucyWench
12-15-2008, 03:54 PM
Oh, it would be interesting, all right!

How long are you willing to wait for the awards?

Rolling Smoke
12-15-2008, 04:29 PM
If waiting a little while longer would produce more consistant scores, I would gladly wait.

Podge
12-15-2008, 05:31 PM
Oh, it would be interesting, all right!

How long are you willing to wait for the awards?

i would hope that within that 30 minutes between catagories that those 6 judges could have discussed and scored 6 entries in that time. But, since i haven't judged, it's easy for me to make that assumption.

If it did take more time to compile, but if it meant more consistent judging, i'll sit with Rolling Smoke for a couple more hours, or how long it takes and drink more beer before awards.

SaucyWench
12-15-2008, 05:43 PM
I'm trying to figure out how this could work. Would we sample #1, talk about it, come to a consensus, then go to #2 and so on? Would we have a time limit per sample to come to an agreement? I'd think at the very least, there would need to be longer times between turn-ins. Since after turn-in, the reps must change the numbers, then they have to try to make sure no table is judging all of the same team's entries, the last table may not get their boxes until 5 or more minutes after the turn-in deadline.

How do you convince the one who thinks #5 is too spicy, salty, peppery, whatever, that their palate is wrong? What if the "holdout" won't change their mind? (I see the noise level in the judging area becoming deafening, especially at larger contests.)

Since KCBS rules call for discarding the lowest score, would this really be worth it?

Podge, you and all cooks really should take the judging class & judge a few contests, some cooks we've talked to who have, get a totally new perspective.

Podge
12-15-2008, 06:15 PM
Podge, you and all cooks really should take the judging class & judge a few contests, some cooks we've talked to who have, get a totally new perspective.

I agree with you. I'm sure if I'd take the class and judge about 10 or so contests, I would definitely have a new perspective. But right now, all I'm interested is competing. If i were to start judging in the middle of still competing, I really don't want that kind of new influcences and perspective on me yet. I'm afraid that it could potentially negatively (but think it's a positive at the time) affect how I tackle flavor profiles, tenderness, and presentation. I think what I have now is good stuff, and just want to evolve what I have now to become better and better. One day when I do quit competing, I think I'll start judging. Then, when I do start judging, I'll probably wished I did it earlier. :icon_smil

In hindsight, if i was smart when i very first started BBQ competitions, I would've taken a judging class, and judged a few to at least get a better clue on what I was doing. Actually, That is not a bad idea for a team who's only cooked one or so contests.

Rolling Smoke
12-15-2008, 06:15 PM
Since KCBS rules call for discarding the lowest score, would this really be worth it?

That comment when coming from a rep is one of the biggest insults to a cook when they are questioning a broad deviation of scores.

Podge, you and all cooks really should take the judging class & judge a few contests, some cooks we've talked to who have, get a totally new perspective.

I'm sorry but that almost sounds like a "what do you know cause you don't judge competitions" attitude. Not trying to start a fight here but when I read your first post regarding the attitudes that a lot of cooks have against judges, I gave you credit for it even though I did not consider myself as part of the percentage that didn't want to hear feedback. Now, I'm starting to wonder whether it's the cooks or the judges that really have the attitudes. Speaking for myself, I have been a CBJ since 2003 and have also judged. Just guessing here but I'd say there are a lot more cooks that are CBJ's than there are CBJ's who compete. Having been at the tables before, I think Podge's idea has some merit. Dropping the lowest score is not the answer.

drbbq
12-15-2008, 06:34 PM
When I started judging I definitely got a new perspective.

Spydermike72
12-15-2008, 06:37 PM
Great observations. There are two issues:
1st is feedback. I worked hard to bring about the comment card program. Unfortunately it was ended. I do not believe those who were against it, truly understood the value of the project. I would love to bring it back.

And yes sometime there were comments which made little sense. And yes sometimes the comments were off base. But the majority helped with useful information.

2nd. There judge who is inconsistent is to be talked to by the Contest Rep. He/she should be questioned if they understand the scoring system. The should be shown the cards on the table to show how they are outside of the table. If this does not work, the Rep does have the authority to remove the judge. (This is not going to happen often) But should be writing a letter the the CBJ chairman (currently Ed Roith) and make a report.

We must begin a program for continuing cbj education. The rules change, and cooking changes and we owe it to the judges and the cooks to provide on going education.

Just a few thoughts.
Yours in Que
Merl Whitebook

Merl and Podge,

Please correct me if I am wrong, but in referrence to your 2nd point - Isnt that the job of the Table Captain ? If a score is tottally out of whack shouldnt the Table Captain be the first one to catch it and approach the judge about the variation in the score ? Then the Table Captain should speak to the KCBS Rep, that is the correct process right ??

I may be way out here, so any help is appreciated.

SaucyWench
12-15-2008, 06:52 PM
RS, I'm sure you're right about more cooks who've judged, than judges who've competed, that's one reason KCBS has the rule about working with a team before getting master judge, so we can see it from the other side too. I learned a lot when I did, and I entered a sauce that did really bad, and beans that came close, but no prize. That's not the same as what you do, but did give me a small clue on the pressure of competing.

My point about the low score is that since it appears from the comments here, and from my experience, that at a table of 6 judges, there often tends to be one offball score, if it's low and discarded, why change the whole system? If the offball score is high, all to the good. I don't understand why 6 different judges are expected to score exactly the same. (btw, I'm not a rep.)

This is so frustrating, and I imagine that if we were face to face, we could understand each other better.

Rolling Smoke
12-15-2008, 07:06 PM
RS, I'm sure you're right about more cooks who've judged, than judges who've competed, that's one reason KCBS has the rule about working with a team before getting master judge, so we can see it from the other side too. I learned a lot when I did, and I entered a sauce that did really bad, and beans that came close, but no prize. That's not the same as what you do, but did give me a small clue on the pressure of competing.

My point about the low score is that since it appears from the comments here, and from my experience, that at a table of 6 judges, there often tends to be one offball score, if it's low and discarded, why change the whole system? If the offball score is high, all to the good. I don't understand why 6 different judges are expected to score exactly the same. (btw, I'm not a rep.)

This is so frustrating, and I imagine that if we were face to face, we could understand each other better.

I agree with the KCBS rule where a CBJ has to cook with a team before they can become a Master Judge but that is a one time requirement, if I'm not mistaken. In order for a team to stay ahead of the curve, they really should judge more than one competition a year and the same should be said for judges.

I know you're not a rep and that was not what I was saying when I made that comment. All I was saying is this tends to be a pacifier used by some reps to appease teams when questioning broad deviations. I have seen low scores in one category offset the overall ranking by as much as 4 or 5 places before which could keep a team out of the money.

It can be frustrating and face to face would definitely bridge some communication gaps. Maybe if you come and judge Kettering next year, we can continue this discussion face to face. And if it makes things any less frustrating, I won't be competing.

Jorge
12-15-2008, 07:34 PM
For the sake of discussion....if table talk is allowed do you feel that the meat, as presented might not necesarily be judged on it's own merit? While it could even out the oddball judge, it could also allow a persuasive speaker to sway judges to lower their scores for a worthy entry.

In my opinion it could just as easily create another bad table. If entries aren't discussed at ANY table that part of the equation is fair and balanced. If they are discussed it's a crapshoot as to what happens when your box hits the table with the successful trial lawyer.

Jeff_in_KC
12-15-2008, 09:15 PM
Seems when you watch other food competitions on Food Network, their judges are discussing the merits of the samples they're judging. Why would it work any differently for KCBS? Like Podge said, I'm willing to wait if it makes things more consistent. If judging is not about personal preferences and if judges REALLY know BBQ then there's never an excuse for a 4 and 5 from a judge or two when everyone else gives the same sample 8's and 9's. That tells me that either a judge is judging on their own preferences and not whether or not the sample is a good representation of the kind of BBQ the cook was trying to produce OR the judge doesn't know a good example of spicy BBQ, sweet BBQ, Carolina style BBQ or whatever. A little discussion might put the heat on someone to get it right and not score down because they don't personally like spicy BBQ.

SaucyWench
12-15-2008, 09:48 PM
Jeff, are you saying that every piece of meat you in your turn-in box is exactly the same?

I was a table captain at a competition this past summer, and noticed one judging slip with a really low score. (I don't remember what category it was, I've slept since then.) I asked the judge about it, because all the other judges scored that particular entry much higher. The judge stated that the piece of meat was a wad of gristle and fat, and had to be spit out. I made sure that the rep knew that I saw the low score and asked about it. The rep acknowledged it, and that was it.

Personally, I would have tried another bite, to make sure I hadn't just picked the wrong bite, but if a judge is presented a gristly, fatty piece of meat, should that judge be judged as a bad judge?

Rolling Smoke
12-16-2008, 05:16 AM
For the sake of discussion....if table talk is allowed do you feel that the meat, as presented might not necesarily be judged on it's own merit? While it could even out the oddball judge, it could also allow a persuasive speaker to sway judges to lower their scores for a worthy entry.

In my opinion it could just as easily create another bad table. If entries aren't discussed at ANY table that part of the equation is fair and balanced. If they are discussed it's a crapshoot as to what happens when your box hits the table with the successful trial lawyer.

While I have given that possibility some consideration Jorge, I don't think it is any more likely to happen than it is for KCBS to reprogram the judges by telling them what tastes good and what doesn't.

Here's a perfect example of how table talk would help eliminate oddball scores. Last year we put on a CBJ class with Mike and Theresa. The perspective judges were all given an ungarnished box with 6 of the most beautiful chicken thighs I have ever seen. After they scored on appearance, Mike began asking "Who gave this a 9?....Who gave it an 8?....Who gave it a 7?" etc. Nobody in the class of 59 people gave it below a 7 except one person who gave it a 3. Mike asked why he gave it such a low appearance score and the guy said it was because it didn't have any garnish in the box.

Now, imagine this same scenario playing out at a competition where there is no table talk allowed, very little supervision by the table captain and a rep who knows he can easily blow off a disgruntled cook by telling him that the lowest score gets dropped. That's pretty much what we have right now. Now try to imagine the same scenario where the scores are discussed and it is pointed out to the errant judge that lack of garnish is not a reason to score down. The can either change the score or leave it as is but if they continue to play by their own set of rules, they will eventually get weeded out.

I see the table talk being more effective with appearance since each piece of meat might vary in taste and tenderness.

As a cook, I expect fairness in the scores I receive and it pisses me just as much to receive a 9 from one judge when my entry only deserves a 7 at best.

As a CBJ, I try to stay on top of the rules and I try to judge each entry based on its own merit as I was instructed to do. I also TRY not to score based on comparison.

As an organizer, I can tell you up front that there are some judges that will not be judging in Kettering this year because their scores were so far out of line with the rest of the pack.

Rolling Smoke
12-16-2008, 05:32 AM
Merl and Podge,

Please correct me if I am wrong, but in referrence to your 2nd point - Isnt that the job of the Table Captain ? If a score is tottally out of whack shouldnt the Table Captain be the first one to catch it and approach the judge about the variation in the score ? Then the Table Captain should speak to the KCBS Rep, that is the correct process right ??

I may be way out here, so any help is appreciated.

I think you're right Mike but the problem as I see it is, most of the time the person acting as TC is one of the six judges that has been appointed and since they have not taken the TC class, all they know to do is present the boxes and make sure the table has the necessary supplies they need. The first time I ever judged, I was appointed TC and I had no more idea what I was doing than the man in the moon. I cooked a contest last Sep where one of the TC's was a veteran judge with 3 prior contests under her belt and her method of scoring was based on how the entry compared to the stuff her Dad cooked at home.

CaptTable
12-16-2008, 06:34 AM
Good morning, folks.

First, let me correct an item that be misunderstood.

A Table Captain can bring possible out-of-line scores to the attention of the Rep, but they should NEVER take it upon themselves to approach the judge. That is purely the Reps job to talk to judges about their scoring.

It really wouldn't be fair to put that type responsibility onto a Table Captain.

With all due respects to Podge, judges discussing entries then arriving scores would never work in my opinion. I really don't belive the scores could be arrived at within the 30 minutes. And, I could see some judges wanting an entry to have a 9, yet drop it down to an 8 just to speed up the process. That could mean a Grand Championship for someone.

In my opinion, the best tool for a cook would be comment cards. The ones used in the test program just needed modifying to include the score being commented on and only be given for scores of say 5 or lower. That would be anything below average.

Right now, when I see a low score, I approach the judge and ask for an explaination so I will have an answer for the team when they see the score. The trouble is I have had contests where there were several incidents like this and trying to keep them straight is rough for an old man. And I don't want to pull out a long written list in front of a team.

Sorry for the long post,

Phillip

CajunSmoker
12-16-2008, 06:50 AM
I'm not sure of the answer to the deviation in scores, but making a longer time between turn ins is definetly not the answer IMHO. I cooked an IBCA contest for my last contest this year and the 1 hour between turn ins was waaaaay to long:roll:

Podge
12-16-2008, 07:25 AM
With all due respects to Podge, judges discussing entries then arriving scores would never work in my opinion. I really don't belive the scores could be arrived at within the 30 minutes. And, I could see some judges wanting an entry to have a 9, yet drop it down to an 8 just to speed up the process. That could mean a Grand Championship for someone.

I just threw it out there. Tought it would be interesting to try in a contest. Just trying to come up with a way to help with more consistent judging. I also had the idea of droping the highest score, along with the lowest score, and just keep the 4 middle ones. i don't know if that would help either.

CaptTable
12-16-2008, 07:42 AM
Podge,

I think dropping the high score would not be a bad idea. There may be some judges out there who are giving out 9's just to keep anyone from fussing at them.

Phillip

Rolling Smoke
12-16-2008, 07:43 AM
A Table Captain can bring possible out-of-line scores to the attention of the Rep, but they should NEVER take it upon themselves to approach the judge. That is purely the Reps job to talk to judges about their scoring.


Phillip, I didn't mean to imply that the TC should approach the judge. From my understanding, in addition to making sure the table has all of the supplies they need, the TC should also bring any out of line scores to the reps attention. Based on what I have seen on other contests, a lot of TCs don't know that this is one of their responsibilities.


In my opinion, the best tool for a cook would be comment cards. The ones used in the test program just needed modifying to include the score being commented on and only be given for scores of say 5 or lower. That would be anything below average.


I agree that the comment cards would be helpful to the cook. How would the comment cards help eliminate the broad range of scores and improve consistency in the scoring? I know it's possible to have one piece of meat turn out a little tougher than another so it's likely to have broader scores in the tenderness or even taste category. I've cooked enough comps to know that one piece of brisket or one rib might be more flavorful or more tender than another and I have learned to accept an oddball score in either of these two categories but when I see all 9's and 8's and one four for appearance, it doesn't make sense. This is where the comment cards would serve no purpose other than to tell me that one judge was FOS and didn't know what they were doing. It still makes more sense to me to police the oddball scores and correct them before they get keyed in.


Right now, when I see a low score, I approach the judge and ask for an explaination so I will have an answer for the team when they see the score. The trouble is I have had contests where there were several incidents like this and trying to keep them straight is rough for an old man. And I don't want to pull out a long written list in front of a team.


I know how you and Mrs. Table officiate and I appreciate the job you do. Unfortunately, not all contests are governed in the same manner anymore than all judges score by the same standards.

CaptTable
12-16-2008, 08:22 AM
Jim,

I guess I'm thinking that by having the comment cards and being able to read what some judges are thinking, I would be able to catch the ones who are off base in their preception of judging. May be kinda far fetched, but thought it could help the reps too!

Phillip

Rolling Smoke
12-16-2008, 09:45 AM
I don't think it's far fetched at all Phillip. If the feedback is being used to help determine who the rogue judges are and establish corrective action in the scoring process, then it's a great idea in my opinion. But, if the only purpose of using the comment cards is to tell the cook why one judge scored lower than the rest, it's not doing much more than offering a band-aid for a slit throat.

Maybe we should drop the lowest and the highest scores and take the four in the middle for the average?????

Scottie
12-16-2008, 09:51 AM
Nobody in the class of 59 people gave it below a 7 except one person who gave it a 3. Mike asked why he gave it such a low appearance score and the guy said it was because it didn't have any garnish in the box.




You know... I think I found that one judge on all 4 of my categories last year.... Yeah, that's the ticket.... :icon_smil:shock::eek::roll:


You know I am kidding...

Scottie
Gravel Throwing BBQ.

Rolling Smoke
12-16-2008, 10:20 AM
Maybe another one slipped through Scottie because the one I'm talking about did not judge Kettering......OMG! That means there's more than one of them!!! :shock:

Scottie
12-16-2008, 10:54 AM
I think I have a stalker then. Cause I sure was finding them all summer!!! :icon_shy

LindaM
12-16-2008, 11:03 AM
Maybe along with the comment cards which I highly support, tracking of the judges would work, our software is designed to do it, unfortunately it will require longer keying times, since we need to add that judges number in for each card. Requiring judges to go to refreshers is yet another idea especially those who only judge 1 or 2 contests a year.

Rolling Smoke
12-16-2008, 11:35 AM
All great ideas Linda but I don't know if retraining would be effective with the judges that are playing their own game and scoring by their own standards. The incident I cited in an earlier post that took place in 2006 was when I first heard that the software was in place and KCBS was going to start tracking judges but apparently it never happened. I think tracking the judges and taking action against the ones that do not score by KCBS standards would be the biggest step that could be taken to help minimize the inconsistency in scoring

CaptTable
12-16-2008, 12:19 PM
Jim,

Must be something in the air today, but I can't agree with the judge tracking either.

If a judge were to find out (and they would) that they are being tracked, I believe they would just put down middle-of-the-road scores so as to not bring attention to themselves. They may feel that as long as they are giving 7's and 8's and an occassional 9 or 6, they would fly under the radar. And they would.

Phillip

Rolling Smoke
12-16-2008, 12:58 PM
It's really starting to sound like we can't get there from here. I think the majority of the judges would continue to score as they always have regardless of whether they knew they were being tracked or not. If some judges started giving middle of the road scores to stay off the radar, wouldn't that in itself say that there's a problem with consistency?

I don't know why a judge would have a problem with being tracked unless they're intentionally trying to cause dissention. Everything a cook does at a comp is under the microscope and subject to scrutiny so why should it be any different with the judges?

And (while I'm on a roll) what the hell happened with Phillip Fulmer? I've heard all kinds of things since I watched him walk off the field and I figure if anyone knows the truth, you do.

Scottie
12-16-2008, 01:00 PM
Must be something in the air.. As the Lady Vols are out of the top 10 for the first time in 12 years I believe it was...

As to the point... I can see valid issues with both sides for tracking judges...

SaucyWench
12-16-2008, 06:48 PM
Why would tracking judges need to be done without their knowledge? If the "rogue" judge knows he/she is being tracked, wouldn't that be incentive to play by the rules?

I have no problem being tracked, after all, my name and CBJ number are on every judging slip I fill out, but I would take issue with it being done clandestinely.

Rolling Smoke
12-17-2008, 05:27 AM
I agree SW, I never considered the tracking of judges to be done without their knowledge. When I was told about the plan to track judges two years ago, it was just another part of collecting data from a contest. It was never presented to me as a 'black op' designed to undermine the judges but rather a tool to identify the problem(s) with inconsistant scoring and correct them.

As a competitor, I would have just as much of a problem with a judge who didn't want their scores tracked as I would with a team that didn't want their meat inspected. If the tracking process is implemented, I think the majority of the judges will accept it as a tool to improve the overall scoring system.

Dale P
12-17-2008, 09:21 AM
Just the thought of being tracked could be in the mind of a rogue Judge. Everyone knows that there is a butthead somewhere in the crowd no matter what you are doing, and it is a shame that it only takes one to ruin somebodys weekend.

I dont have the solution for the problem but I cant see where tracking low scores would hurt anything.

Rolling Smoke
12-17-2008, 10:14 AM
Dale, I'm not saying track just the low judges. I'm saying track ALL judges. All of the scores entered by the judges is data that we are not using. The only was we know if the CBJ classes are really getting the message across to the judges is to track the scores that are being put on paper. Look at what happened a few years ago when they changed the scoring process...you hardly ever see a perfect 180 anymore and when you do, it's usually dessert. That down curve in the overall points was not a result of the judging classes as much as it was a result of the rules change. We know we have rogue judges and it's not just the ones that are lowballing everything that hits their table. The ones that give all 9's to something that is garbage are just as guilty. The problem as I see it is a broad deviation or inconsistency in scores. The only way I see to eliminate the problem is to go to the source (i.e. the judges) and correct it by retraining or removing them from the equation. Our reps at Kettering do a great job of catching rogue scores and talking to the people who are giving them but unfortunately this is not being done on a global basis and the reps I speak of cannot officiate at all the contests that we cook.

SaucyWench
12-17-2008, 08:34 PM
I'm totally with you on the high scoring judges being just as much an issue as the low scoring ones. I know of a couple judges who score high ( 8s or 9s) almost all the time, and if they ever give a 6, it means they pretty much hated it. One thing that might help the scoring is if we defined what "average" is. Is it the kind of Q you would get at an average restaurant? What we can cook ourselves? Grandpa's BBQ? I know that DH has cooked some above-average backyard, sometimes better than I've gotten in competition, but where is the bar set?

As far as tracking goes, if it were to be implemented, it should be all judges, with all of us knowing it will happen. If we're all in this to promote the sport of BBQ, none of the components of a competition should feel like we're the opponent. Just as any competitor who doesn't like the rules, shouldn't compete in KCBS comps, any judge that feels threatened by tracking, shouldn't judge.

I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?

On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!

Dale P
12-17-2008, 08:42 PM
We have never had a problem with high scoring judges. We are actually fond of them.:wink:

SaucyWench
12-17-2008, 08:47 PM
Yeah, well...

Coz
12-17-2008, 10:07 PM
From the view of you folks with more experience?Do you know if the judges that score lower than average might be people who compete more often then judge.I have been thinking of taking the class and judging a few times a year to see what the other guys are turning in.What I wonder is as a competitor if you end up being more critical on the entries.Any ideas ....

SaucyWench
12-17-2008, 11:08 PM
Coz, I haven't noticed lower or higher scoring by a cooker judging at any table I've been at, and we do introduce ourselves, tell what got us into bbq and judging, talk about our impressions of the entries and how we scored after we turn in our cards.

The only time I've ever seen a cook trying to judge outside the rules, was once, when a newly graduated judge, who proudly told everyone at the table that he was a veteran competitor, had only taken the class because his friend was taking it and he (the cook) didn't really need it. He decided he would compare instead of judging each entry on it's own merit. I was table captain, noticed that he hadn't written down an appearance score for the first box as I was starting to show the second. I stopped, told him that he needed to enter his score for the first box, he told me he was waiting to see all the entries. I told him we wouldn't go on until he entered his appearance score for the first entry, apologized to the rest of the table for the delay. He argued, I looked around to find a rep, then he realized I was serious, and he decided to go by the rules before the rep was needed.

I guess there will always be someone who doesn't want to play by the rules, but in 7 1/2 years, that incident was the most blatant I've seen, and I think most competitors who judge would be completely fair in their judging.

Rolling Smoke
12-18-2008, 05:09 AM
I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?


CaptTable or LindaM would be better to answer this than I would but from what I understood, the comment cards were only to be used for the extreme deviations.


On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!


We have a nice little contest with a bunch of super people making it all come together for a great cause. We have two of the best reps in the circuit and we're always looking for quality judges to help round everything out. The entry form on the website is from last year so don't send that one in as there will be a few changes on the new one. I'll let you know as soon as it goes online.

Scottie
12-18-2008, 09:36 AM
From the view of you folks with more experience?Do you know if the judges that score lower than average might be people who compete more often then judge.I have been thinking of taking the class and judging a few times a year to see what the other guys are turning in.What I wonder is as a competitor if you end up being more critical on the entries.Any ideas ....


Coz, what I have found is if there was a fresh judging class in the area, just prior to the contest. You will get a low scoring contest. Or if the judges in the area are basically new (under 5 contests judged) you will get lower scores. If you have judges off the street, just about anything can happen. Just hope you don't get 2 at your table or make sure your product is right down the middle of the BBQ avenue...

jminion
12-18-2008, 10:03 AM
I'm totally with you on the high scoring judges being just as much an issue as the low scoring ones. I know of a couple judges who score high ( 8s or 9s) almost all the time, and if they ever give a 6, it means they pretty much hated it. One thing that might help the scoring is if we defined what "average" is. Is it the kind of Q you would get at an average restaurant? What we can cook ourselves? Grandpa's BBQ? I know that DH has cooked some above-average backyard, sometimes better than I've gotten in competition, but where is the bar set?

As far as tracking goes, if it were to be implemented, it should be all judges, with all of us knowing it will happen. If we're all in this to promote the sport of BBQ, none of the components of a competition should feel like we're the opponent. Just as any competitor who doesn't like the rules, shouldn't compete in KCBS comps, any judge that feels threatened by tracking, shouldn't judge.

I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?

On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!

Merl and I ran the committee that tested the new score cards and comment cards in 2007. The comment cards were voted down by the BOD at the end of the test but the new score card was passed. My personel feelings were that the vote would go the other way. The surveys we recieved from cooks and judges where about 85% in favor of the comment cards.