Judging (moved from KCBS NAshville thread)

Jim,

Must be something in the air today, but I can't agree with the judge tracking either.

If a judge were to find out (and they would) that they are being tracked, I believe they would just put down middle-of-the-road scores so as to not bring attention to themselves. They may feel that as long as they are giving 7's and 8's and an occassional 9 or 6, they would fly under the radar. And they would.

Phillip
 
It's really starting to sound like we can't get there from here. I think the majority of the judges would continue to score as they always have regardless of whether they knew they were being tracked or not. If some judges started giving middle of the road scores to stay off the radar, wouldn't that in itself say that there's a problem with consistency?

I don't know why a judge would have a problem with being tracked unless they're intentionally trying to cause dissention. Everything a cook does at a comp is under the microscope and subject to scrutiny so why should it be any different with the judges?

And (while I'm on a roll) what the hell happened with Phillip Fulmer? I've heard all kinds of things since I watched him walk off the field and I figure if anyone knows the truth, you do.
 
Must be something in the air.. As the Lady Vols are out of the top 10 for the first time in 12 years I believe it was...

As to the point... I can see valid issues with both sides for tracking judges...
 
Why would tracking judges need to be done without their knowledge? If the "rogue" judge knows he/she is being tracked, wouldn't that be incentive to play by the rules?

I have no problem being tracked, after all, my name and CBJ number are on every judging slip I fill out, but I would take issue with it being done clandestinely.
 
I agree SW, I never considered the tracking of judges to be done without their knowledge. When I was told about the plan to track judges two years ago, it was just another part of collecting data from a contest. It was never presented to me as a 'black op' designed to undermine the judges but rather a tool to identify the problem(s) with inconsistant scoring and correct them.

As a competitor, I would have just as much of a problem with a judge who didn't want their scores tracked as I would with a team that didn't want their meat inspected. If the tracking process is implemented, I think the majority of the judges will accept it as a tool to improve the overall scoring system.
 
Just the thought of being tracked could be in the mind of a rogue Judge. Everyone knows that there is a butthead somewhere in the crowd no matter what you are doing, and it is a shame that it only takes one to ruin somebodys weekend.

I dont have the solution for the problem but I cant see where tracking low scores would hurt anything.
 
Dale, I'm not saying track just the low judges. I'm saying track ALL judges. All of the scores entered by the judges is data that we are not using. The only was we know if the CBJ classes are really getting the message across to the judges is to track the scores that are being put on paper. Look at what happened a few years ago when they changed the scoring process...you hardly ever see a perfect 180 anymore and when you do, it's usually dessert. That down curve in the overall points was not a result of the judging classes as much as it was a result of the rules change. We know we have rogue judges and it's not just the ones that are lowballing everything that hits their table. The ones that give all 9's to something that is garbage are just as guilty. The problem as I see it is a broad deviation or inconsistency in scores. The only way I see to eliminate the problem is to go to the source (i.e. the judges) and correct it by retraining or removing them from the equation. Our reps at Kettering do a great job of catching rogue scores and talking to the people who are giving them but unfortunately this is not being done on a global basis and the reps I speak of cannot officiate at all the contests that we cook.
 
I'm totally with you on the high scoring judges being just as much an issue as the low scoring ones. I know of a couple judges who score high ( 8s or 9s) almost all the time, and if they ever give a 6, it means they pretty much hated it. One thing that might help the scoring is if we defined what "average" is. Is it the kind of Q you would get at an average restaurant? What we can cook ourselves? Grandpa's BBQ? I know that DH has cooked some above-average backyard, sometimes better than I've gotten in competition, but where is the bar set?

As far as tracking goes, if it were to be implemented, it should be all judges, with all of us knowing it will happen. If we're all in this to promote the sport of BBQ, none of the components of a competition should feel like we're the opponent. Just as any competitor who doesn't like the rules, shouldn't compete in KCBS comps, any judge that feels threatened by tracking, shouldn't judge.

I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?

On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!
 
We have never had a problem with high scoring judges. We are actually fond of them.:wink:
 
From the view of you folks with more experience?Do you know if the judges that score lower than average might be people who compete more often then judge.I have been thinking of taking the class and judging a few times a year to see what the other guys are turning in.What I wonder is as a competitor if you end up being more critical on the entries.Any ideas ....
 
Coz, I haven't noticed lower or higher scoring by a cooker judging at any table I've been at, and we do introduce ourselves, tell what got us into bbq and judging, talk about our impressions of the entries and how we scored after we turn in our cards.

The only time I've ever seen a cook trying to judge outside the rules, was once, when a newly graduated judge, who proudly told everyone at the table that he was a veteran competitor, had only taken the class because his friend was taking it and he (the cook) didn't really need it. He decided he would compare instead of judging each entry on it's own merit. I was table captain, noticed that he hadn't written down an appearance score for the first box as I was starting to show the second. I stopped, told him that he needed to enter his score for the first box, he told me he was waiting to see all the entries. I told him we wouldn't go on until he entered his appearance score for the first entry, apologized to the rest of the table for the delay. He argued, I looked around to find a rep, then he realized I was serious, and he decided to go by the rules before the rep was needed.

I guess there will always be someone who doesn't want to play by the rules, but in 7 1/2 years, that incident was the most blatant I've seen, and I think most competitors who judge would be completely fair in their judging.
 
SaucyWench said:
I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?
Click to expand...


CaptTable or LindaM would be better to answer this than I would but from what I understood, the comment cards were only to be used for the extreme deviations.


SaucyWench said:
On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!
Click to expand...


We have a nice little contest with a bunch of super people making it all come together for a great cause. We have two of the best reps in the circuit and we're always looking for quality judges to help round everything out. The entry form on the website is from last year so don't send that one in as there will be a few changes on the new one. I'll let you know as soon as it goes online.
 
Coz said:
From the view of you folks with more experience?Do you know if the judges that score lower than average might be people who compete more often then judge.I have been thinking of taking the class and judging a few times a year to see what the other guys are turning in.What I wonder is as a competitor if you end up being more critical on the entries.Any ideas ....
Click to expand...


Coz, what I have found is if there was a fresh judging class in the area, just prior to the contest. You will get a low scoring contest. Or if the judges in the area are basically new (under 5 contests judged) you will get lower scores. If you have judges off the street, just about anything can happen. Just hope you don't get 2 at your table or make sure your product is right down the middle of the BBQ avenue...
 
SaucyWench said:
I'm totally with you on the high scoring judges being just as much an issue as the low scoring ones. I know of a couple judges who score high ( 8s or 9s) almost all the time, and if they ever give a 6, it means they pretty much hated it. One thing that might help the scoring is if we defined what "average" is. Is it the kind of Q you would get at an average restaurant? What we can cook ourselves? Grandpa's BBQ? I know that DH has cooked some above-average backyard, sometimes better than I've gotten in competition, but where is the bar set?

As far as tracking goes, if it were to be implemented, it should be all judges, with all of us knowing it will happen. If we're all in this to promote the sport of BBQ, none of the components of a competition should feel like we're the opponent. Just as any competitor who doesn't like the rules, shouldn't compete in KCBS comps, any judge that feels threatened by tracking, shouldn't judge.

I do have a question about the comment cards. We were in the process of moving across country when they were being tested, so we never used them. Were you supposed to comment on every sample, or just the ones that were really great or really bad?

On another note-Jim, we will send our judging applications for Kettering, from the comments on another thread, sounds like one we want to be a part of!
Click to expand...

Merl and I ran the committee that tested the new score cards and comment cards in 2007. The comment cards were voted down by the BOD at the end of the test but the new score card was passed. My personel feelings were that the vote would go the other way. The surveys we recieved from cooks and judges where about 85% in favor of the comment cards.
 
You must log in or register to reply here.

Similar threads

Hotch
The 2023 Texas BBQ Brethren Fall Bash Planning Thread
2
Replies
22
Views
4K
Sid Post
S
Q
Smokefire vs RT 1070?
Replies
13
Views
1K
glitchy
G
LYU370
Nashville KFC
Replies
16
Views
1K
HogGrog
HogGrog
rwalters
Pellet Cookers 101 - Part One
Replies
0
Views
558
rwalters
rwalters
cayenne
FIRST COOK on the new Offset Smoker!!! Pictures and comments...
2
Replies
22
Views
901
Bury Me In Smoke
B
Back
Top