PDA

View Full Version : Should we get something from scoring?


Bentley
09-05-2007, 07:28 PM
With the debate going on about judging, it has got me to thinking about a poll! http://www.cbbqa.org/forum/style_emoticons/default/ohmy.gif

As a team, all I have to go on for scores is the final sheet provided by the reps. If I judge a contest I get nothing showing my scores. I would be interested to see how many teams and judges would like to see this addressed by KCBS?

I would like to see the scoring sheet be a three part card. 2 NCR(carbon-less paper) sheets of paper and the last sheet a hard copy. The top sheet would go to the judge, the second sheet to the team and the third sheet to KCBS. It has been awhile since I judged, I am not even sure if the judges name and KCBS number go on the current sheet. If it does that info would only be on the sheet that goes to KCBS.

It would be nice to know that the scores I received from the judge are the same that were on my final tally sheet. I would also like to see KCBS institute a policy of pen ink instead of pencil. That way there is no way a person can say a score was changed by anyone else!

MilitantSquatter
09-05-2007, 07:50 PM
As a cook, I have no desire to get a dirty scorecard with BBQ sauce induced fingerprints. Aren't the score's double counted when entered into the computer for accuracy by two different people ?

Where I thought you were asking was for the judges to get to see the scores from the other judges at their table so they can evaluate their measurements compared to their peers who judged the same food. That may or may not help bring more consistency to a judge who finds out they are overly critical or too free in their high assessments.

Pens might help, but it can be crossed out. What is the exact rule about a judge fully erasing a score or crossing out. If the table captain or rep see a score changed but can't makle out the judges original scoring, what is supposed to get done ? You can't penalize the team right ?

BBQchef33
09-05-2007, 07:54 PM
jsut to clarify..

are u asking if we(cooks) would like to see the actual judging cards in addition to the computrer printout?

Sawdustguy
09-05-2007, 08:36 PM
Bentley,

IMHO, I can see no reason to get the gopped up saucy score sheets from the judges. I am very satisfied that that were transferred properly and the score sheets reflect my true scores. I don't think there is any reason for "Hanging Chad" allegations at a BBQ contest. I am however, in favor of having checklisted feedback cards filled out by every judge at the contest and distributed to the teams.

KC_Bobby
09-05-2007, 09:07 PM
I really don't see what you are seeing with the cards that you don't already see on the sheets - other than a possible change - which I'm sure has happened in the past, but I have no reason to believe it's the norm.

I'd like to know how many judges were certified vs non-certified that were judging as well as scores given by certified vs. non.

Bentley
09-05-2007, 10:30 PM
jsut to clarify..

are u asking if we(cooks) would like to see the actual judging cards in addition to the computrer printout?


Yes.

But I guess it would have to be an 8 part form so that each team would get results. Chit, we all know this will never happen, just dreaming!

Bentley
09-05-2007, 10:37 PM
[quote=KC_Bobby;456884]I really don't see what you are seeing with the cards that you don't already see on the sheets - other than a possible change - which I'm sure has happened in the past, but I have no reason to believe it's the norm.

quote]

I guess most people are just a lot more trusting than me! I guess I just have seen what some reps will do at contests and I did not like it!

slat
09-05-2007, 10:51 PM
I really wouldn't care to see the judging cards when I cook, but when I judge I would like to see how my scores were compared to the other judges at my table. It wouldn't have to have any judges number on it except yours. I think I judge very fairly, but I'm not sure that that is the case with all judges.

I think that the KCBS needs to make sure that everyone understands that the BBQ they are going to judge is competition BBQ not commercial. I think that is why some of the scores are so crazy sometimes.

ihbobry
09-06-2007, 03:51 AM
Where I thought you were asking was for the judges to get to see the scores from the other judges at their table so they can evaluate their measurements compared to their peers who judged the same food. That may or may not help bring more consistency to a judge who finds out they are overly critical or too free in their high assessments.



What Vinny said.

MilitantSquatter
09-06-2007, 06:55 AM
I guess most people are just a lot more trusting than me! I guess I just have seen what some reps will do at contests and I did not like it!

Not sure what you saw the reps do, but if they did something inappropriate, did you report it to KCBS ? If so, did you get a response ?

Westexbbq
09-06-2007, 08:36 AM
Usually judges are reminded about not changing their score, no erasing or crossing out. Also the table captain will usually check each card to ensure the judge filled out each entry/category, i.e. no blanks and that their CBJ number is entered.

River City Smokehouse
09-06-2007, 08:51 AM
To me it is what it is. You aren't going to change anyone's mind. You already get your scores on a sheet and you can see by that if and where you need to concentrate on improvements and that's good enough for me.
I used to complain over and over but have come to the conclusion that there is a judge at every table that scores something low. You are not always going to get great scores.
http://smileys.smileycentral.com/cat/36/36_11_8.gif

benjet
09-06-2007, 11:33 AM
Aren't the score's double counted when entered into the computer for accuracy by two different people ?


Well I've certainly never seen that, unless it happens after the CBJs leave the room, even though they are entering the scores during the next turn in...

Mo-Dave
09-06-2007, 11:34 AM
What Vinny said.

I second that emotion.
Dave

Bentley
09-06-2007, 11:48 AM
Not sure what you saw the reps do, but if they did something inappropriate, did you report it to KCBS ? If so, did you get a response ?

Yes and NO!

ModelMaker
09-06-2007, 01:42 PM
I really wouldn't care to see the judging cards when I cook, but when I judge I would like to see how my scores were compared to the other judges at my table. It wouldn't have to have any judges number on it except yours. I think I judge very fairly, but I'm not sure that that is the case with all judges.

I think that the KCBS needs to make sure that everyone understands that the BBQ they are going to judge is competition BBQ not commercial. I think that is why some of the scores are so crazy sometimes.

I too am curious as to what the rest of the judges at my table scored,
BUT what I don't want to do is learn how to score the same as the other judges at my table. I know the rules to judge BBQ and I know MY taste I don't want to EVER score a entry a certain way to keep even with the other judges.

Better off not knowing, the table captain is responsible for watching for really offbeat scoring. If there is a judge scoring way off the mean he is to report it to the rep. I think that works well enough.
ModelMaker

The_Kapn
09-06-2007, 05:33 PM
With the debate going on about judging, it has got me to thinking about a poll! http://www.cbbqa.org/forum/style_emoticons/default/ohmy.gif



I really don't see a lot of "debate" about judging except on the BASSO forum. It is an almost daily bashing over there. :evil:

Most of us understand that judgng "is what it is" and only incremental change will take place over time.

KCBS is trying feedback cards and folks are up in the air over that!

I think we just need to go cook, or judge, and then have a beer! :lol:

TIM

Bentley
09-06-2007, 07:22 PM
I really don't see a lot of "debate" about judging except on the BASSO forum. It is an almost daily bashing over there. :evil:

TIM

Lots of debate on the CBBQA forum!

It seems like a few cooks on this forum would like to see judging cards. But like you said, changes may be incremental, I don't think you will see any change from KCBS!

Sawdustguy
09-06-2007, 08:19 PM
Actually the KCBS is testing and trying to refine Judging Feedback cards at specific contests now. It's going to take a while but I have faith they will get it right.

Kirk
09-07-2007, 06:52 AM
Actually the KCBS is testing and trying to refine Judging Feedback cards at specific contests now. It's going to take a while but I have faith they will get it right.
I hope they do. I'd love to have some idea as to why my entries were scored the way they were, so that I might work to improve them for the next go 'round. I do hope that they will discourage comments like "It was great" or "Inedible." We can glean that kind of information from the scores themselves. We need to know why it was great or inedible if we're going to do any better.
In the interest of the judges, I hope they can do this for each competitor without it getting tedious though. I can see this being a lot of work for a judge to have to so thoroughly evaluate an entry and then try to convey the reasons for their scores. I think that would get old by the end of the day and the judge's enthusiasm (and the scores on the later entries) might suffer.

Merl
09-07-2007, 09:27 AM
Aren't the score's double counted when entered into the computer for accuracy by two different people ?
Where I thought you were asking was for the judges to get to see the scores from the other judges at their table so they can evaluate their measurements compared to their peers who judged the same food.
Pens might help, but it can be crossed out. What is the exact rule about a judge fully erasing a score or crossing out.

Vinny,
Unfortunately the scores are not double counted. You are relying upon the accuracy of the person who is entering the scores. Most of the time this is done very well and the system works. However there have been those times when people do not do their work, and allow mistakes. Not all mistakes can be caught, but many common data entry failures and mistakes can be identified, if you take the time to review your work product.

I agree it would be great to have judges see how they judged compared to the table. We encourage judges to discuss entries after they turn in their score cards. Currently the software will not track judges for us. Their is a program that might, but is not in use, and has not been beta tested. We are looking for programers and Don Harwell had a request for bid to write a new program for KCBS.

Judges are not to change a score once written. We take pens to our contest to discourage changes. If a mistake has been made, we require, (not mandated by KCBS) that the judge initial the change and the table captain initial the change. This also protects us as Reps, as to why a score card has a change upon it. Then the table captain has to explain to the us why the change was made. This does, in my opinion discourage the temptation to change scores. Most changes of scores are as a result of comparing entries which is forbidden.

Yours in Q
Merl

slat
09-07-2007, 11:19 AM
I don't want to keep even with the other judges. I score what I think it deserves. The point is just like the teams see where they are it would be nice if a judge could see where they are at. That way if a judge notices that they are always in the lower half then maybe they need to think about how they are judging. Maybe they are getting stuff that is that bad, but if they judge 5 or 6 contest and are always on the lower half then it's time for them to evaluate what they are doing. Maybe competition BBQ isn't to there liking or maybe it's just a chore to go and judge. Who knows? I just think that it could be used as a tool to help.

Bentley
09-07-2007, 12:26 PM
Merl,

Thanks for taking the time to respond. I am not a KCBS hater, but I am not there biggest fan either. I appreciate you pointing out some of the changes that may occour. You appear to be a rep that takes their position seriously, there are a few in CA like that, but a few on the oppisite end of that spectrum as well!

Muzzlebrake
09-07-2007, 04:20 PM
I think what we really need is instant replay

:twisted:

:biggrin:

rbinms33
09-08-2007, 11:12 PM
I personally don't care to see what the other judges at my table scored an entry. When we're done with a category, we generally discuss what we liked and didn't like. Generally, we all agree on the bad one if there is one and there's always one or two that everyone thought was better than the rest. Also, about halfway through judging someone will always ask the Table Captain about our scoring consistency. The response I got today while judging at Ashland was "I can tell which ones ya'll like and don't like". Of course, I'm sure it doesn't always happen like that.

I'd like to add my .02 cents to the whole judging thing. I still consider myself a newbie judge (3 contests) but when I'm consistent with everyone else without knowing how they scored, that tells me everyone is on the same page and is using the same "start at 6 average" when rating the entries. That "average" is sooooo subjective though. And if a judge's "average" is too high, somebody gets low scores. On the flip side, if a judge's "average" is too low, then somebody gets high scores. Either way, somebody is getting scores they don't deserve.

I listened to one of the BBQ Forum podcasts a while back and Ray was interviewing Ed Roith. Ed was talking about a program that would "track" judges and their scoring habits so those that had screwed up "averages" could be identified. I think this is what Merl is referring to in his earlier post. I think this is an awesome idea and would go a long way in helping to better train those judges that just don't know any better and identifying those that know better but think their own way is better.

Something else that I thought was a great idea was something I read in the Jun '07 issue of the Bull Sheet (Merl, I think your name was on this one too) about putting keywords next to the scoring numbers. For newbie judges, this would be like a legend on a map and a truly good thing. It went something like this: 9 - Excellent, 8 - Very Good, 7 - Above Average, 6 - Average, 5 - Below Average, etc. It would also give the judges the ability to comment on any 9, 4, 3 or 2 score they gave to give the cooks constructive feedback.

Just looking at these two ideas, I like the direction that judging is headed. I think both of them would greatly improve our judging system.

Rookie'48
09-09-2007, 12:02 AM
I think that the #/keyword system is being tested along with the feedback cards by a few reps this year at each of the contests that they do. KCBS is then supposed to figure out weather to expand or scrap the project. At least that's what I remember hearing.
Dave