PDA

View Full Version : Comment Cards-Mandatory?


Rooter-Tooter BBQ
04-11-2013, 09:07 AM
In lou of the previous thread on Judges Scoring, I was wanting to see if the powers at be and any comments from the Brethren would agree or disagree.

Personally I feel that an excellent way of us competitive cooks getting better is having an outside source, such as the judges, comment on our food, hence comment cards. I know that the Reps tell the judges to do comment cards, but in my 3 yrs of competing I have only had 1 comment card, which was well deserved.

So here is the question, if a team receives a score of 6 or lower should a comment card be mandatory?

We put alot of money, time, and effort into our cooks, so the least we could get is a comment card on why the score was 6 or lower. Agree?

This is post is not in a negative or mean way just a suggestion to help us get better.

nucornhusker
04-11-2013, 09:10 AM
That would be great, but how many judges would score something up just so they don't have to fill out a comment card?

Rooter-Tooter BBQ
04-11-2013, 09:21 AM
True, I guess its a win win :grin:

rweller
04-11-2013, 10:15 AM
Anything lower than a 6 and I will fill out a comment card, not for a 6 though.

Vince RnQ
04-11-2013, 10:52 AM
So here is the question, if a team receives a score of 6 or lower should a comment card be mandatory?


This has been proposed, just as you wrote it, and the reply back was, "Well, if we do that then judges will never give scores lower than 6 anymore."

The reality is that this will never happen because KCBS does not have the desire or the will to make it happen. The current mindset seems to be to dumb down the process (i.e. revised pork rule).

EMTTLC
04-11-2013, 10:52 AM
I wouldn't have a problem mandating that judges fill out a comment card on each entry regardless of score. They have thirty minutes between turn-ins, thats five minutes for each piece they sampled. It wouldn't take 30 seconds each. I think a judge should be able to articulate what they liked or didn't like about an entry. I consistantly do more comment cards than everyone at my table combined. Feedback makes for better cooks.

Uncle Buds BBQ
04-11-2013, 11:28 AM
I consistantly do more comment cards than everyone at my table combined.

That's 'cause you're a COOK! :hungry:

Slamdunkpro
04-11-2013, 11:40 AM
I wouldn't have a problem mandating that judges fill out a comment card on each entry regardless of score. They have thirty minutes between turn-ins, thats five minutes for each piece they sampled. It wouldn't take 30 seconds each. I think a judge should be able to articulate what they liked or didn't like about an entry. I consistantly do more comment cards than everyone at my table combined. Feedback makes for better cooks.

They really don't have 30 minutes - I've seen tables get food as late as 15 minutes into the judging period.
<broken record mode>
It's not a comment card issue - it's a lack of quantifiable standards issue
</broken record mode>

bbq.tom
04-11-2013, 12:20 PM
Personally, I always give a comment card on anything 6 or below, and even sometimes on a 7 if I have time.

Judges DO NOT get 5 minutes per sample as indicated above!!! It is common to see a table not get their boxes until the first table is almost done scoring.
Usually judges get about 1-2 minutes to: pick up a sample, pull all the parsley off so it doesn't effect the taste, bite, think/chew, bite again on a different side of the thigh/rib or a different type of meat in the same box, wipe off their hands and mouths, write down their scores, eat a small piece of cracker and take a drink of water before starting over again with the next sample. Believe it or not (cooks), judges don't have 5 minutes for each sample!
I FIRMLY believe that judging could be improved immensely by having the judges write down on the score cards WHY they gave each sample the score they did. This would be for EACH and EVERY entry! Of course, this means that turn-in times would need to be extended to at least 45 minutes between instead of 30. But would be actual useful information for the teams and would make the judges think more about the score and why they gave what they did.

Cayman1
04-11-2013, 12:52 PM
Feed back would be great, even for good scores, but I would be satisfied with cards for 7's or less. We don't get many 7's but they will kill a chance at a win and it would be nice to know why a judge felt it deserved a 7. Maybe it was just a bad piece of meat that judge got, but I guarantee, a team will fret over a 7 for a long time trying to figure out what went wrong and maybe nothing did.

Our last comp, we had, if I recall exactly, ten 9's and the rest 8's on chicken, except for one judge that gave all 7's. Knowing why that judge felt it deserved only 7's would help to know if we need to change anything or what exactly the issue was.

Alexa RnQ
04-11-2013, 12:58 PM
I feel the mandatory-card trigger should be a score of 5. If 6 is "average", then if you have determined an entry is below average you'd better be able to come up with two to five words as to WHY it's below average.

Make it even easier -- put checkboxes for common issues like "too salty", "dry", "tough", whatever.

The time issue is a claim without merit. My son is a CBJ and has served at a contest where he judged, was table captain AND had time to generate relevant comment cards when merited. If boxes aren't getting back to the tables in a timely manner, I'd be looking squarely at the reps overseeing the sorting/renumbering of boxes.

Divemaster
04-11-2013, 01:28 PM
I consistently do more comment cards than everyone at my table combined. Feedback makes for better cooks.

That's 'cause you're a COOK! :hungry:

Feed back would be great, even for good scores...

Ramona and I did Vegas last year and had the rep complain that we were doing too many cards. I did them for just about every entry. some were statements like 'under done' others were 'great job'.

olewarthog
04-11-2013, 01:42 PM
the GBA scoring system goes from 7 to 10 in each of the 3 criteria. A comment card is mandatory for any 7 or 8. Yes, there will be some judges who will not score low to avoid filling out comment cards. IMHO, the value of the comment cards outweighs the few judges who score up just to avoid writing a card. Judges are also encouraged to give "good" comment cards also. I do the scoring at a lot of GBA contests. There will normally be between 5-6 comment card for each of the three meats.

cpw
04-11-2013, 02:22 PM
How much time can it possibly take to write "Your chicken was too salty"? I don't see that as a valid argument against comment cards.

Outnumbered
04-11-2013, 02:23 PM
Why not require comment cards period? Any judge should be able to justify a 9 or a 6 or a 3.

K-Train
04-11-2013, 02:43 PM
As long as they are constuctive. If it's not constructive then I'd rather not get one. Positive comments are nice to get as well.

bignburlyman
04-11-2013, 03:42 PM
Don't forget though that the new KSBScore program may not even be able to deal with comment cards, unless they get a method worked out to match up the cards with the correct teams.

sdbbq1234
04-11-2013, 04:15 PM
We do comment cards for low scores AND for high scores!

And, trying to do cards for each entry just is not realistic. You do not have 30 minutes.

You also have to try a bite, wipe your hands, records your comments (usually with a dull pencil), then try the next piece and repeat process. Heck, I have been to comps where the judges table are like little card tables and to have 6 people sitting there; need some elbow room! :mrgreen:

Some comps get behind in bringing the boxes to the tables, so you are really crunched for time.

The last comp we judged was Smokin' On The Bay last year. I know I filled out 15-18 comment cards. Some were for low scores, and some for really high scores.

If there was more time between turn ins, that should give more time for comments.

As for me now being a competitor, I want comment cards!

I also believe there is a lot of stuff that could and should be done by KCBS and the reps to promote the comment cards. But, there is not really much said.

wallace

ckelly
04-11-2013, 04:30 PM
IMHO, high remarks need not be commented on.
Anybody who's cooked in a comp more than once in his life should have a timeline, recipes or otherwise have documentation on what he did. If he does well, he'll know exactly what he did to acheive that.

If an entry is marked down? Absolutely be prepared to justify your lower score.

nthole
04-11-2013, 04:37 PM
Your assumption is you'll get valuable information back form a judge that will help you.

I once got a comment card back that said 'Your beef tasted to beefy.' :crazy:

So if you actually FORCE a judge to write a comment card, you're assuming the feedback will suddenly be more useful?

sdbbq1234
04-11-2013, 04:42 PM
IMHO, high remarks need not be commented on.
Anybody who's cooked in a comp more than once in his life should have a timeline, recipes or otherwise have documentation on what he did. If he does well, he'll know exactly what he did to acheive that.

If an entry is marked down? Absolutely be prepared to justify your lower score.

Don't agree. I have seen a lot of posts where people have said they turned in what they thought was average at best, but yet received a great score.

As for low or marked down scores, yes, anything of a 5 or lower needs to have a comment card.

wallace

Vince RnQ
04-11-2013, 05:01 PM
Ramona and I did Vegas last year and had the rep complain that we were doing too many cards. I did them for just about every entry. some were statements like 'under done' others were 'great job'.


And Part 2 of the problem surfaces. Heaven forbid those poor, tired and overworked Reps have to deal with too many comment cards.

sdbbq1234
04-11-2013, 05:19 PM
And Part 2 of the problem surfaces. Heaven forbid those poor, tired and overworked Reps have to deal with too many comment cards.

Maybe, time to retire?

For the most part, I believe most are out there trying to make something of their work and money they put into it.

Maybe the reps and KCBS should really take another look at what this "past time" thing has become. If the entry fee was a few bucks, and everything related to a comp was cheap (smokers, meat, etc...), it would be different IMHO.

wallace

nucornhusker
04-11-2013, 05:25 PM
True, I guess its a win win :grin:It may be good for one comp, but a false high score isn't going to help me as a cook. I want to be judged as accurately as KCBS allows for, on how the judge thought my food was. If I'm cooking 5's (I'm not) but the judge scores me 7 or 8 because he doesn't want to fill out a card, that doesn't help me.

ckelly
04-11-2013, 05:48 PM
Don't agree. I have seen a lot of posts where people have said they turned in what they thought was average at best, but yet received a great score.

As for low or marked down scores, yes, anything of a 5 or lower needs to have a comment card.

wallace


Alas, the COOK thought it was average but record keeping and journals will tell them what steps they took to produce the product they turned in. (within reason of course)

Stoke&Smoke
04-11-2013, 06:04 PM
Never got one worth much. My favorite was a pork entry that said it tasted rancid. Scores for taste were 9 8 8 8 7 4

Have had one or two that said ribs tasted like lighter fluid, something we never use. Haven't really ever found them helpful

sdbbq1234
04-11-2013, 06:42 PM
Alas, the COOK thought it was average but record keeping and journals will tell them what steps they took to produce the product they turned in. (within reason of course)

Ah, yes! And if they had a few comment cards saying what the judges liked about it, I would think that would be helpful......

wallace

NazBQ
04-11-2013, 07:08 PM
If comment cards were required it would probably just piss everyone off even more. It sucks, but you're always going to have judges judging based on personal tastes. And to see those opinions written on comment cards at a comp would make for some issues.

Pappy Q
04-11-2013, 07:13 PM
Don't need them, don't want them, don't find them helpful. But that's just my worthless opinion.

Svenmac
04-11-2013, 09:59 PM
The best thing Kcbs could ever do is have all judges shadow a team and see how much $$$$ and time the cooks put into a contest. Secondly, the Kcbs should track judges scores and judges who constantly judge outside the bell curve should be dismissed. I know taste and tenderness is subjective but when you get five 9s and a 6 on appearance it makes you wonder about the judging pool !!!

Smoke'n Ice
04-12-2013, 06:55 AM
The criteria for, and use of, comment cards is a “top down” initiative and no amount of discussion or debate at this level will ever change that.

Correct me if I am wrong, for I slept since then, but was there not a motion and debate at a BOD meeting regarding the comment cards and the final solution was, they will go away with the new score “program” which does not have the ability to get out of its own way.

There are more judges than cooks so is this "By design?"

gettinbasted
04-12-2013, 07:17 AM
Don't forget though that the new KSBScore program may not even be able to deal with comment cards, unless they get a method worked out to match up the cards with the correct teams.

Yep, they are gone with the new system. It won't even be an option. They will, however, have the means to track a judges performance. I doubt it will get used, but the data will be there.

Hawg Father of Seoul
04-12-2013, 07:59 AM
Here's my problem with tracking a judge's performance. They would have to evaluate the judge against all tables they have judged to see if they are actually a statistical outlier. I doubt that any one in KCBS could handle the stats for that. (You could use the I/O psychology program at MSU to do it)

The entire beauty of the current program is that you have the "samples" of judges that resemble randomness. Sure you have some systematic variance and some silos, but it is better than medical "random samples".

If they start a BBQ inquisition, they may rid the system of GOOD variance. Believe it or not a LOT of these scores that are all over the place are accurate and would hold true if the same sample was provided to a larger audience. Is that not the point, to give an accurate score?

Vince RnQ
04-12-2013, 10:47 AM
If judges were to be tracked, they might get some feedback on their performance. You know, kind of like the way the judges give cooks feedback on their performance.

Cooks use the constructive feedback they get from judges to improve the quality of their cooking so it would stand to reason that judges could use the constructive feedback they get from tracking to improve the quality of their judging.

So, tell me, how is this a bad thing?

Hawg Father of Seoul
04-12-2013, 12:04 PM
If judges were to be tracked, they might get some feedback on their performance. You know, kind of like the way the judges give cooks feedback on their performance.

Cooks use the constructive feedback they get from judges to improve the quality of their cooking so it would stand to reason that judges could use the constructive feedback they get from tracking to improve the quality of their judging.

So, tell me, how is this a bad thing?

I assume you are directing this to me. I really thought I did explain it.

The question is calibration vs. Systemic bounding.

Do you really think that judge 6 does not know what they are doing?

Vince RnQ
04-12-2013, 12:49 PM
Let me give an example of why I think tracking is good...

We all know there are judges who scores entries consistently lower than every other judge at his/her table at every contest. Tracking will allow the organization to clearly identify this type of problem and do something about it. Retraining comes to mind as a possible corrective measure but there are certainly other steps. And, since judges can be tracked, those who have been identified as being well outside the norm, have been retrained, etc., their performance after corrective steps can also be tracked to see if the retraining, etc., made any difference. If not, well, now there are clear and well defined reasons for saying "Adios, Judge #6!".

Any tool, if used improperly, can result in abuse. That, however, should not prevent us from using tools at all.

Hawg Father of Seoul
04-12-2013, 01:13 PM
I get what you are saying.

Question is do you want consistent scores or scores that best represent what the population thinks of your Que?

What punishment is fitting for some one who gives out 10.1% more nines? Retraining is punishment in lipstick.

gettinbasted
04-12-2013, 01:16 PM
Judges already operate in too narrow of a range of possible scores in my opinion. If we start to point out ones that don't it will get worse. Many judges will come in to eat, mark down an 8, and go home so they don't get labeled as a statistical outlier. This will make the current "3 point scoring" problem worse.

olewarthog
04-12-2013, 02:18 PM
IMHO, high remarks need not be commented on.
Anybody who's cooked in a comp more than once in his life should have a timeline, recipes or otherwise have documentation on what he did. If he does well, he'll know exactly what he did to acheive that.

Here's a situation from a GBA contest I judged recently. I had given 10's in both appearance & tenderness. I would have also given it a 10 in taste except it was submitted with sauce on the side & GBA rules require it to be judged with the sauce. Without the sauce, I would gave scored straight 10s. I wrote a comment card telling the team why I scored down in taste. After the judging was done, almost every judge at the table said they scored the entry down due to the sauce. So here was a pitmaster that got mostly 10s with a couple of 9s, who hopefully got some valuable feedback from the comment card I turned in.

Q-Dat
04-12-2013, 03:22 PM
The more I learn about "Certified" judges, the more I like the idea of judges who have not been "educated" in what is good BBQ. They have to decide for themselves.

Vince RnQ
04-12-2013, 03:42 PM
I get what you are saying.

Question is do you want consistent scores or scores that best represent what the population thinks of your Que?

What punishment is fitting for some one who gives out 10.1% more nines? Retraining is punishment in lipstick.


I understand what you're driving and I agree that we don't want everything to be vanilla, so to speak. We need diversity in our judges just as we need diversity in our cooks.

I have seen a growing trend in the last few years where contests have the lowest scoring teams in each category averaging a score of 777 (140.000) or higher. How much sense does that make? Just as an example, the contest last weekend in Pleasant Hill, MO (http://www.kcbs.us/events.php?year=2013&month=4&id=4106) had 74 teams and only 4 entries out of the 296 possible were scored below the 140 point mark and one of those four was a DQ. So, 3 out of 295 entries were below the 140 point mark in a 74 team field. I think there were some mighty generous judges out there that day.

What I'm trying to say is that a judge who only uses a 7, 8 or 9 is just as bad as a judge who only uses a 5, 6 or 7. Both situations need to be addressed. By tracking, they can identify those who are always in the extreme and do something about it.

If you want to call retraining punishment in lipstick, that's OK with me. I think judges who are always at the extremes have no place in the sport and can dish out a form of punishment that has no lipstick to it at all. While judges do have some expenses involved in pursuing their passion, it doesn't compare to the expenses of the teams and everything that can possibly be done to have the best trained judges in the tent should be done.

Keep in mind that I am a CBJ as well as a cook and I think both deciplines take a lot of skill to master.

CivilWarBBQ
04-12-2013, 03:54 PM
The best thing Kcbs could ever do is have all judges shadow a team and see how much $$$$ and time the cooks put into a contest.

This sentiment has been going around for as long as I can remember, and frankly it doesn't hold any water. What goes into preparing an entry is completely irrelevant to the judging processs. An entry is to be scored for how it performs in the three judging criteria, period.

It matters not if an entry took 100 years and a 100 billion dollars, or ten minutes and ten cents to get to the judges. Only Appearance, Taste and Texture matter by KCBS rules.

Vince RnQ
04-12-2013, 06:32 PM
This sentiment has been going around for as long as I can remember, and frankly it doesn't hold any water. What goes into preparing an entry is completely irrelevant to the judging processs. An entry is to be scored for how it performs in the three judging criteria, period.

It matters not if an entry took 100 years and a 100 billion dollars, or ten minutes and ten cents to get to the judges. Only Appearance, Taste and Texture matter by KCBS rules.


I know a number of judges who have cooked with a team as part of their Master CBJ requirement and every single one of them has said that they had no idea how much work teams put into getting that meat to the judging table. Their cooking experience has given them a better perspective and inspired them to be much more mindful of their responsibilities when judging. That's something very positive that can come from a judge cooking with a team.

Q-Dat
04-12-2013, 07:25 PM
I know a number of judges who have cooked with a team as part of their Master CBJ requirement and every single one of them has said that they had no idea how much work teams put into getting that meat to the judging table. Their cooking experience has given them a better perspective and inspired them to be much more mindful of their responsibilities when judging. That's something very positive that can come from a judge cooking with a team.

Agreed. It won't and shouldn't change their judging criteria, but it could possibly correct their attitude about judging if they are too cavalier in scoring.

CivilWarBBQ
04-12-2013, 08:51 PM
Sorry, but that still doesn't play. Empathy for the participant is not a requirement of a good judge. By that standard, you'd expect every courtroom judge to grow up in poverty and struggle with drug addiction before they'd be allowed to hear a criminal case.

Is understanding the role of the competition cook a positive thing that adds to the experience of the CBJ? Yes. Is it a requirement or guarantee of competency? No.

AppalachianAmericanBBQ
04-13-2013, 09:11 PM
I got a comment card on my chicken that said it tasted like lighter fluid. I didn't understand it because I dont use lighter fluid at all. So how can something taste like lighter fluid if youve never use it when you cook

Svenmac
04-13-2013, 11:05 PM
This sentiment has been going around for as long as I can remember, and frankly it doesn't hold any water. What goes into preparing an entry is completely irrelevant to the judging processs. An entry is to be scored for how it performs in the three judging criteria, period.

It matters not if an entry took 100 years and a 100 billion dollars, or ten minutes and ten cents to get to the judges. Only Appearance, Taste and Texture matter by KCBS rules.


So you don't think judges seeing how much work that goes onto BBQ and takes that knowledge into the tent with them would make them a more accurate scorer?

I wished I lived in your utopian world where personal preference, bias and preconceived ideas were abolished. Because in this world people including the judges have preconceived ideas on appearance, taste and texture. This translates to scores all over the place. Since you obviously think you are so smart why don't you develop a machine to test all three with laboratory accuracy?

Svenmac
04-13-2013, 11:13 PM
Sorry, but that still doesn't play. Empathy for the participant is not a requirement of a good judge. By that standard, you'd expect every courtroom judge to grow up in poverty and struggle with drug addiction before they'd be allowed to hear a criminal case.

Is understanding the role of the competition cook a positive thing that adds to the experience of the CBJ? Yes. Is it a requirement or guarantee of competency? No.

Ok so you argue that judges should not shadow teams in your first post and then say it adds to the experience do a CBJ?

Ill agree it won't guarantee a thing, but maybe it will help the grumpy "nobody gets a nine" judge or the just there to eat "everybody gets a nine" judge determine if they should really be a CBJ because their attitudes can cost a cook a lot of money should they hit or miss , respectively, one of their tables.

Hawg Father of Seoul
04-14-2013, 09:50 AM
I got a comment card on my chicken that said it tasted like lighter fluid. I didn't understand it because I dont use lighter fluid at all. So how can something taste like lighter fluid if youve never use it when you cook

Have a theory on that... Step away from the blue bag.

Rich Parker
04-14-2013, 10:11 AM
So you don't think judges seeing how much work that goes onto BBQ and takes that knowledge into the tent with them would make them a more accurate scorer?

I wished I lived in your utopian world where personal preference, bias and preconceived ideas were abolished. Because in this world people including the judges have preconceived ideas on appearance, taste and texture. This translates to scores all over the place. Since you obviously think you are so smart why don't you develop a machine to test all three with laboratory accuracy?

Wow...relax on the personal attacks. I agree with civil it doesn't matter how much it costs that shouldn't be in a judges mind. I have seen guys with motorhomes and jambos get beat by guys in ezups and drums or folks with $100 briskets getting beat by Walmart selects. I don't see where any of that should matter to a judge?

I don't want to see the comments mandatory but leave them as an option. I would also like to see reps give some direction to the judges to explain to them not to try and tell us how to cook (e.g. don't use so much injection) but just what they are tasting and don't like (e.g. spicy) or what they don't like about the texture (e.g. tough) not a comment I have received in the past "you should cook it a little hotter or longer to be more tender".

Svenmac
04-14-2013, 11:22 AM
Sorry I didn't mean to get personal but he refuted my opinion and I did the same of his. You have yours and I have mine also. To be the best at something you must know every aspect of the process. Sorry if you don't feel the same. I'm sure just as many agree with me as do you.

Svenmac
04-14-2013, 11:29 AM
Also rich, I think you're missing the point. I said I only want them to understand what all goes into a contest. I don't care if its Lotta Bull and his 300k setup or joe bob with his WSM either. For the exact reason you mentioned on your rebuttal armchair pitmasters that score up or down based on their personal idea of what BBQ is should checked.

DawgPhan
04-14-2013, 12:20 PM
I got a comment card on my chicken that said it tasted like lighter fluid. I didn't understand it because I dont use lighter fluid at all. So how can something taste like lighter fluid if youve never use it when you cook


I am going to clear this up for you. Hopefully when the next guy says the exact same thing you can clear it up for him.

You didnt manage your fire properly. That taste the judge ID'd as lighter fluid is dirty smoke on your meat.

You are going to say that you always have perfectly clear blue smoke rolling when you cook.

I am going to say, well keep doing what you are doing.

The problem that this highlights is not stupid judges or judges need to understand the labor of love that is competition bbq. What it shows is that the process works exactly as it should and that comment card while attempting to provide you with great feedback failed because of terminology.

The card should have read there was an off taste to the chicken. Taste scored down accordingly. Too often judges are trying to diagnose the issue on a comment card, rather than just highlight the comment card.

Comment cards should almost always be a description of a problem and not a diagnosis. This is something that could be handled with a simple reminder and explanation before each contest to the judges. No need for retraining, no need for mandatory, just explain when and how they should be using the comment cards. Let the reps pull any cards that dont conform and then the rep can remind that judges privately and individually about why and how they are to use comment cards.

buttburnersbbq
04-14-2013, 03:03 PM
I would like to have comment cards from every judge in every category. I scratch my head every comp when I get my scores back. It is hard to figure when judge #1 gives you 9-8-9 and judge #2 gives you 7-7-7. Why? Comment card would really help justify score. Constructive criticism is always good.

Rich Parker
04-14-2013, 03:52 PM
Sorry I didn't mean to get personal but he refuted my opinion and I did the same of his. You have yours and I have mine also. To be the best at something you must know every aspect of the process. Sorry if you don't feel the same. I'm sure just as many agree with me as do you.

Also rich, I think you're missing the point. I said I only want them to understand what all goes into a contest. I don't care if its Lotta Bull and his 300k setup or joe bob with his WSM either. For the exact reason you mentioned on your rebuttal armchair pitmasters that score up or down based on their personal idea of what BBQ is should checked.

I think I see the point you are trying to make but I just still don't get why a judge should be concerned with the work that goes in to a comp. I have had judges cook with me and tell me they can't believe how easy it is as we sit around and drink beer and that when they cooked with another team they were working sun up to sun down. What should they do then?

Hawg Father of Seoul
04-14-2013, 04:14 PM
I think that judges cooking with teams is a good thing. First, it might help the attitudes of each group. Neither group is god's gift and neither could do it without the other. Second, I think that if a judge understands the work that any team puts in, then they MIGHT be compelled to judge as ACCURATELY as possible.

Also, rest assured that if you get a 9-8-7-7-7-5... then you deserved a 7. Don't worry about the 5, you were not getting paid any way. And the 9 was just as wrong as the 5.

sdbbq1234
04-14-2013, 04:24 PM
I think the reps and sanctioning body should stress to the judges that "most" competitors take these events very serious and note that they invest a lot of time, effort and for the most part money into this.

They need to stress that their scores have a great effect on the outcome and DO NOT TAKE A SCORE LIGHTLY!

Let it go from there.

As for mandatory comment cards, I don't think that's a reality and it would probably end up being worse then it is now with too few.

Yes, I would like comment cards, but when all is said and done, I pretty much know when I have cooked a bad product.

wallace

Svenmac
04-14-2013, 08:58 PM
I guess I'm one of those that work from sunup to sundown at contests so I take it a but more personal I guess. I totally Agree with the last three posts.

bbq.tom
04-15-2013, 08:18 AM
I think the reps and sanctioning body should stress to the judges that "most" competitors take these events very serious and note that they invest a lot of time, effort and for the most part money into this.

They need to stress that their scores have a great effect on the outcome and DO NOT TAKE A SCORE LIGHTLY!

Let it go from there.

As for mandatory comment cards, I don't think that's a reality and it would probably end up being worse then it is now with too few.

Yes, I would like comment cards, but when all is said and done, I pretty much know when I have cooked a bad product.

wallace

At MOST of the contests that I've judged the REPs did exactly that! They almost always tell the judges that the cooks spent a lot of time and money to present the boxes that we are to judge and that we should give them the best scoring effort as well.

I will throw this out (IMHO), I cook in competition as well as judge and know the time, effort and money that it takes; HOWEVER, the boxes we judge are to be judged on particular criteria relating to Appearance, Taste, and Tenderness, and I've never seen an indicator on the scorecard for how much a team spent to submit that box as a criteria.

Judge seriously and to the best of your ability - YES
Take into consideration the amount of money spent - NO
otherwise the EZ-UP/WSM team could NEVER win!!!

wmaes47
04-15-2013, 08:41 AM
Exact verbiage as required to be recited or read by a KCBS Rep at the Judges Meeting:

"Please remember JUDGING IS VERY SERIOUS to the contestants. They have dedicated a lot of time and money to compete here today. This is blind judging and the numbers have been changed on the entries. Please do not remove any label from the container and do not take any numbered container from this area.

Score carefully. Once you have recorded a score it cannot be changed unless directed by the KCBS Contest Representative.

KCBS has implemented the use of Comment Cards at all contests. Providing Comment Cards is part of the obligation of Contest Reps. Writing comments is absolutely voluntary on the part of the Judges. Instructions for use of comment cards will be provided if you need them."

Jorge
04-15-2013, 12:46 PM
Here's my problem with tracking a judge's performance. They would have to evaluate the judge against all tables they have judged to see if they are actually a statistical outlier. I doubt that any one in KCBS could handle the stats for that. (You could use the I/O psychology program at MSU to do it)

The entire beauty of the current program is that you have the "samples" of judges that resemble randomness. Sure you have some systematic variance and some silos, but it is better than medical "random samples".

If they start a BBQ inquisition, they may rid the system of GOOD variance. Believe it or not a LOT of these scores that are all over the place are accurate and would hold true if the same sample was provided to a larger audience. Is that not the point, to give an accurate score?

I think you are probably ahead of the curve at this point.

The software will have the ability to collect the data necessary to do the analysis. Just about everybody has an opinion about what it will show. Having worked with statistics quite a bit I've got no doubt that some trends will emerge quickly. But, based on that experience I'd suggest not making any decisions until a year of data had been captured, and used to test various algorithms.

I don't know if anyone currently employed or serving on the board has the experience to do the analysis or consult, but I know of at least one member that contacted me while I served on the board that does. I don't see any hurdles that can't be negotiated.

Hawg Father of Seoul
04-15-2013, 01:46 PM
I think you are probably ahead of the curve at this point.


Pun intended?

Not trying to be disrespectful at all. If correctly handled, the data will speak for itself. The problem is that a lot of people think that they understand stats. A few easy questions to weed out the posers- 1. How do you handle outliers? 2. What if they are outliers in X and Y space? 4. How and when do you correct for skewness and kurtosis? 3. What program do you use for nonparametric data?

These are all super simple questions for any one that is allowed a crack any commercial data set.

Smoke'n Ice
04-15-2013, 06:26 PM
Based on currently available data, the BOD, the committee responsible for the the Score program, the programmers for the score program, the owers of the software company responsible for the score program do not have a clue about these last posts. Any analysis will require a new committee, and good luck. The folks in the technology committee are not taking advantage of the resources available to the orgainization from the membership.

The bottom line is "Don't Hold Your Breath!"

Outnumbered
04-15-2013, 08:10 PM
The best thing Kcbs could ever do is have all judges shadow a team and see how much $$$$

Why does this matter? We all know the cost of participating. If I get a 7 and the judge thinks it's the worst thing he/she has ever tasted, what does a 7 tell me about that?

When I've judged, the table captain tried to tell me all this. When I said I cooked contests, the TC and the other judges quickly stopped telling me anything. I wanted to learn from them about what judges were thinking. A lot of times, it wasn't what I was, that's for d$#^ sure.

My point is that if you're going to play this game, learn the game. If not, get out. It costs money like anything other "sport" or game.

Personally, if I'm going to spend this much money to compete, I want to get a true indication of how my food is being received.

Svenmac
04-16-2013, 12:30 AM
I play the game very well thank you and I pay the costs. Go back and read most if this has been hashed out. If you get a 7 and truely deserve a 7 then you are a low to mid packer and never will win a contest. My whole point was when you go to contests and 5 judges give you all 9s and the last judge gives you 777. I do have a problem with that. But I'm not restarting the debate for sake of the board

Bentley
04-16-2013, 06:24 PM
The best thing Kcbs could ever do is have all judges shadow a team and see how much $$$$ and time the cooks put into a contest.


I have been a Chief Cook and Judge for 11 years...Do you think when I judge, I really care how much they spent to do it? That was their choice!

I am also gonna score the way I do whether you track me or not. I agree you should be able to articulate ANY score you give and I can. If it comes down to KCBS does not like that my scores deviate too much from the others, you better be ready to ban me, cuz it aint gonna change.

G'pa Herb
04-16-2013, 07:53 PM
My whole point was when you go to contests and 5 judges give you all 9s and the last judge gives you 777. I do have a problem with that. But I'm not restarting the debate for sake of the board

According to the official KCBS 2013 Official Rules and Regulations Judging Procedures, under Judging Procedures, rule #7: "The low score is thrown out.", basically DQing that judge. How is this not a suitable "fix" to the stated problem?

Q-Dat
04-16-2013, 09:38 PM
Ok does anyone really think that anyone here is suggesting that if the judges see what we spend and how hard we work that they should have sympathy on us and give higher scores?

No really I am asking a serious question. Does anyone ACTUALLY think that ANYBODY is dumb enough to suggest that EXACTLY the way I put it in the first sentence?

Ok nobody? Good.

The suggestion is not being made to improve the judging of judges on this site. What it COULD do is maybe give some judges who have far too cavalier of an attitude about judging, a better scope and comprehension of the responsibility and the privilege that they have taken on. A judge who likes to just put 7's out of mental laziness might be encouraged to step it up if they see what we have invested.

The idea of judges cooking with a team was NEVER intended to even begin to change criteria. Only attitudes.

Now is this idea realistic? Who knows...

Svenmac
04-17-2013, 01:17 AM
At least you get what I'm saying.

Guys you are taking my statement out of context. Like I stated before I don't care if your lotta bull with a 300k setup or joe bob with a WSM I made that statement for the exact reason Qdat just said

I know they drop the lowest but it shouldn't happen in first place

Outnumbered
04-17-2013, 11:03 AM
^^^But there are times that could happen. (And I honestly believe we're agreeing here.)

For instance, if you turn in 7 bones and one wasn't done properly and it was tough. The other 5 judges got a bone with good tenderness and you get 8s and 9s, but that 6th judge selected the wrong bone and a jackhammer wouldn't pull it off the bone. So you get a 7 for tenderness. By most judging standards today, that would give you a 7 in tenderness, but the scoring range says that should actually be a 4 or maybe even a 3.

But nobody is going to score a 4 because the "teams spend too much money" or "I don't want to fill out a comment card."

I don't see how giving me a 7 to make me feel better about myself when I really deserved a 4 for tenderness is going to help anyone.

But if you're going to go that way, you have to make sure you're going to get an honest opinion from all the judges. If not, you're opening the door to really getting a lucky table draw for a win. If that happens, I hope I get the table of nice old ladies who don't want to hurt anybody's feelings and everyone else gets the judges who do it correctly. :grin:

Bentley
04-17-2013, 11:22 AM
The suggestion is not being made to improve the judging of judges on this site. What it COULD do is maybe give some judges who have far too cavalier of an attitude about judging, a better scope and comprehension of the responsibility and the privilege that they have taken on. A judge who likes to just put 7's out of mental laziness might be encouraged to step it up if they see what we have invested.


I don't think that anything you do is going to effect change in judges like that, it is just a pit fall of competing...If KCBS thinks tracking them will help, by all means do it, just please have a system in place to be able to deal with the judge...or it is all just blowing smoke.