PDA

View Full Version : Judging Parity


QansasjayhawQ
12-18-2009, 12:51 PM
I've been thinking about this and I would like to hear everyone else's thoughts too.

I do both - cook on a couple of teams AND judge. I started judging simply because I realized after a couple of events that I had NO IDEA what it was that we were shooting for (other than simply having some fun cooking BBQ).

As it turns out, I really enjoy judging and I've been doing it for a couple of seasons and I've enjoyed getting to know 'the regular' judges who travel far and wide to provide their services.

One thing that bothers me is that the scores from a "celebrity" judge (like a local radio personality or the mayor or the fire chief, etc.) are weighted just the same as a non-certified judge that are weighted just the same as a 'rookie' judge (say, less than 10 competitions judged) and their scores are weighted the same as 'experienced' judges and all of them have just as much say in their scores as Master judges.

This doesn't seem like a very fair set up.

Couldn't the KCBS do some things to help level this out? For example;

1) Track all scoring by all certified judges.
2) Weight judge's scores by some kind of 'experience factor' giving greater credence to experienced and Master judge's scores.

That way, the celebrity or non-certified judges who are tasting competition BBQ for the first time don't just give everything a 9 across the board - as they tend to do. And the tendency for rookie judges to compare entries against each other could also be accommodated using a system like this.

This would require more data entry time on site by the rep doing the data entry. Are there any other problems this might cause that I am missing?

Thank you for your thoughts on this.

Lake Dogs
12-18-2009, 01:07 PM
Good point you're making, and this is a large factor in why many/most sanctioned
cookoffs try hard to get/recruit only certified judges.

Nothing is more frustrating to a team than having someone with mediocre product
outscore their tried-and-true better product because a judge or two has no clue
as to what they're doing (or score theirs down for the same reason).

This is a problem with non-santioned cookoffs and other sanctioned cookoffs where
they cant get trained judges (ala. chili).

KC_Bobby
12-18-2009, 01:12 PM
I'd say an organizer that wants to get some celebrity judges should get six of them. Then set them at a table together in a corner and have some volunteers cook their food. The celebrity table wouldn't get used for the comp teams and they'd be non the wiser.

If the mayor of Williamstonburgsonville doesn't have time to take the class, he doesn't have time to judge. :biggrin:

Am I kidding - sorta but if an organizer did it I'd laugh with him

goodsmokebbq
12-18-2009, 01:23 PM
I have had some requests for these "celeb" types to judge and I tell them they have to take the class and they usually back out. They usually don't have tht two hours to give anyways.

Alexa RnQ
12-18-2009, 01:24 PM
I'd say an organizer that wants to get some celebrity judges should get six of them. Then set them at a table together in a corner and have some volunteers cook their food. The celebrity table wouldn't get used for the comp teams and they'd be non the wiser.

If the mayor of Williamstonburgsonville doesn't have time to take the class, he doesn't have time to judge. :biggrin:

Am I kidding - sorta but if an organizer did it I'd laugh with him
This is the best idea I've heard yet. Seriously.

As far as "fairness", the entire field is subjected to the judging pool. Reps here try hard to spread less-experienced judges evenly between tables, so no table is too hard hit. But for sure, teams ask about percentage of CBJs, and are cognizant that if a class was held the night before...

The other judging system I've seen that works around this somewhat is that used by the American Rose Society. In that organization at that time, a newly-minted judge had an apprenticeship period. They would score shows in the company of an experienced judge, to make sure that their initial judging experiences were in line with the milieu in which they judged.

Whether it's through appenticeships or continued education, there are things to be said for doing what can be done to ensure that the greater body of available judges share a common and current worldview.

goodsmokebbq
12-18-2009, 01:40 PM
I've been thinking about this and I would like to hear everyone else's thoughts too.

I do both - cook on a couple of teams AND judge. I started judging simply because I realized after a couple of events that I had NO IDEA what it was that we were shooting for (other than simply having some fun cooking BBQ).

As it turns out, I really enjoy judging and I've been doing it for a couple of seasons and I've enjoyed getting to know 'the regular' judges who travel far and wide to provide their services.

One thing that bothers me is that the scores from a "celebrity" judge (like a local radio personality or the mayor or the fire chief, etc.) are weighted just the same as a non-certified judge that are weighted just the same as a 'rookie' judge (say, less than 10 competitions judged) and their scores are weighted the same as 'experienced' judges and all of them have just as much say in their scores as Master judges.

This doesn't seem like a very fair set up.

Couldn't the KCBS do some things to help level this out? For example;

1) Track all scoring by all certified judges.
2) Weight judge's scores by some kind of 'experience factor' giving greater credence to experienced and Master judge's scores.

That way, the celebrity or non-certified judges who are tasting competition BBQ for the first time don't just give everything a 9 across the board - as they tend to do. And the tendency for rookie judges to compare entries against each other could also be accommodated using a system like this.

This would require more data entry time on site by the rep doing the data entry. Are there any other problems this might cause that I am missing?

Thank you for your thoughts on this.


I like your line of thought, but it would be very hard to calculate what the weights should be. It would take mountains of data and a real statistician to try and work it out. And it would only work if there really is a relationship between judging experience and scores. If on average the new judge always scores low then ok, but if it goes both ways, some new judges score high and some low then a weight won't work. The mean would be about the same as a experienced judge only there variance would be larger. Almost impossible to adjust for.

Skip
12-18-2009, 01:42 PM
Avctually I think FBA does the best job at this. They track all their judges and categorize them according to their judging style. Those that are tough markers those that mark favorably and those who are middle of the road. When they go to seat these judges they do their best to offer each table a equal amount of each type so that each table is "fair". Of course it doesn't always work but it does do something to change that table with four friends who haven't given a 9 in their career or adversely that group of newbies, that gauge BBQ on Charlie Browns thursday night BBQ special, who "can't believe how good all of this food is. I gave all 9's because each one was better then the next." Its these judges I worry most about not the celebs.

goodsmokebbq
12-18-2009, 01:52 PM
I do like the tracking idea, you could weight specific judges (which is pretty analogous to what the FBA does by spreading them out).

Skip
12-18-2009, 02:00 PM
I do like the tracking idea, you could weight specific judges (which is pretty analogous to what the FBA does by spreading them out).


The only problem I see with weighting each judge is that specific areas yield different flavor profiles. A judge could mark more favorably in one area then the next yet still be a tough judge. I think the mixing of judging styles would even out more then weighting. Even if they like the flavor profile they are still judging the same whereas weighting to a specific number may artificially bias the scoring.

Sorry I tried to explain myself. Does that make sense to anyone?

goodsmokebbq
12-18-2009, 02:07 PM
I really like the spreading out idea based on classification (high marker, average marker, etc.). By doing this you are in effect weighting the judges to create parity across the tables. You are using the judges classification to weight the other judges on the table.

I want to hit the table of all high markers!

Weighting each judge would be very ineffiecnt and very impratical, and it would fail if you don't truely understand a judges distribution of scores (like scoring higher in the south, etc). Any sort of weighting is biasing the data, you just need to make sure you are biasing in the right direction, towards the population average. Weighting can be very tricky business.

Buster Dog BBQ
12-18-2009, 02:43 PM
Do you see that with reps? There are some reps I like to go to contest they are at because the scoring is a little higher and more consistent then others.

Skip
12-18-2009, 02:47 PM
Do you see that with reps? There are some reps I like to go to contest they are at because the scoring is a little higher and more consistent then others.

I can't think of a way that particular reps could affect scoring. Only through illegal ways and I won't make that claim at all. The only thing I can think of is the reps have a following of judges and these judges are either easier scorers or the percentage of CBJ's goes up so the caliber of judging goes up.

Lake Dogs
12-18-2009, 02:52 PM
They can if and how they "charge the judges", like by stressing what a 9 means, what
an 8 means... Stressing. If not, even with certified judges, you end up with some
pretty bizarre variances. I do this for CASI (charge the judges) every time and stress
it. Pretty much the scores on the same chili rarely if ever vary by more than a point
or two...

Skip
12-18-2009, 02:54 PM
They can if and how they "charge the judges", like by stressing what a 9 means, what
an 8 means... Stressing. If not, even with certified judges, you end up with some
pretty bizarre variances. I do this for CASI (charge the judges) every time and stress
it. Pretty much the scores on the same chili rarely if ever vary by more than a point
or two...


Ah that makes a lot of sense. Thank you.

cmcadams
12-18-2009, 02:59 PM
I have had some requests for these "celeb" types to judge and I tell them they have to take the class and they usually back out. They usually don't have tht two hours to give anyways.

Just because they're local celebs (usually VERY local) doesn't mean they're any busier than anyone else. If you want free bbq, at least take the time to get an idea what you're doing.

BTW, great logo!

Ford
12-18-2009, 03:46 PM
#1 the KCBS has no control over selection of judges. The Jack is a good example. Good cooks still win with celebs. If we tell the organizer they need to have all CBJ's then the opranizer says I'll run it without you unless you provide the judges I need. Not going to happen.

#2 there are a lot of cbj's out there myself included that have not judged a contest in years but we're still cbj's. Doesn't make us better judges.

#3 I know a lot of cbj's that smoke as in cigars, etc. Now I would far prefer a non smoking non certified judge to taste my entry. Not cutting at smokers here as I enjoy cigars at contests but fdact is after a couple of cigars my taste buds are messed up. That's why I like having a program and sticking to it every time. I don't need to taste for the best slab of ribs.

#4 new cbj's that took the class the night before are often tough scorers. But others are not. there's no formula to apply here

ZILLA
12-18-2009, 04:42 PM
I think it's a mistake placing so much importance on the judges and sometimes it seems, from what I read about it, self importance. IMO If you cook the best BBQ you will still win. It's been shown over and over again with cookers that cook nationwide and cross sanction. IMO CBJs are not necessary nor do we need experienced "professional" judges making the tail wag the dog at competitions. I believe the CBJ and their training is starting to shape comp BBQ turn in's and to me that's wrong. This endeavor we love so much should be about the cooks not the judges

goodsmokebbq
12-18-2009, 04:42 PM
Just because they're local celebs (usually VERY local) doesn't mean they're any busier than anyone else. If you want free bbq, at least take the time to get an idea what you're doing.

BTW, great logo!

Thanks!

Rich Parker
12-18-2009, 06:29 PM
I judged my first comp this last Fall and the reps asked the judges if there were any newbies and made sure that we were not all sitting at the same table and had an experienced judge sitting next to us in case we needed assistance. I thought that was really good and fair.

monty3777
12-18-2009, 06:38 PM
My feeling is that the difference between a judge who took a class and one that hasn't is not significant. The criteria to be a BBQ judge is ridiculous. One day where you are taught how to fill out a form and told that ribs shouldn't fall off the bone - that's hardly training. To become a Certified Beer Judge you have to take a series of classes and then pass a difficult test. Plenty of people are willing to go through the hassle of getting certified. And the reality is that it costs less than $100 to brew a beer that will be judged. It can cost several hundreds to participate in a BBQ contest. Doesn't it make sense that people who spend that kind of money have judges who have REAL training?

BYW, I'm judge # (oh well, I forgot)

Alexa RnQ
12-18-2009, 07:01 PM
IMO CBJs are not necessary
And in my opinion, they are more than "necessary" -- they are the reason I will go to a particular contest, or not.

Yes, good BBQ will rise to the top. We've been fortunate enough to cook well across two sanctioning bodies this past year. But the fact remains that a team must cook differently to please "judges" from the general public who think boiled Chili's and mushy McRibs are the bomb, than they do for a Master CBJ.

Contests/organizers that make lackluster or nonexistent efforts to recruit experienced judges will not see me return. Contests that rely on holding a judging school the night before a contest to make up their "100% CBJs" will not be at the top of my list of places I want to compete again.

Too much goes into our product to not care whether any attempt has been made to educate the judges to a common standard. Do teams deserve something more? Yes, and that's why we value experienced judges so highly. Do I think it could be better than that? Absolutely, through apprenticeship and continuing education.

To become a Certified Beer Judge you have to take a series of classes and then pass a difficult test. Plenty of people are willing to go through the hassle of getting certified. And the reality is that it costs less than $100 to brew a beer that will be judged. It can cost several hundreds to participate in a BBQ contest. Doesn't it make sense that people who spend that kind of money have judges who have REAL training?

This is tremendously interesting, to hear how other sanctioning bodies train judges. Following that model, wouldn't it make sense to have a class entirely on each of the four meats?

There's a HUGE amount of information a prospective new BBQ judge has to absorb, so many variables in how the meat can present, and it's just human nature that nobody's going to sponge it all up and apply it correctly in one go. It seems like either splitting up the category classes, or establishing continuing education, would make sense.

monty3777
12-18-2009, 07:06 PM
There's a HUGE amount of information a prospective new BBQ judge has to absorb, so many variables in how the meat can present, and it's just human nature that nobody's going to sponge it all up and apply it correctly in one go. It seems like either splitting up the category classes, or establishing continuing education, would make sense.


At the VERY LEAST let this level of training be the criteria for being a Master Judge. What is the criteria right now? I think it is the number of contests judged - though I may be wrong.

Here's a link to the Certified Beer Judge program, for comparison. http://www.bjcp.org/index.php

Alexa RnQ
12-18-2009, 07:08 PM
Thirty contests, plus cooking with a team -- invaluable! -- and I'm not certain of the testing involved. In any event, it's not a cakewalk.

BigJimsBBQ
12-18-2009, 09:13 PM
I have cooked in a competition and was scored the lowest every in the chicken category, with turn-ins that I thought were my best ever. Thank goodness the FBA shows the tables by numbers, and the judges at that table by numbers (Table 1, Judge 1, 2 3 4 5 & 6). When I looked at the Table and judges score that judge my chicken, I looked at the other 5 Teams that were judged for the chicken at that table and found out I was the highest scores of the 6 of us. 7 was the highest given in all 3 categories and saw several 5 and majority 6s. I then followed this Table in the other 3 categories and they were consistantly the lowest scorers in all 4 categories.

If any one thinks a low scoring table would not cause a consistant top 5 Team be in the bottom with same product, wake up. The majority of the time it is fair, but there is always that table of judges at a event that are all very low scorers and piss your $1K down the drain for you.

I respect Judges, Organizers very much. I just expect the same respect back as a very serious competitor that is there to win fairly.

landarc
12-18-2009, 09:24 PM
Getting certified to be a BJCP judge is extremely difficult, and you have to be able to identify all of the recognized styles of beers, by taste and description. A lot more than 4 meats, for certain. This does not make the judging any less controversial when someone gets lower scores than they expect. The one KCBS judge I know locally has quit, as he became fed up with the lack of thought and carefree attitudes he saw as a judge.

Rich Parker
12-19-2009, 09:37 AM
My feeling is that the difference between a judge who took a class and one that hasn't is not significant. The criteria to be a BBQ judge is ridiculous. One day where you are taught how to fill out a form and told that ribs shouldn't fall off the bone - that's hardly training. To become a Certified Beer Judge you have to take a series of classes and then pass a difficult test. Plenty of people are willing to go through the hassle of getting certified. And the reality is that it costs less than $100 to brew a beer that will be judged. It can cost several hundreds to participate in a BBQ contest. Doesn't it make sense that people who spend that kind of money have judges who have REAL training?

BYW, I'm judge # (oh well, I forgot)

I was told that comps already have a hard time getting 100% CBJ participation so wouldn't more classes and more money make that even harder?

QansasjayhawQ
12-19-2009, 10:23 AM
#1 the KCBS has no control over selection of judges. The Jack is a good example. Good cooks still win with celebs. If we tell the organizer they need to have all CBJ's then the opranizer says I'll run it without you unless you provide the judges I need. Not going to happen.

#2 there are a lot of cbj's out there myself included that have not judged a contest in years but we're still cbj's. Doesn't make us better judges.

#3 I know a lot of cbj's that smoke as in cigars, etc. Now I would far prefer a non smoking non certified judge to taste my entry. Not cutting at smokers here as I enjoy cigars at contests but fdact is after a couple of cigars my taste buds are messed up. That's why I like having a program and sticking to it every time. I don't need to taste for the best slab of ribs.

#4 new cbj's that took the class the night before are often tough scorers. But others are not. there's no formula to apply here
I've also thought that there's another way that judges could be tracked.

If a judge volunteers to judge - and then doesn't show up - that should be kept track of so that organizers have an idea of how many judges are likely to show up.

Tracking a judge to see if it's been a couple of years since they judged would be another way to ensure that the pool of judges is better. If a judge hasn't judged in a couple of years, they should have to take a refresher course, or take the judging class all over again.

QansasjayhawQ
12-19-2009, 10:30 AM
Thirty contests, plus cooking with a team -- invaluable! -- and I'm not certain of the testing involved. In any event, it's not a cakewalk.
Yes - and there is a written test. I haven't seen the test - only heard about it from some other people judging last year as they hit their Master level.

I can say that the skills of a judge that has judged less than about 10 contests are radically different than they are of a judge who has judged several more.

I think that judges are necessary and that it's a good system of judging that we (the KCBS) has created. However, I also see a HUGE number of opportunities for improvement and refinement.

Thanks to everyone who has thrown in their comments here - I really appreciate them.

P.S. The efforts that I see to improve the judging are mostly coming from the serious, experienced judges.

CajunSmoker
12-19-2009, 11:19 AM
I personally would prefer my box to hit a table of newer judges than one with supposedly "experienced judges" who think they know everything there is to know about BBQ. Some of the opinions I've heard in the judges tent make me cringe when I turn a box in. One that sticks out in my mind was from a judge with 20+ contests who said before the brisket turn ins started "Well it's time for brisket next. I just hate brisket":!:

BigJimsBBQ
12-19-2009, 11:31 AM
I personally would prefer my box to hit a table of newer judges than one with supposedly "experienced judges" who think they know everything there is to know about BBQ. Some of the opinions I've heard in the judges tent make me cringe when I turn a box in. One that sticks out in my mind was from a judge with 20+ contests who said before the brisket turn ins started "Well it's time for brisket next. I just hate brisket":!:

Roger - I 2nd that Opinion.

QansasjayhawQ
12-19-2009, 04:34 PM
I personally would prefer my box to hit a table of newer judges than one with supposedly "experienced judges" who think they know everything there is to know about BBQ. Some of the opinions I've heard in the judges tent make me cringe when I turn a box in. One that sticks out in my mind was from a judge with 20+ contests who said before the brisket turn ins started "Well it's time for brisket next. I just hate brisket":!:
Well, that's one judge . . . that doesn't mean that ALL judges are that way.

I can tell you that the vast majority of judges that I know are open minded and certainly not picky about what they like or don't like. Most of the judges I know are aware that, one day, pork with white sauce may hit their table and they have to be ready to judge that entry on its own merits - while keeping their own preferences for KC style (or whatever) in check.

Heck, I LOVE hot, spicy foods! But if someone has created an excellent work of BBQ art with no heat at all in it, I will NOT score it down. I am judging that BBQ on its own merits - not comparing it to what I would personally prefer.

Good judges practice objectivity.

Non-certified and inexperienced judges lack that practice of objectivity and it seems like there should be a way to accommodate for that.

monty3777
12-19-2009, 05:36 PM
I was told that comps already have a hard time getting 100% CBJ participation so wouldn't more classes and more money make that even harder?

I suppose so - but with all that work there are tons of people who want to put forth the effort to be beer judges. And because of that training the beer judges are taken very seriously in the comp beer world!

The other big difference is that each entry is given comments by the judge that help the competetor better understand scores. That just seems logical. I think the reason that doesn't happen in KCBS is that cooks would explode if they got a 7 on taste because "it ain't what I like." Better to leave it all a mystery.

monty3777
12-19-2009, 05:39 PM
case in point: (this is a link to another forum. Please come right back!! :)

http://www.rbjb.com/rbjb/rbjbboard/messages/730928.html

ZILLA
12-19-2009, 05:41 PM
And in my opinion, they are more than "necessary" -- they are the reason I will go to a particular contest, or not.

Yes, good BBQ will rise to the top. We've been fortunate enough to cook well across two sanctioning bodies this past year. But the fact remains that a team must cook differently to please "judges" from the general public who think boiled Chili's and mushy McRibs are the bomb, than they do for a Master CBJ.

Contests/organizers that make lackluster or nonexistent efforts to recruit experienced judges will not see me return. Contests that rely on holding a judging school the night before a contest to make up their "100% CBJs" will not be at the top of my list of places I want to compete again.

Too much goes into our product to not care whether any attempt has been made to educate the judges to a common standard. Do teams deserve something more? Yes, and that's why we value experienced judges so highly. Do I think it could be better than that? Absolutely, through apprenticeship and continuing education.


I'd rather have my BBQ judged like a Tennis game rather than like an Olympic Ice Skating routine. With judges that call the boundrys and fouls and that's it! The other thread sheds light on what I mean.

QansasjayhawQ
12-19-2009, 05:48 PM
I suppose so - but with all that work there are tons of people who want to put forth the effort to be beer judges. And because of that training the beer judges are taken very seriously in the comp beer world!

The other big difference is that each entry is given comments by the judge that help the competetor better understand scores. That just seems logical. I think the reason that doesn't happen in KCBS is that cooks would explode if they got a 7 on taste because "it ain't what I like." Better to leave it all a mystery.
In 2009 the KCBS introduced Comment Cards on each table for just such an occasion.

The judges have been instructed all year long to NOT leave vague, general comments such as "it ain't what I like". The comments on the cards are to be specific comments on the details with the intent that the comments are to help the teams improve their art.

The comment cards are separate from the scoring card.

I liked the fact that we started out leaving comments on the back of the scoring cards but that those comments were separated out this year. I hope the KCBS continues this.

Falcon 83
12-20-2009, 08:19 AM
This is my first post or comment so forgive mistakes and I hope I don't offend anyone.

I just became a KCBS certified master judge. The requirements are judging 30 KCBS sanctioned events, cook with a team and pass a 51 question written test with a score of 90% or better. Some of the questions are true/false, some are multiple choice and some are essay. Believe me its not easy.

I'm also a certified table captain. Part of my duties are to check each score sheet at my table for completeness before giving them to the rep. I can also see the scores from the various judges. Relative to the discussion about new judges certified the night before, I can honestly say from my experience that some score hard, some score in the middle and some score easy. The same can be said for master judges.

Previous comments above relative to some kind of parity would seem to me to be labor and cost prohibitive. KCBS would have to pass these costs on to the organizers and they would add these to entry fees for the cooking teams. Judges do receive recurrent training at each competition as at the judges meeting, we listen to a 10 minute CD recapping the scoring system and the judging particulars for each catagory. If judges are required to much more training periodically, they're going to want to be paid; again adding to your entry fee. This would bring in judges who are only in it for the money. At least now, the volunteer judges are there for the love of BBQ.

I have two suggestions to keep things simple for the time being until some more elaborate system can be devised and acceptable to all parties involved.

When judges are walking around the comp area after the close of scoring, invite them in, explain your procedures, answer their questions and invite them to cook with you at a future comp, even if they are not ready for master judge certification. Each little bit of training will help. I know I'm a better judge after cooking with a team.

Second, the comment slip program is already in service, although it is voluntary on the judges part. I try to submit a slip for any score 5 or below; my philosophy being if I can help you make better BBQ, it will benefit my stomach at a future comp.

The rules committe is meeting in Phila. next month. Get your e-mails into them asking for mandatory comment slips for a score of 5 or below. At least then, if a judge gives a low score, you'll know why. This pressure is going to have to come from the cooking teams; volunteer judges are not going to ask for more work.

I've sai enough for now. Thanks.

Rich Parker
12-20-2009, 08:43 AM
I suppose so - but with all that work there are tons of people who want to put forth the effort to be beer judges. And because of that training the beer judges are taken very seriously in the comp beer world!

The other big difference is that each entry is given comments by the judge that help the competetor better understand scores. That just seems logical. I think the reason that doesn't happen in KCBS is that cooks would explode if they got a 7 on taste because "it ain't what I like." Better to leave it all a mystery.

Don't get me wrong - I think it would be awesome if they offered that type of training for judges so then you would only have qualified judges. I just think KCBS is to big to go back and change that.

QansasjayhawQ
12-20-2009, 10:23 AM
This is my first post or comment so forgive mistakes and I hope I don't offend anyone.

I just became a KCBS certified master judge. The requirements are judging 30 KCBS sanctioned events, cook with a team and pass a 51 question written test with a score of 90% or better. Some of the questions are true/false, some are multiple choice and some are essay. Believe me its not easy.

I'm also a certified table captain. Part of my duties are to check each score sheet at my table for completeness before giving them to the rep. I can also see the scores from the various judges. Relative to the discussion about new judges certified the night before, I can honestly say from my experience that some score hard, some score in the middle and some score easy. The same can be said for master judges.

Previous comments above relative to some kind of parity would seem to me to be labor and cost prohibitive. KCBS would have to pass these costs on to the organizers and they would add these to entry fees for the cooking teams. Judges do receive recurrent training at each competition as at the judges meeting, we listen to a 10 minute CD recapping the scoring system and the judging particulars for each catagory. If judges are required to much more training periodically, they're going to want to be paid; again adding to your entry fee. This would bring in judges who are only in it for the money. At least now, the volunteer judges are there for the love of BBQ.

I have two suggestions to keep things simple for the time being until some more elaborate system can be devised and acceptable to all parties involved.

When judges are walking around the comp area after the close of scoring, invite them in, explain your procedures, answer their questions and invite them to cook with you at a future comp, even if they are not ready for master judge certification. Each little bit of training will help. I know I'm a better judge after cooking with a team.

Second, the comment slip program is already in service, although it is voluntary on the judges part. I try to submit a slip for any score 5 or below; my philosophy being if I can help you make better BBQ, it will benefit my stomach at a future comp.

The rules committe is meeting in Phila. next month. Get your e-mails into them asking for mandatory comment slips for a score of 5 or below. At least then, if a judge gives a low score, you'll know why. This pressure is going to have to come from the cooking teams; volunteer judges are not going to ask for more work.

I've sai enough for now. Thanks.
Welcome to the friendliest most helpful forum I've ever been a part of on the Internet, Falcon. Please be sure to stop by Cattle Call and introduce yourself.

How do you envision parity values being assigned to a judge as being cost prohibitive?

If I were required to take an on-going recertification class, that would be fine with me. Heck, it might even cull out the recreational judges. Myself and many other CBJs consider ourselves professional and take our volunteer responsibilities very seriously. It's true that we do get that ten minute reminder before we start, but I think that the most training comes from actually judging. Those who have judged less than 10 contests have far less of a clue as to what is going on than those who have judged 20 or 30.

I think that it's not fair to the cooks (which includes myself - I also cook on two teams now) that a rookie judge's score counts for just as much as a Master judge's score. I've been around these judges and I can say that the experienced judges who are professional about their duties score much more accurately and fairly than most rookie judges. That's why I started the discussion about judging parity.

I do like your idea about a comment card being required for any score 5 or below. There are a few teams out there that need the help and if I ever hit a 5 in my results, I would REALLY like to know what I could do to improve it.

Thanks for your comments!

Falcon 83
12-20-2009, 06:54 PM
Hello QansasjayhawQ,
I think the cost for the parity of judges would have to come from the time and research to establish the program, keeping track of each judge's value, and plugging it into the scores for each competition. Each competition would have a different field of judges so it would take extra time for the Rep to establish the computer scoring program for a particular contest. Last minute substitution of a few judges would also add to the Rep's pre-comp preparation. Regional records couldn't be kept and used as I have judged comps in 10 different state ranging from New York to North Carolina and two trips to Kansas for The Great American comp.

Then there would be the upgrading of each judge's value as he judge's more contests. Can anyone say that two judges who have judged 20 contests have the same parity value? Does a judge who judges 10 comps in 5 years deserve the same value as a judge who does 10 comps in one year? Does a newer master judge deserve a higher value than an older master judge who attained this position before the rules were added about cooking with a team and passing a written test? This list of situations could go on forever. Evaluation tests would have to be given on a regular basis to determine if a judge deserves and receives a higher value.

In closing this part of my response to your honest question, I think KCBS would have to hire more staff and supply another Rep or two to each comp for the added paperwork.

And finally, I do take my judging seriously, as seriously as your comments imply you do. Yes, there are judges I would like to see "fired," especially those filling their take home cooler while I'm still scoring my 3rd sample (my personal pet peeve). I have asked discrete questions of several Reps about just this subject but there is no established procedure for firing a judge unless it is for serious rule infractions, etc.

Somebody is going to have to build the better mouse trap. Until then, I'll judge the best I can and enjoy the BBQ.