PDA

View Full Version : I gotta complain just a bit here......


abangs
06-12-2011, 09:23 PM
I've been competing for 2 1/2 years. This year we have completed 4 events and done fairly well. I completed the KCBS judges class and judged one event recently. I have always been puzzled at the scores. For example, my chicken scores from an event where I finished 17th out of 85 in chicken: J1 988, J2 966, J3 999, J4 787, J5 999, J6 988. Score order is appearance, taste, tenderness. My first puzzling issue is how can all judges think the appearance is a 9 except for one prick judge that throws a 7. The second puzzling issue is how can the taste and tenderness go 8's and 9's and one throws 6's??

Well, the short of my recent judging experience is this (and maybe this was just an anomoly): In speaking with other judges, I was SHOCKED at how many do not compete and have never competed. This explains the one prick in each category that scores so much lower than the others. That prick has no idea what goes into those boxes. You would think that something has got to be wrong with you if your opinions are that far from the others.

I think it should be a requirement that all certified judges (any sanctioning body) should be required to cook at least one event in entirety in order to judge so that when he gives a 7 in appearance where the others gave 9's, then maybe, just maybe he'll think twice and score it at least an 8. Same goes for the prick that gave me 6's in t&t where the others were 8's & 9's. Thank goodness for KCBS dropping the lowest score.

Sorry for the rant, but if your a certified judge, and your one of these guys..do the rest of us cook teams a favor and go cook an event.

BBQ Grail
06-12-2011, 09:32 PM
If you made that requirement you would, effectively, eliminate enough judges so you wouldn't have competitions.

You don't need to have judges who have cooked an event. You don't need to have cooked a competition to follow the guidelines.

Rookie'48
06-13-2011, 12:48 AM
Andy, I'm in total agreement on having judges cook with a team before they can judge their 5th, 10th, (pick a number) whatever contest. I really think that the judges who have never cooked a comp have very little, if any, idea of what comp cooking is.
However, the CBJ program is a major source of income for the sanctioning body, so .....

If you made that requirement you would, effectively, eliminate enough judges so you wouldn't have competitions

BBQchef33
06-13-2011, 01:29 AM
If you made that requirement you would, effectively, eliminate enough judges so you wouldn't have competitions.



Would be pretty sad if that was true.. That would say that some judges wouldnt be willing to cook a contest with a team and instead would just drop out of the judging pool. Regarding those that do drop out, I would draw the conclusion that they weren't serious enough about it and was just there for free food. I would think(ok, hope).. that is NOT the majority.

Hub
06-13-2011, 07:32 AM
I fully support the requirement that judges participate with a team at least once to learn what it is like (I also cook) BUT:

Your assumption that the lower scores came from a judge who has never or not recently participated with a team is not necessarily correct. In general, I find that judges who are also cooks tend to be more critical and harder to please than non-cook judges. Last weekend I sat next to a master judge who was a long time cook, too. He was a nice guy and we talked a lot and compred notes about our findings following the categories. We tended to think a lot alike, but his scores were significantly lower than mine on some entries that I thought were quite good. That's his opinion and he's not "wrong". Just tough.

Judging will always have a large subjective component to it -- if it was "cut and dried" then a computer could do it :doh:

I understand and appreciate your thoughts, but those lower scores might have come from a judge who also cooks.

Lakeside Smoker
06-13-2011, 07:46 AM
That prick has no idea what goes into those boxes.

Sorry, but what does that matter? We're all out in a wet field cooking overnight and it's a lot of work. I agree with that. But every competitor has the same conditions so why judge me on how hard this hobby is? Just judge the food.

Everyone has different tastes, it's human nature. I don't think calling a judge a prick will change that.

-Mike

ssbbqguy
06-13-2011, 07:57 AM
Starting at the six number is probably where your problems arise.Old classes tought starting higher and couting down. Now start a six and move.What might be thrown into this thought is, there are alot of good cooks out there now compared to several years ago. As a long time cook and CMJ, I am probably tougher on little things and more critical than someone that hasn't even been to to a cooksite, let alone cooked. I am a firm believer in a better continuing education program. Our current KCBS board hasn't grasped this idea with enough passion in my opinion. Steve.

NRA4Life
06-13-2011, 07:58 AM
That prick may have bit into a piece of cartiledge that you missed when you were trimming the chicken. A chewy piece of cartiledge would impact my tenderness scores if I were judging.

deepsouth
06-13-2011, 08:04 AM
is judging bbq without every having cooked "competition bbq" similar to judging a beer without having ever brewed a beer or critiquing an album without ever haven't written a piece of music?

Jorge
06-13-2011, 08:26 AM
If you made that requirement you would, effectively, eliminate enough judges so you wouldn't have competitions.


I doubt it. CBJs are nice, but not necessary to have a contest.

The_Kapn
06-13-2011, 08:40 AM
Take a look at the many "rate my ...." threads here.

The Brothers are all well meaning, but the scores are all over the place.
No different than at a contest, sadly.
If you read the critiques carefully, many of the reviews seem to be more in the line of "what I would do to make the box better".
Also, a lot of speculation about tenderness and taste based on appearance.

As to cooking experience, I do not need a judge giving me what I call "pity points' due to the effort or money I have invested.
I want the judges to just judge my meat as it is.

I have seen the most critical judges to be the Master and other experienced judges and those who cook a lot also.
The seem to have developed very specific criteria that they are looking for, and if you don't hit it, you pay on the score sheet.

Until we judges are held accountable for our scoring that is way out of line, nothing will change.

Just my thoughts, FWIW.

TIM

Fatback Joe
06-13-2011, 08:48 AM
The second puzzling issue is how can the taste and tenderness go 8's and 9's and one throws 6's??



IMO taste & tenderness can easily be all over the board on chicken. Assuming you did thighs and put 6 in the box, it is most likely from 6 different chickens, 3 different ones for sure. Very possible that they did not cook up the same when you have different animals in the mix. Last comp I did, my wife tried a thigh that was spot on..........I tried a thigh that was kind of rubbery. They were all handled the same way, similar size, etc, but the one I had would not have scored well.

Harder to explain/understand if looking at ribs that came all from one rack, slices from all from one brisket, etc. But chicken, seems really easy to have that all over the board IMO.

Now big variations in appearance always make me go :confused:.

Ford
06-13-2011, 11:06 AM
Being a CBJ means you paid the KCBS money. Not much else to it. I have many good friends that are CBJ's and they'd be good judges with or without the class. Most of them are at least backyard cooks.

Having said this much I really don't care if judges are CBJ's or off the street. I've seen both and had good and bad scores. My biggest worry is hearing that they taught a CBJ class the week before and all those folks get to judge. They scare me. In Texas they don't have certified judges and it seems to work pretty darn good.

And I really don't think the word "prick" should be applied to a person or group of people just because you don't agree with them.

MoKanMeathead
06-13-2011, 01:55 PM
I completely agree with Ford on this one. The brand new CBJ and the overly critical CBJ/CMJ scare me the most. I would rather cook for someone off the street than either of these guys.

A varience from 9 to a 7 is not that much IMO. A 9 to 6 is a little harder to understand but with taste and tenderness its all about personal tastes much of the time. I judged a couple of weeks ago and tasted one pork entry that I thought was outstanding (999) afterwards I mentioned it to another judge (someone who has judged almost 30 contests) and he thought it was mushy! - same entry from the same muscle. Not sure what he gave it for a score but if he thought is was mushy I'm sure it was not more than a 7 - and probably a 6.

I also am not sure about the need for your adjectives.

Greendriver
06-13-2011, 02:08 PM
I agree with the Kapn 100% on this issue and I'm 100% against on site training with judges cooking with teams, as I don't see the need for it. Simply letting the table cap't point out the discrepancies should help IMHO.

Alexa RnQ
06-13-2011, 02:13 PM
The brand new CBJ and the overly critical CBJ/CMJ scare me the most.

Me three.

And in addition, I am not happy when our scores come back and reflect that there were TWO obviously out-of-range judges at one table. The system accounts for one by tossing a low score, but when you have two on the table you're screwed. Whether it's the result of an organizer seating too many VIPs or a rep that doesn't sort judges by experience among the tables, it shouldn't happen -- but it does, and we're all equally at risk.

Name-calling doesn't do anything to improve the situation. Tracking judges will -- accountability will. As we see on the Internet, people think twice when they have to put their real name on what they write.

BBQ Bandit
06-13-2011, 02:22 PM
That may happen...

The last I heard... to garner the Master CBJ badge... one must be part of a team during at least one KCBS event and have 30 rep/event signatures.

I'm the unusual one.. do judge and assist teams (not at same event)... without racing for the Master CBJ badge.

[Badges... we don't need no steenkin' badges.... Blazin' Saddles mod]

billygbob
06-13-2011, 02:50 PM
Not to hijack the thread but its as good a place to ask this question as any - at what point do/should "9's" start becoming "6's"? Six is average. If everyone has a muffin-top chicken thigh slathered with BH and honey, and you've seen 110 of them out of the last 120 chicken entries, does that, by definition, not become "average"? Doesn't matter if'n you like them or not, if every one looks the same can they all be a nine? Maybe it becomes drudgery and not so appetizing?

Just wondering how cooks think the word should turn.

boogiesnap
06-13-2011, 02:58 PM
this is a valid point.

BUT,

entries are judged individually, NOT comparatively.

Alexa RnQ
06-13-2011, 02:58 PM
Maybe it becomes drudgery and not so appetizing?

If a person finds it drudgery, they should not be at the table.

abangs
06-13-2011, 03:29 PM
Ok. In reading all of these responses, it boils down to this: I should not have used the word "prick" and I apologize....and It's one big crap shoot on the judging!! I still would like to see some accountability for scores that are far off from the others...maybe require the judge to fill out a quick comment card if the score is much lower than the others.....for example if all of the T&T scores are 8's and 9's and one throws 6's....I think that would be fair. On the other side of the coin, if all judges throw 6's and 7's then I think those scores would speak for themselves.

Texana
06-13-2011, 03:42 PM
....... In Texas they don't have certified judges and it seems to work pretty darn good.......

This Texas competitor wants to say thank you Ford .....

Texana
06-13-2011, 03:45 PM
this is a valid point.

BUT,

entries are judged individually, NOT comparatively.

I agree with this in theory ... but not in practicality .... It is the rare judge that can not judge one piece of meat from the one he/she just judged .... and I feel that applies to CBJ's as well as our Texas off the street judges .... people like what they like and in the end if it's not up to their standards the scores reflect it .....

I cant explain it but thats just the way it is ...

Texana
06-13-2011, 03:49 PM
Ok. In reading all of these responses, it boils down to this: I should not have used the word "prick" and I apologize....and It's one big crap shoot on the judging!! I still would like to see some accountability for scores that are far off from the others...maybe require the judge to fill out a quick comment card if the score is much lower than the others.....for example if all of the T&T scores are 8's and 9's and one throws 6's....I think that would be fair. On the other side of the coin, if all judges throw 6's and 7's then I think those scores would speak for themselves.

I think that your idea is excessive ..... I mean after all if you make him fill out a comment card and you dont agree with his comments what are you going to do next .... have him shot?

It's bbq competition people. It is not a competition for a Nobel Peace prize ..... we have all recvd good scores for bad product and I dont see any bitching about that .... so it only stands to reason we are going to get bad scores on occasion for good product ..... in the end it's a wash ......

The good new is this ... there will be another event next weekend and we get to do it all over again ..... besides if was easy every one would do it and where would the fun be then ..... ????

huminie
06-13-2011, 04:01 PM
I try not to get too wound up about scoring discrepancies. I focus my energy on trying to cook better each contest. At least that is something I can control.

Jorge
06-13-2011, 04:07 PM
This Texas competitor wants to say thank you Ford .....

I alluded to that. Guess I'm already branded as a renegade;)


Sent from my iPhone using Tapatalk

Harbormaster
06-13-2011, 04:24 PM
If you want to see a score that reflects an out of touch judge, you need to check out my thread here (http://www.bbq-brethren.com/forum/showthread.php?t=108582) if you haven't already.

Sledneck
06-13-2011, 05:04 PM
I think it should be a requirement that all certified judges (any sanctioning body) should be required to cook at least one event in entirety in order to judge so that when he gives a 7 in appearance where the others gave 9's, then maybe, just maybe he'll think twice and score it at least an 8. Same goes for the prick that gave me 6's in t&t where the others were 8's & 9's. Thank goodness for KCBS dropping the lowest score.

Sorry for the rant, but if your a certified judge, and your one of these guys..do the rest of us cook teams a favor and go cook an event. I'm going with the flip side. how do you know the "prick" wasn't a reg comp cook? I don't judge but would it be crazy to think that a cook with be more critical of your food? Or perhaps disgruntled cook who has the " my food is way better than this stuff" attitude. Many, many variables, palates etc. Making a judge cook as a requirement imho will not make a much of a difference. They are not all ignorant, im sure theyve seen, read and heard what we do.

boogiesnap
06-13-2011, 08:15 PM
I agree with this in theory ... but not in practicality .... It is the rare judge that can not judge one piece of meat from the one he/she just judged .... and I feel that applies to CBJ's as well as our Texas off the street judges .... people like what they like and in the end if it's not up to their standards the scores reflect it .....

I cant explain it but thats just the way it is ...

the end of your post sorta proves my point.

it's either good or not, regardless of what else you've tried.

motoeric
06-13-2011, 09:01 PM
The only benefit I can see in having a judge cook with a team is that he will be sympathetic and appreciative of the work and expense that goes into competing.

Can you see any other benefit?

If that is the only benefit, I personally wouldn't want to see a 7 become a 9 due to sympathy and appreciation for effort.


Eric

monty3777
06-13-2011, 09:03 PM
If KCBS wants a decent judging program that respects the time and money invested by competitors they would do well to emulate the Beer Judge Certification Program. Allow me to also restate that the notion that judges should 't talk to one another while judging is ridiculous. If there were some programmed interaction between judges before the final score is registered, as is the case in beer competitions, then the strange 6s amongst a bevy of 9s would be absolutely eliminated.

monty3777
06-13-2011, 09:14 PM
If that is the only benefit, I personally wouldn't want to see a 7 become a 9 due to sympathy and appreciation for effort.


Eric

I'll take "sympathy" 9s all day long :becky::thumb:

CTSmokehouse
06-13-2011, 09:34 PM
The only benefit I can see in having a judge cook with a team is that he will be sympathetic and appreciative of the work and expense that goes into competing.

Can you see any other benefit?

If that is the only benefit, I personally wouldn't want to see a 7 become a 9 due to sympathy and appreciation for effort.


Eric

I look forward to tasting your entries at the BBQ Brethren Battle of the BBQ Brethen in LI in Manorville....

Yours in BBQ,

Cliff

Capn Kev
06-14-2011, 05:57 AM
I'm not necessarily in agreement that a CBJ should have to cook with a team, but I am totally in favor of tracking individual judge's scores in a database and analyzing their trends. We all know there are certain judges that seldom give out a score above a 7 regardless of how good they think it is, even if everyone else at the table is handing out 9s. Sussing out those judges, and either re-educating them or getting them out of the pool would be in the best interest of everyone involved.

Lake Dogs
06-14-2011, 07:12 AM
> If you read the critiques carefully, many of the reviews seem to be more in the line
> of "what I would do to make the box better".

Kapn, I think this is the brethren/judge trying to help the cook. Otherwise, for appearance, the answer is either "Yes, darned appetizing", or "No, not appetizing". The "No, not appetizing" doesn't help without articulating why and what might be done to improve.

> Also, a lot of speculation about tenderness and taste based on appearance.

See above. If it looks burned, it might not be burned, but it could be. Either way, if it "looks" burned it's therefore less appetizing. The person was simply trying to explain why it was less appetizing. Same goes for dry (wow, thousands of comments on brisket here), or over-cooked (a brisket falling apart), or under-cooked (in our recent pork debate), etc. Otherwise they'd simply say "it doesnt look appetizing to me", and that's not very helpful, is it?


To abangs, I know it's extremely frustrating to get scores all over the place. I am however of the opinion that 7's-9's isn't all over the place, but 6's-9's is. However, it happens, frequently too. Part of it is that different pieces look, taste, and are different even in feel/moisture. Also, beauty truly is in the eye of the beholder. I've seen money muscles displayed so that they look like a penis; something like this gets scores from 5's to 9's. Did the judges giving the 9's not see it, or did they see it and have a sense of humor, or did they see it and think "I want me some of that"? I dont know the answer... I've seen pork with brown sauce put in 6 very neat piles. Half of the judges loved it, the other half though they looked like something the cat yak'd up. I had one where I was that lone judge. They opened the box and I (apparently I was the only one) had to do a double take and look VERY hard. They didnt look like chicken; they looked like potatoes, literally.

Anyway, my point is this; beauty is in the eye of the beholder. It's subjective, more than we'd like to think.

Sawdustguy
06-14-2011, 09:14 AM
I think it should be a requirement that all certified judges (any sanctioning body) should be required to cook at least one event in entirety in order to judge so that when he gives a 7 in appearance where the others gave 9's, then maybe, just maybe he'll think twice and score it at least an 8. Same goes for the prick that gave me 6's in t&t where the others were 8's & 9's. Thank goodness for KCBS dropping the lowest score.


I am going to be totally honest. Judges have different opinions about what is good and what is not good. No matter what the spread of scores it seems that the cream always rises to the top and the best team wins. Thats why teams like Pellet Envy, Quau Lotta Bull etc. always seem to win.

Making it a requirement for a judge to cook a contest is rediculous. You decided to cook BBQ competitively and spent the money and time to do so. I personally see no reason why a judge should be forced to cook a contest because you don't like your scores.

I think that if you can't handle the scores that you get at a competition and choose to disparage the judges you should choose another hobby.

bover
06-14-2011, 09:33 AM
I'm not necessarily in agreement that a CBJ should have to cook with a team, but I am totally in favor of tracking individual judge's scores in a database and analyzing their trends. We all know there are certain judges that seldom give out a score above a 7 regardless of how good they think it is, even if everyone else at the table is handing out 9s. Sussing out those judges, and either re-educating them or getting them out of the pool would be in the best interest of everyone involved.

Rumor has it that the new scoring software currently contracted out for development by KCBS will have some sort judge tracking component. What all that involves is anyone's guess, but my hope is that it's exactly what you just stated. In conjunction with that I'm of the opinion that there should definitely be some continuing education program for all CBJs. Even just a simple online rules and procedures document that you have to acknowledge as having read would be beneficial. There are a lot of judges out there still operating under the "old" method of scoring and lots operating under the "new". In order to maintain the program's integrity, some consistency would really be nice.

That being said, there will always be cooks that think their scores are too low (ever hear of a cook complaining when their scores were actually too high? it happens, but no mention of it) and there will always be judges that don't adhere to the procedures fully. All we can do is try to reduce the frequency of those occurrences and I believe more education and patience on both sides is the best way to do that.

big brother smoke
06-14-2011, 09:33 AM
let's not rake the brother over the coals for venting out loud here :becky:

Muzzlebrake
06-14-2011, 09:33 AM
No matter what the spread of scores it seems that the cream always rises to the top and the best team wins. Thats why teams like Pellet Envy, Quau Lotta Bull etc. always seem to win.


and they also get out of wack scores from time to time. Scottie from CSC just commented on how he got a 4 at the Sam's Club event (http://www.bbq-brethren.com/forum/showthread.php?t=108600)


I do like the Texan's idea of shooting the low scoring judge though!:twisted: In the Northeast we have much more stringent gin control laws, so we may need to have them shackled and stoned on the town square but same general idea..........

Capn Kev
06-14-2011, 09:50 AM
No matter what the spread of scores it seems that the cream always rises to the top and the best team wins. Thats why teams like Pellet Envy, Quau Lotta Bull etc. always seem to win.


To some extent yes. But when consistent winners like the names you mention above get whacked by a "rogue" judge, do you think they're happy with it? This last weekend Johnny Trigg took 41st out of 47 teams in Ribs. Something tells me that the cream didn't quite make it to the top that day. I'm sure if you ask Alexa from R&Q about some their scores this past weekend, I'm sure she will have some categories that were a bit jacked up. Talent does rise, but one "rogue" judge can cost a team thousands of dollars.

Carnivorous Endeavors BBQ
06-14-2011, 09:57 AM
It was mentioned earlier but i'll state it again. When a judge is obviously way off base with the rest of the table, It should be the Table Captains responsibility to inquire as to why they scored that way. On appearance i think it is more simple, if you have all 9's and one 7 then it should be easy to resolve. Taste and texture are more difficult as they could have gotten one bad piece of chicken or one bad piece of another meat which can easily be explained and accepted. If KCBS does move forward with tracking judges scoring, I think it will point out some simply harsh overly critical judges. It does make me wonder though, what is a 999 to these 766 judges?

Alexa RnQ
06-14-2011, 10:08 AM
I'm sure if you ask Alexa from R&Q about some their scores this past weekend, I'm sure she will have some categories that were a bit jacked up.
Wait, what? You mean the ones that went 999, 999, 988, 889, 667, 778?
And we got two comment cards for that category, that both indicated were from NON-CBJs.

Capn Kev
06-14-2011, 10:14 AM
We were right there with you Alexa ;) We drove home scratching our heads. Those scores look almost identical to some of our scores from Pueblo :doh:

QansasjayhawQ
06-14-2011, 10:16 AM
. . . Until we judges are held accountable for our scoring that is way out of line, nothing will change.

Just my thoughts, FWIW.
I think that's an excellent thought because I've had similar thoughts.

There is A LOT of data that is captured on each individual judge at each competition.

The judges who are consistently judging 'outside' the norm could be directed to retrain . . . for example.

OR - a judge who scores outside the norm could be avoided (culled from the pool) when organizers are reviewing prospective judges for an upcoming event.

But I suppose dropping the low score leads to the same effect and keeps the amount of data entry on site to a minimum.

goodbuddiesbbq
06-14-2011, 10:24 AM
I'm not a big "these judges don't know what they're talking about" kind of person, because for the most part they seem to be pretty fair (for the most part).

With that being said, all sports officials from recreational fields to high school athletics to college to major sports must have annual assessments to move up the ranks. That requires a "senior" official reviewing your performance and talking you through some of the points as to what they assessed. Judges should be assessed and if they are not interested in growing...they should be used for a contest when there is not another more qualified judge around.

This is exactly how they do it for most athletic events...they want to get the best possible people, that have proven experience, the most opportunity so that the results are skewed the least. Everybody should want to be better, just like most q'rs want to get better.

This is how they weed out those that are just there to be there.

CajunSmoker
06-14-2011, 10:48 AM
My cbj instructor made a statement at the end of our class that stuck with me. He said in effect "if you go in looking to find bad BBQ you will. If you look for good BBQ you will find that. Let's all go find the good BBQ." I liked that way of thinking and have always looked for the good in every entry I judged. Didn't always find it, but I always looked.

Lake Dogs
06-14-2011, 10:58 AM
Wait, what? You mean the ones that went 999, 999, 988, 889, 667, 778?
And we got two comment cards for that category, that both indicated were from NON-CBJs.

At some point in time, some people just should not be judging. 667 to 999? That's way off in everything! It looks to me like you got one or two folks who just dont like barbecue, and shouldn't judge.

Curious, what meat, and what were the comments?

motoeric
06-14-2011, 11:10 AM
I look forward to tasting your entries at the BBQ Brethren Battle of the BBQ Brethen in LI in Manorville....

Yours in BBQ,

Cliff

Feel free to stop by after the judging! Team Grill Scouts.

Eric

CBQ
06-14-2011, 11:44 AM
Sorry, but what does that matter? We're all out in a wet field cooking overnight and it's a lot of work.

A very, very wet field last weekend. :becky:

I judged the Anything Butt category at Cape Cod because I was doing only the KCBS contest. After judging, we discussed what we liked and disliked. I, and the three judges to the right of me, all picked the same dish as the worst one. The guy on my left said the same dish was has favorite.

There is a subjective element to judging, but we all have to deal with that, even the mysterious 9 9 8 6 6 9 on appearance.

Slamdunkpro
06-14-2011, 11:57 AM
At some point in time, some people just should not be judging. 667 to 999? That's way off in everything! It looks to me like you got one or two folks who just dont like barbecue, and shouldn't judge.

Curious, what meat, and what were the comments?

I'll throw a little gas on this.

It's always the 667 judge that gets the hit for being out of whack, in the example you cited you had 999, 999, 988, 889, 667, 778. Who's to say the the 2 999 judges weren't the ones out of line? When I table captain or tabulate scores I've observed that new judge's scores trend higher than more experienced judges, and that judges who tend to be the most critical are cooks.

abangs
06-14-2011, 01:01 PM
Intersting point of view. Never thought of it this way. I can see this argument, however, the contest rep will not put that many newbie judges on a table will they???? Unless there is a dire shortage of judges? On these scores, I would say that at a minimum J5 should at least fill out a card for the team......what team out there competiting would NOT WANT to know why this judge gave them a 6 in appearance when the others threw higher scores????? I do not think this would be too much to ask from the judges. It would take that guy a whole 30 seconds to scetch something down for the team that turned the box in. If I received these scores, and that judge gave me a comment on "hey, i thought this looked xxxxx is why I scored a 6 and I thought the taste was xxxx and I my piece was tough" then I might not like it, but at lease I would have the satisfaction and basis of something to improve upon.

I'm glad I started this thread. In all honesty, all of our/your points are good and all valid. I think we could all argue this endlesly!!!! I've cooled down considerably since orignally posting this because from looking at some of these other "rate my box" posts, I think maybe my perception of scores was off.

That being said, I'm still standing by my thought that if a judges score is far lower than the others and comment card should be rendered to the team. Again, not asking him to change his score, but at least justify it to the team.


I'll throw a little gas on this.

It's always the 667 judge that gets the hit for being out of whack, in the example you cited you had 999, 999, 988, 889, 667, 778. Who's to say the the 2 999 judges weren't the ones out of line? When I table captain or tabulate scores I've observed that new judge's scores trend higher than more experienced judges, and that judges who tend to be the most critical are cooks.

Slamdunkpro
06-14-2011, 01:52 PM
On these scores, I would say that at a minimum J5 should at least fill out a card for the team......what team out there competiting would NOT WANT to know why this judge gave them a 6 in appearance when the others threw higher scores?????
The judge gave them a 6 - above average, not a 4. J5 doesn't know how the other judges scored unless the table captain tells them (and they aren't supposed to) The TC or rep might have asked why J5 rated that entry a 6 and suggest a comment card, but they won't come out with "the other judges gave this a 9 and you gave it a 6, why?", again, they aren't supposed to. If they did, it could be considered score tampering, influencing J5 to raise their scores on the next turn in - which is also unfair to other teams. The other issue with comment cards is that in this scenario, let's say the entry in question was sample #1; the TC or rep is asking the judge (maybe a new judge) to think back 6 samples ago after 15 minutes has elapsed and remember what they didn't care for.

Texana
06-14-2011, 03:07 PM
To some extent yes. But when consistent winners like the names you mention above get whacked by a "rogue" judge, do you think they're happy with it? This last weekend Johnny Trigg took 41st out of 47 teams in Ribs. Something tells me that the cream didn't quite make it to the top that day. I'm sure if you ask Alexa from R&Q about some their scores this past weekend, I'm sure she will have some categories that were a bit jacked up. Talent does rise, but one "rogue" judge can cost a team thousands of dollars.

I guess I am confused. I really do understand and believe the cream will rise to the top ... but I also understand that there is no body that can win every contest every time out. I dont care who they are. Everyone can have an off day and everyone can catch a bad table. It is very feasible to get a table that has 6 people that just dont care for your product .... and if you had a bad day as well ....

I really enjoy this hobby. I really like most of the people I meet. What I dont like are the ones that always want to blame everyone or everything else when they have an off day. It takes more than "one" rogue judge ... and why is he "rogue" just because maybe he/she did not like what they saw or tasted?

Texana
06-14-2011, 03:10 PM
I'll throw a little gas on this.

It's always the 667 judge that gets the hit for being out of whack, in the example you cited you had 999, 999, 988, 889, 667, 778. Who's to say the the 2 999 judges weren't the ones out of line? When I table captain or tabulate scores I've observed that new judge's scores trend higher than more experienced judges, and that judges who tend to be the most critical are cooks.

Excellent observation. Reminds of the guys that never complain when the get good scores for crappy product. And yes we have all been there!

Texana
06-14-2011, 03:16 PM
Just one more thought ... for all of those that are unhappy with CBJ's and/or their scores ....

I issue an invitation to have any or all of you come on down to Texas and cook. Different exposure might make some of you feel better about what you have ... or not. Either way you will meet a lot of good folks and have some fun along the way .....

Alexa RnQ
06-14-2011, 03:28 PM
Yes, no doubt cooking in Texas would make me very, very grateful for KCBS and its foibles http://www.divaherself.com/crylaugh.gif


Disclaimer: Yes, we've cooked our share of IBCA contests.

YankeeBBQboy
08-05-2011, 07:18 PM
I read through this thread and found all of your comments very interesting since I plan on taking a judging class in Nov. and then I'll be one of those "green" judges trying to learn the ropes. I do quite a bit of BBQ at home on my BGE but have not competed or really have a burning desire to compete due more to cost and time considerations. As a result, I see judging as a way to experience the competitive BBQ world and to give back to those who have inspired me and learned from.
I have also been a teacher for 30+ years which I think gives me an interesting perspective on the grading of BBQ. One of the consistent veins of this discussion is the thought that judges who do not compete themselves have a difficulty appreciating the "blood, sweat and tears" that goes into a competition box. Teachers hear this same argument but it goes something like this - "I should get an "A" because I worked REALLY hard on this project, assignment, test, etc." While I will acknowledge their work ethic, I base my score on what I components feel a paper or project should have. When I taught an advanced placement class, I had to grade papers based on a set criteria. The scores that students got were based on how well they knew the content as well as being able to perform to AP standards on the essays. It had very little to do with how hard they worked during the year. Bringing this back to BBQ, judging shouldn't reflect how much sleep you got the night before, that you had a poor cut of meat or that you have $400 in entry fees and meat costs. Quality is quality and that is what should be judged.
That being said, I can assure you that BBQ judges are not the only ones that can give scores that wildly vary or are inflated. Teachers who grade AP tests are trained similarly and spend at least a day reviewing so you don't get a wide variance in scoring. Two things those make this different from BBQ judging, first, the graders have a very strict rubric that they have to follow. If their scores are way off, the table leader will have a discussion with them and if need be, they will be dismissed. There is just too much riding on these test to allow incompetence or personal prejudices to influences scores. The problem though is that the students are all writing the same essay. Compare that to a BBQ contest where a person's' chicken can be different even though it was cooked by the same person for the same time period. Maybe a stronger and more consistent rubric is in order - I don't know since I haven't judged but it is a thought. Also, it is my understanding that judges aren't paid (but they can eat well!). With the number of judges in a contest with wide ranges of experience, its probably amazing that the scores are consistent as they are.
Being a newbie at all this, I look forward to the day that I judge my first contest and hope that I'm not the jerk judge that we all dread. I would also like to help a team so that I can learn firsthand what it takes to cook competitive BBQ.
Good luck to you all and happy BBQ'ing.

motoeric
08-05-2011, 08:13 PM
I look forward to tasting your entries at the BBQ Brethren Battle of the BBQ Brethen in LI in Manorville....

Yours in BBQ,

Cliff


I appreciate the support, but we are more enthusiastic than talented. As long as we have fun, we're happy (but I'm sure we'd be happier w/ a trophy or two!).

Eric

Still Smokin
08-06-2011, 09:08 AM
In the SCBA (South Carolina BBQ Assoc). we have a way of dealing with rogue Judge scores. In SCBA, (6 Judges & a Table Captain) The Table Captain samples the entry and judges as if it counted. If all is well, the Table Captains scores are not used, but if one judge has a total score lower than 10 (out of 1-17) then the Table Captain asks for an explanation of the low score and will usually substitue his score replacing the low score.

jbrink01
08-06-2011, 10:41 AM
I'll throw a little gas on this.
I've observed that new judge's scores trend higher than more experienced judges, and that judges who tend to be the most critical are cooks.
I've been a cook for 7 years, and during my judging class the rep advised me that I was being too critical. I disagreed in that the box we were discussing was "thrown together" and if the cook can't take time to make it nice I'll judge accordingly.

tony76248
08-06-2011, 11:00 AM
I will take IBCA scoring over KCBS any day of the week. The cream will rise to the top. Plus you won't have to hear the whining about getting stuck on a bad table.

ModelMaker
08-06-2011, 03:13 PM
Oh Boo Hoo Hoo, I didn't get all nines and there was a very bad judge....
Man I'm tired of the whinning.
First off, I think there has been a enormous case of score creep the last couple years. "I should have gotten more 9's". Do you know how to get a 9 from me?
Work for it, 9's are the tops the very best, none better!! I don't think there are that many very bests at any given contest. Yes, I give 9's but they are deserving of my best score.
Secondly, a 6 means your entry was average, not crappy, not inedable. If a cook gets a 6 right away the dukes go up and they start looking for the terrible judge that gave them a "average" score. If your BBQ is average that ain't a bad thing.
Get better.
A 7 from me means your on your way to greatness.
An 8 means I considered your entry might be a 9 but maybe next time.
You get a 9 from me you earned it and are one of the best cooks that day.
It's not rocket science. Cook your best each event and I'll judge to the best of my ability each event.
Deal?
Ed

RangerJ
08-06-2011, 06:23 PM
I will take IBCA scoring over KCBS any day of the week. The cream will rise to the top. Plus you won't have to hear the whining about getting stuck on a bad table.

Both have their merits but I'd rather have some idea as to why I walked or why I didn't walk. Pretty hard to improve your game if you don't have data to base it on.

JimmyDAL
08-07-2011, 06:28 AM
Personally I think that there should always be comment cards, that way everyone can get a view as to what they are looking for. In the International Chili Society every Judge makes comments and I was amazed when I judged how many were more concerned about bringing home a cooler full. No Coolers and Doggie bags should be allowed. One Guy was so P****ed that after the brisket was turned in and he missed the leftovers that he got into his vehicle and drove off on the quick side.

porkerfacebbq
08-07-2011, 11:34 AM
I think it is a 2 way street the judges should have to come out and cook with the teams, but I also think the cook teams should have to judge. I know the judges aren't out there all night cooking and not sleeping like a lot of us but they are still a important part of the sport. I agree with ford that the only real thing being certified means is that you gave a sanctioning body money. At the same time though it also means you are at least somewhat serious about being involved with BBQ as a competitive sport. Hey I say were all out there because we LOVE BBQ and we should all be having fun.



If BBQ is a sport does that make me a athlete?

Bunny
08-07-2011, 09:41 PM
is judging bbq without every having cooked "competition bbq" similar to judging a beer without having ever brewed a beer or critiquing an album without ever haven't written a piece of music?

I'm not aware of too many things...I know what I know if you know what I mean. Pretty clear to me!

Bunny
08-07-2011, 09:49 PM
Rich and I would like to know how many of the teams that are discussing this issue have actually taken a CBJ class ( we know some of you have)..and how many have judged without certification? Just wondering? After you have had certification....how many contests have you judged? Totally a different ball park. If you've taken the class and have never judged..you haven't actually judged.

boogiesnap
08-07-2011, 09:56 PM
I'm not aware of too many things...I know what I know if you know what I mean. Pretty clear to me!


really?:laugh:

jbrink01
08-07-2011, 09:57 PM
Hi Bunny,
Took your class THEN judged. Kinda feel that if your not schooled in it, stay out of it.

Crash
08-08-2011, 05:01 AM
Rich and I would like to know how many of the teams that are discussing this issue have actually taken a CBJ class ( we know some of you have)..and how many have judged without certification? Just wondering? After you have had certification....how many contests have you judged? Totally a different ball park. If you've taken the class and have never judged..you haven't actually judged.
Since you asked Bunny.... my wife and I have taken 2 KCBS judging classes. We've never judged and prefer to cook as competitors. I agree that if you've taken the class and have never judged..you haven't actually judged.

That being said, judges can be wrong... and sometimes very wrong. Then again, they do the best that they can do. They're just human.

mfreeman73
08-08-2011, 11:34 AM
I think it would be great if most competitors earned their CBJ and judged at least once. Sometimes they think their BBQ is the best out there. Maybe if they were able to eat some of the other competitor's BBQ, they might rethink how great theirs is.

As someone who's cooked and judged and who's gotten all kinds of wild scores, I understand the frustration. I've gotten a 9 and a 5 all in the same category. Talk about frustrating! So, I've thought long and hard about any solutions. Still thinking on that one.

I think keeping track of judge's scores and finding out who's really out way off from others may help. Also, if someone doesn't judge regularly maybe they should have to retake the class sometimes. Maybe you have to judge at least once a year to avoid the class. Not sure, just throwing stuff out there.

I think part of the judging is based on the personalities of the people. Some people are just critical by nature and some are more lenient. I do think one of the best things to help a judge is to keep judging. The more comps a person judges the better they should become over time. Over time it should be easier to spot the better Q in a tournament.

The system isn't necessarily a bad system. They even throw out the lowest score to help eliminate the oddball judge. I've seen some competitors who consistently place near the top and some who don't. Over time, if you're not near the top then it says more about your food than the judges eating it. Sure, there can be some improvements, and I hope that they will come, but it's not a broken system. They only true way to be completely fair is to have one table judge every competitor's food. And we know that isn't going to happen (nor should it).

harley
08-09-2011, 03:11 PM
Whether a judge scores high or low, the key is consistancy. As long as each team is scored relative to the others, then the playingfield is level and the better meats still rise to the top. If a judge is inconsistant then that's poison and it doesn't make a difference from what sanctioning group they come from. Regards from Canada!:cool:

BRBBQ
08-09-2011, 03:41 PM
Take a look at the many "rate my ...." threads here.

The Brothers are all well meaning, but the scores are all over the place.
No different than at a contest, sadly.
If you read the critiques carefully, many of the reviews seem to be more in the line of "what I would do to make the box better".
Also, a lot of speculation about tenderness and taste based on appearance.

As to cooking experience, I do not need a judge giving me what I call "pity points' due to the effort or money I have invested.
I want the judges to just judge my meat as it is.

I have seen the most critical judges to be the Master and other experienced judges and those who cook a lot also.
The seem to have developed very specific criteria that they are looking for, and if you don't hit it, you pay on the score sheet.


TIMYou took the words right out of my mouth:thumb:

rxcellentq
08-09-2011, 07:32 PM
I cooked at the maples this weekend got 999 from one judge and 555 from the guy next to him. Still scratching my head.

mfreeman73
08-10-2011, 01:43 PM
I cooked at the maples this weekend got 999 from one judge and 555 from the guy next to him. Still scratching my head.

I judged at the Maples and we got together as judges and decided to just screw with random competitors by putting in wildly different scores. :P

Just kidding.

What were your your other scores in that category? I'd be interested to see what other people were scoring on that.

Something would have to be very good at to get all 9s from me, but it does happen sometimes. It would have to really suck to get all 5s from me. I haven't ever scored something all low like that yet. Maybe I'm lucky and get better food.

Matt_A
08-10-2011, 02:25 PM
I have two equally good spares sitting in front of me at the table. Unknown to me is that one is cooked by a first timer using a computer controlled FEC 100 and the other is from a first timer that's a stick burner and slaved over fuel and temperature control for hours. One used a commercial rub and one created their own custom rub. One sat in his trailer watching satellite TV and the other napped in a hammock next to his pit so he could keep an eye on it.

How would knowing that information affect my judging? What a judge MUST look at is the finished product, not the roads it traveled to get to my judging mat.

Would cooking with a team give a judge a different perspective? Probably. Would that difference in perspective change the way they judged? Probably a bit. Chances are it will make the judge more aware of the tricks of boxing for turn-in and how the meats can be "staged" to show better in the appearance round. Will this help improve the judging? Doubtful. Every judge brings their own prejudices into the judging process.

ique
08-10-2011, 03:03 PM
Would cooking with a team give a judge a different perspective? Probably.

Not to mention the team they end up cooking with could give very different impressions. A judge could go out and cook with Cool Smoke and the cooking thing would seem easy as pie. Go cook with a newer team that is also cooking for a catering gig tomorrow and it will seem like crap load of work.

I'd rather just see them judge more contests and get a better feel for what excellent turn ins look and taste like.

jalon
08-10-2011, 06:48 PM
My two cents...

I've been doing cue for years. Smoked meats since I was a kid, but competition bbq, being different than backyard bbq, is new to me. Reading the forum here the last few months, I found more than a few posts actually recommending that I judge a few contests before I dive in being a competitor. This is to get an idea of what good, quality entries are like, and to see the right/wrong way to judge is.

I'm actually signed up for a class here on the 20th down in GA for CBJ. Along with recommending a class and judging before competing, someone also mentioned to speak with the Table Captain first, and ask him/her to keep an eye out for you. Not saying I can't make up my own mind, but a little mentoring in there after judging won't hurt either.

I think this is a great way to get my feet wet, since I don't have the means yet to compete. I also travel quite a bit supporting our men and women in the Navy, and its easier to show up and judge then it is to bring a rig and compete.

If anyone disagrees, I respect their opinion. However, if the option is there to take a class, learn to judge, and get a feel for what goes on in there before I compete, why should this be a bad thing? I feel you gotta start somewhere - and this sounds about as responsible way of doing it as I can think of.

Jalon

Matt_A
08-10-2011, 08:03 PM
Jalon, Once the first box is opened at your table, the table captain will not be able to guide you in any way until all judges at the table have written all their scores down and turned them over to the captain. Then you will be able to talk to the table captain and other judges about your experience with that round.

KC_Bobby
08-10-2011, 10:05 PM
Oh Boo Hoo Hoo, I didn't get all nines and there was a very bad judge....
Man I'm tired of the whinning.
First off, I think there has been a enormous case of score creep the last couple years. "I should have gotten more 9's". Do you know how to get a 9 from me?
Work for it, 9's are the tops the very best, none better!! I don't think there are that many very bests at any given contest. Yes, I give 9's but they are deserving of my best score.
Secondly, a 6 means your entry was average, not crappy, not inedable. If a cook gets a 6 right away the dukes go up and they start looking for the terrible judge that gave them a "average" score. If your BBQ is average that ain't a bad thing.
Get better.
A 7 from me means your on your way to greatness.
An 8 means I considered your entry might be a 9 but maybe next time.
You get a 9 from me you earned it and are one of the best cooks that day.
It's not rocket science. Cook your best each event and I'll judge to the best of my ability each event.
Deal?
Ed

Ed, I just always pray we don't land on your table. I don't have any factual knowledge, but I sense your score is about 1 score lower than many other judges - which can add up to 4 points (give or take).

Nothing wrong with it :icon_blush: but I pray.

jalon
08-11-2011, 07:23 AM
Matt, I gotcha, I've picked up on that. But no reason it can't be discussed after, just to make sure I've got the right idea.