PDA

View Full Version : Calling all kcbs judges!


BABYGOTBUTT
05-12-2013, 10:41 AM
I am relativly new to competition bbq having completed 11 competitions. We just wrapped up a competition in Fredericksburg and after reviewing our scores, I am left scratching my head. I am not, and I repeat, not playing the sore loser role because we did very well. Its that I saw something in the scoring that I have not personally expierenced in the previous competitions. Our scoring was ALL over the map. And I mean all over the map. I had two judges give us 9 9 8 and 8 8 9 in brisket. Two other judges gave us 5 6 5 and 6 6 6 for the same box. I know taste in subjective but as far as appearence? How to I go from a 5 from one judge to a 9 from another, both of which are looking at the same box. Tenderness? Same comment. I'm not saying it should have been a 9 or a 5. I'm just wondering how there can be such a swing. The biggest difference in scoring we have expierenced to date is 2 plus or minus. Same thing went for other meat categories?:mmph::mmph::mmph: Any feedback is very much appreciated.

fnbish
05-12-2013, 11:06 AM
While I am a cbj I've only judged once, but I cook a lot so my opinion is from that perspective......

We basically had the same thing happen this weekend and our last 3 previous competitions have been really good for us, especially in chicken. This time around I though the food (other than brisket) was exactly the same if not better than the previous 3 competitions and we had a few 6's and 7's in taste tenderness in chicken and pork.

In my short 3 years of competing (around 33 competitions total) I have now come to accept that 1 competition of bad scores doesn't need over analyzing especially if you thought the food was good. If your scores drop for multiple comps in a row then maybe you need to dig deeper into what went wrong. I really thought our food was great this weekend, but the judges did not. I actually thought our brisket was bad and that was one of our better scores :becky:.

So chalk it up to that. If your scores hurt for 2 in a row then it might be time to retool or think hard at what might have been different to make the scores drop.

For appearance when scores are way different for one judge I do wonder what happened as you would think scoring for appearance would see similar scores. For taste and tenderness though it is possible for 2 burnt ends or 2 pieces of chicken to have different tenderness. Can't bite into each piece of chicken to know for sure :crazy:.

Hawg Father of Seoul
05-12-2013, 01:51 PM
Yeah, I actually thought of apologizing to Vince (Rhythm and Que) for not accepting some of his statements in the "new software, tracking judges thread".

Two contests in a row we have had a category where some mindless prick has given us 777 when all of other appearance scores were 9's. Give me a bad score, just not a mindless one.

ViciousGame
05-12-2013, 01:51 PM
This weekend we walked in pork with 5th place. All the judges gave us no lower than an 8 in taste in tenderness except one judge. This judge scored us a 6 in taste & a 5 in tenderness. Then they left us a comment card stating, "Meat was very tough, not tender at all. Taste - not much at all." I almost felt offended when I read this. Of course, this score was dropped and we walked regardless; however, I have a sneaking suspicion that this was a new judge who had no idea what they were doing. The taste comment was kind of rude IMO. I can only hope the table capt. & reps inquired as to why their score was so off from the rest of the table. I pray for anyone who gets this particular judge again.

SCSmoke
05-12-2013, 04:08 PM
Looking at the overall scores from that contest they are pretty low. This could mean there were a bunch of new or unseasoned judges there. I wouldn't beat yourself up too much about the scores.

Nordy
05-12-2013, 04:27 PM
Probably caught a table with a couple of cooks judging... :shock:

Smoke'n Ice
05-12-2013, 05:36 PM
Newbie judges and new judges recruited for the contest (yes this happens more than you would believe) are told by the rep, the score card and the cd that average is 6 and a good starting point and this is hammered in. Last contest, we knew that each table had one recruit and a recruit as a TC. I looked at my scores and could tell you exactly which was the new recruit. Ignore this one an move on.

Most "seasoned" judges actually believe the range is 7 to 9 and not the 2 to 9 range. So the question is, which score is more realistic the john doe or the Master Certified BBQ Judge?

Red Valley BBQ
05-12-2013, 05:40 PM
Our scoring was ALL over the map. And I mean all over the map. I had two judges give us 9 9 8 and 8 8 9 in brisket. Two other judges gave us 5 6 5 and 6 6 6 for the same box. I know taste in subjective but as far as appearence? How to I go from a 5 from one judge to a 9 from another, both of which are looking at the same box.

Did you put burnt ends in the box? It may have been that a couple of those judges "expect" burnt ends and judge down if they don't get them instead of judging what is in the box.

Same with ribs, it may have been a "babyback vs. spares" type of thing.

carlyle
05-12-2013, 06:55 PM
Differences of more than 2 points within the same table should catch the attention of the table captain and rep and eventually, hopefully, the organizer.

It is the table captain and rep that can tell if this happened just once or if a judge has a pattern of either high or low scoring compared to the rest of the table.

The reps I know will have a private word with a judge. Then observe to see if scores get more even.

There is just not enough information to know if officials were aware or tried to something so that the trend does not continue. Once the score is on the card, it can not be changed. So what is done is done and can not be undone.

I keep hoping that the new scoring system with judge tracking will help this situation.
Time will tell - and that is if the system ever gets into use.

At this stage, all of us just scratch our heads.

There have been judges I did not seat again when I know that scoring is out of wack
and attempts at rehabilitating have failed. Try to rehab a judge before black balling them. Won't help what is already done, but reduces the chances of it happening again.

Derussi
05-12-2013, 07:45 PM
I was a Table Captain this past weekend and my table consisted of 1 first timer, another with 2 contests, 3 with 15 - 20 and 1 Master Judge. Reviewing the scores, showed that the newbies judged higher then the more experienced judges - although not as severe as your example. Listening to discussions after a catagory it's my opinion that it's the judges in the 10 - 20 contest experience that judge lower.

CivilWarBBQ
05-12-2013, 08:34 PM
I agree that on the whole Masters tend to be more critical judges than average. I keep that in mind when balancing my tables. A lot of the Master Judges in these parts are friends, so if you are not carefully controlling seating assignments you can end up with an entire table of Masters, and that table will be likely to be lower scoring than most.

smoke on this
05-12-2013, 08:54 PM
What most judges do not understand is that a 7 is the kiss of death for a team trying to win. What stands out to me is that a judge that scores an entry 977 or 877 had a positive view of the box but did not like the final product for what ever reason and I look very hard at trying to adjust in some way to appeal to all. This judge knows how to count to 9. At the same time there is someone who sees the box appearance as a 6 or 7. For some reason the 6-7 judge never ups the taste and texture score to a higher score than his first number. It is like that is as far as he ever learned to count. I am very supportive of the judges and have a great relationship with many of them. They do a great job. It is the 5-10 % that I have a problem with . They have a direct negative impact on 24 people who just invested $1000 and 3 days of their life for a competition. I really wish these people who pass out the low scores that are inconsistent with the other judges would have to work a complete cook with a team before they could judge again. The day will come when KCBS addresses this issue. In my opinion , this is the single biggest flaw in the whole organization.

Hozman
05-12-2013, 09:30 PM
I want to thank you for this thread. I did my second comp this weekend and was mad, confused about the scores I got. I couldn't understand how ribs were scored for taste 1-9, 2-8, 1-7 and 2 6's?? I understand taste is relevant to an individual but I can't comprehend how one thinks perfect and 2 think horrible cause in reality that is what a 6 represents to a cook.

Had same score for tenderness. I had 6 bones from same rack so again how is their such an extreme from 1 to another?

This happened in every meat category fir tenderness and taste would get at least 1-9, 1-, couple 7's then a 6?

So as a new competitor I am looking to you guys for help. In my first comp we were in the top 50% every meat and overall. This one in bottom 10 every meat and overall. So do I change how I am doing things or see how the next one goes in 2 weeks?

Vince RnQ
05-12-2013, 10:32 PM
Yeah, I actually thought of apologizing to Vince (Rhythm and Que) for not accepting some of his statements in the "new software, tracking judges thread".

Two contests in a row we have had a category where some mindless prick has given us 777 when all of other appearance scores were 9's. Give me a bad score, just not a mindless one.

Sorry to read about the unfortunate scores. I know it happens to all of us but it still sucks when it does. As for an apology, none is necessary. The beauty of a forum that we all have the opportunity to exchange ideas and opinions. I recall that our dialog in that other thread was just that, no hard feelings or disrespect given or received.

Best of luck at your next event and hopefully the mindless pricks stay home that day!

Rookie'48
05-12-2013, 10:46 PM
Just being a MCBJ doesn't mean that you are a great judge. MOST Master CBJs are very good judges who take their responsibilities seriously, occasionally there's one that isn't.

On an average, I would say that the toughest judges are competition cooks who are judging this weekend. A lot of them are harder on themselves, and other cooks, because they feel that they know what deserves a 9 and if this entry doesn't meet that expectation then too bad.

Almost every organizer or KCBS Rep will do their level best to mix up the judges according to experience, so that one table won't have all Masters while another one has mostly newer judges. Yes, judging is a subjective art and there's going to be differences of opinion on what's a 6 or what's a 9.

The Table Captain should tell a Rep if one judge's scores are way out of line. The Rep should keep an eye on that judge's scores and have a "chat" if the Rep feels that it is needed. The new scoring software should be able to track judges who are consistently out of line with the other judges at his/her table over a period of time. That doesn't mean on one category or at one contest - it means that if there is a pattern of being way too low (or way too high) it will show up.

Bentley
05-12-2013, 11:24 PM
Never heard of this...put me in the 2-9 camp.


Most "seasoned" judges actually believe the range is 7 to 9 and not the 2 to 9 range.

BABYGOTBUTT...truly wish I could give you an answer that could help, but I can't.

BRBBQ
05-13-2013, 12:07 AM
I'd say they didn't have enough judges, so they found someone walking down the street, I've judged a few contest and this happened at my table..No way it can be fair

BABYGOTBUTT
05-13-2013, 06:33 AM
Did you put burnt ends in the box? It may have been that a couple of those judges "expect" burnt ends and judge down if they don't get them instead of judging what is in the box.

Same with ribs, it may have been a "babyback vs. spares" type of thing.

Yes...and they were spot-on melt in your mouth.

MAP
05-13-2013, 06:41 AM
Big John, My score sheet was the same way all over the place. my pork call box was 4 9's 1 8 and a 5 on app. all my strange scores were from judge 3. I am wondering if that "new" judges seat right in the middle of the table.

Hawg Father of Seoul
05-13-2013, 06:44 AM
I want to thank you for this thread. I did my second comp this weekend and was mad, confused about the scores I got. I couldn't understand how ribs were scored for taste 1-9, 2-8, 1-7 and 2 6's?? I understand taste is relevant to an individual but I can't comprehend how one thinks perfect and 2 think horrible cause in reality that is what a 6 represents to a cook.

Had same score for tenderness. I had 6 bones from same rack so again how is their such an extreme from 1 to another?

This happened in every meat category fir tenderness and taste would get at least 1-9, 1-, couple 7's then a 6?

So as a new competitor I am looking to you guys for help. In my first comp we were in the top 50% every meat and overall. This one in bottom 10 every meat and overall. So do I change how I am doing things or see how the next one goes in 2 weeks?

Brother, with those scores... you turned in a 7. That is the way the scoring is supposed to work. Not bad at all for where you are at in the game. Adjust accordingly and hit it smarter next time.

Bratenmeister
05-13-2013, 08:23 AM
Contest reps usually ask the judges about their experience levels and work toward mixing up the tables so that new judges and master judges are spread around and not congregated at one table. Some reps also try to spread around judges based on gender.

I wonder if the reps, in spreading around the judges, should also ask the judges to raise their hand if they have cooked in x-number of contests and then spread around the experienced cooks that are judging the contest as well. I've never seen that done, but it might be a good idea.

QN
05-13-2013, 09:03 AM
Related to the comments about 7,8,9 judges and 7 being a score to kill an entry...
I see more and more judges back at the 7-8-9 scoring; also viewing 7 as a bad score. Per KCBS 7 is above average; that is not bad, it is above average. Unfortunately, when you have several 7-8-9 judges at a table along with judges who are trying to do what they have been taught and use the entire scale from 2 - 9 it can result in scores as described in this thread. While the judge class instructors do teach to use the entire scale from 2 to 9 and the CD played at the judge meeting at every contest does the same, the reps tend to make comments about how hard the cooks work and how much money they spend to compete, etc, etc. I think this can result in more of the 7-8-9 scoring also by some judges. KCBS judging used to teach to start at 9 on every entry and score/mark down from there. Then it was changed to start at 6 and go up or down to score each entry. Now, there is no defined starting point; each entry is supposed to be scored as presented with 6 defined as "average"; 5 "below average" and 7 "above average". I have yet to have anyone be able to clearly define or explain exactly what "average" barbecue is.... maybe it depends on what your definition of "is" is...

BABYGOTBUTT
05-13-2013, 09:11 AM
Brother, with those scores... you turned in a 7. That is the way the scoring is supposed to work. Not bad at all for where you are at in the game. Adjust accordingly and hit it smarter next time.

I think you are missing the point. How should I hit it smarter, let say in appearance? One judge thinks its a 9 and another judge thinks its a 5. What would you do smater or different? If I believe one judge, I change nothing. If I believe another, I change everything. My point is....there should not be a 5 and a 9, especially on appearance for the same box.

I tend to agree with the judges walking of the street theory....which if thats the case, it is a shame given the amount of time, effort and money that we all as competitors put into these competitions.

Again, as I stated in the opening thread, I am not complaining that my food was good and got bad scores or that it was bad and got good scores. Mearly pointing out something that I had not seen in ANY of my prior compeitions, which is the vast difference in scores between judges sitting at the same table.

Thanks for your feedback.

boogiesnap
05-13-2013, 09:28 AM
let's see a pic?

NazBQ
05-13-2013, 10:02 AM
It happens and it will probably never change. Last month I got a 999 in chicken from one judge and a 667 from another.

Slamdunkpro
05-13-2013, 10:51 AM
We've seen more variance in scoring this year than in years past. We can live with the scoring inconsistencies that are a fact of life in KCBS judging.

What is frustrating is the total lack of comment cards so far this year; if you give us a 4 or a 5 help a brother out and write a comment card.

Hawg Father of Seoul
05-13-2013, 11:05 AM
I think you are missing the point. How should I hit it smarter, let say in appearance? One judge thinks its a 9 and another judge thinks its a 5. What would you do smater or different? If I believe one judge, I change nothing. If I believe another, I change everything. My point is....there should not be a 5 and a 9, especially on appearance for the same box.

I tend to agree with the judges walking of the street theory....which if thats the case, it is a shame given the amount of time, effort and money that we all as competitors put into these competitions.

Again, as I stated in the opening thread, I am not complaining that my food was good and got bad scores or that it was bad and got good scores. Mearly pointing out something that I had not seen in ANY of my prior compeitions, which is the vast difference in scores between judges sitting at the same table.

Thanks for your feedback.

I quoted Hozman when I said that. Feel free to look at his scores and form your own opinion. I am at the disadvantage of a 7000 level psychometrics class. It makes you see numbers differently, especially likert scales representing qualitative data.

Editted to add: Post all of your scores and I will give you my impression of your "true" score.

Arlin_MacRae
05-13-2013, 11:08 AM
As a pair of judges in the 10 - 15 comps judged range, my wife and I have already seen a lot of good and not-so-good stuff. Something new that popped up at our last comp was a Rep's suggestion that the judges at each table discuss the turn-in's scoring after it's done. Specifically, not "this was the best and this was the worst," but actually sharing the numbers with each other.
I thought it was a great idea because, even though Dave, below, is completely right, Table Captain's don't always have their acts together either...

The Table Captain should tell a Rep if one judge's scores are way out of line. The Rep should keep an eye on that judge's scores and have a "chat" if the Rep feels that it is needed. The new scoring software should be able to track judges who are consistently out of line with the other judges at his/her table over a period of time. That doesn't mean on one category or at one contest - it means that if there is a pattern of being way too low (or way too high) it will show up.

Honestly, while I know a 7 IS the kiss of death, I won't hesitate to give one! I've cooked on teams and I have a firm picture in my mind of what the numbers mean. A new judge with no cooking experience certainly needs a few comps under their belt before they have that mental picture, but there are experienced judges with mental issues that keep them from giving a 9-9-9 score or - as my wife heard last time - ones that won't score anything low enough to make him fill out a comment card... :doh: Neither attitude helps the team!

Arlin

BABYGOTBUTT
05-13-2013, 01:20 PM
I quoted Hozman when I said that. Feel free to look at his scores and form your own opinion. I am at the disadvantage of a 7000 level psychometrics class. It makes you see numbers differently, especially likert scales representing qualitative data.

Editted to add: Post all of your scores and I will give you my impression of your "true" score.
Post my scores so you can judge my scores to give me what in your jugment is my true score? Yep..thats why I posted this thread. :doh:

Rookie'48
05-13-2013, 01:40 PM
...What is frustrating is the total lack of comment cards so far this year; if you give us a 4 or a 5 help a brother out and write a comment card.

For me, if I give you a 6 then you're going to get a comment card. Yes, a 6 is defined as "average" but if all you're cooking is average BBQ then you'll never get a call much less a GC or RGC.

My personal feeling is that giving you a 6 means that your food is kinda "suck-y" and I should be willing to tell you why I think that way. If you get two 7s, three 6s and a 8 with no card you'll just scratch your head & say "WTF didn't they like about it?" If you get the same two 7s, three 6s and a 8 with a card from one of the judges saying something like "no meat flavor, only sauce" or "very, very mushy" or "way too much salt" then at least you have a clue as to what the rest of the judges thought about your food.

I think that I owe you that courtesy.

Balls Casten
05-13-2013, 01:50 PM
"In general" it is human nature. People of different backgrounds, different levels of bbq education, different level of interest, different tastes ... tasting different pieces of meats will arrive at different results.

The way I read it two judges didn't think you brisket entry was exceptional.

QN
05-13-2013, 01:54 PM
For me, if I give you a 6 then you're going to get a comment card. Yes, a 6 is defined as "average" but if all you're cooking is average BBQ then you'll never get a call much less a GC or RGC.

My personal feeling is that giving you a 6 means that your food is kinda "suck-y" and I should be willing to tell you why I think that way. If you get two 7s, three 6s and a 8 with no card you'll just scratch your head & say "WTF didn't they like about it?" If you get the same two 7s, three 6s and a 8 with a card from one of the judges saying something like "no meat flavor, only sauce" or "very, very mushy" or "way too much salt" then at least you have a clue as to what the rest of the judges thought about your food.

I think that I owe you that courtesy.

So a KCBS score of 6 or "average" barbecue can also be defined as "suck-y" ? Wow, I had no idea. Never imagined something specifically defined as "average" could be considered "suck-y". So, should KCBS make a rules change to the word/definitions associated with each numeric value on the score card? If 6 is suck-y, then 7 is "better than suck-y" and 5 is then some margin of "less than suck-y"? Do you ever give a score lower than 6 since your scale already has that defined as "suck-y"? Here is a public example of a KCBS master judge and board member and how they score entries when they judge. Nothing personal Dave, but I have always had an issue with using the term "average" to define a numeric score for competition barbecue. No one has ever been able to expain what average is. Just a peeve of mine...

Hawg Father of Seoul
05-13-2013, 02:56 PM
So a KCBS score of 6 or "average" barbecue can also be defined as "suck-y" ? Wow, I had no idea. Never imagined something specifically defined as "average" could be considered "suck-y". So, should KCBS make a rules change to the word/definitions associated with each numeric value on the score card? If 6 is suck-y, then 7 is "better than suck-y" and 5 is then some margin of "less than suck-y"? Do you ever give a score lower than 6 since your scale already has that defined as "suck-y"? Here is a public example of a KCBS master judge and board member and how they score entries when they judge. Nothing personal Dave, but I have always had an issue with using the term "average" to define a numeric score for competition barbecue. No one has ever been able to expain what average is. Just a peeve of mine...

Average BBQ does suck. In the general population I am an average juggler. I can juggle 1 ball with out dropping it.

Not being a smartazz, just describing average.

QN
05-13-2013, 03:02 PM
Average BBQ does suck. In the general population I am an average juggler. I can juggle 1 ball with out dropping it.

Not being a smartazz, just describing average.

You described average juggling, not average barbecue...
I think a better word than average might be mediocre based on the comments on 6 not being a good score....
av·er·age
/ˈav(ə)rij/

Noun
The result obtained by adding several quantities together and then dividing this total by the number of quantities; the mean.
Adjective
Constituting the result obtained by adding together several quantities and then dividing this total by the number of quantities.
Verb
Amount to or achieve as an average rate or amount over a period of time: "annual inflation averaged 2.4 percent".
Synonyms
noun. mean - medium - mediocrity adjective. mean - ordinary - medium - middling - mediocre - middle

Arlin_MacRae
05-13-2013, 03:25 PM
In the world of competition BBQ is "average" ever going to win? I say no, and a 6 will ensure just that. In MY mind, above 6 is the better 7, the great but not perfect 8, and the WOW 9.

Really, if there weren't teams out there who can and do produce "WOW" BBQ I'd relax my standards, but they do exist and they do turn in meat like that. So average comp turn-ins get a 6, saving the 7's, 8's and 9's for those who do better.

Like Rookie Dave, I give comment cards for 6's and below. At my last comp I had to rate two of the three scores on a rib below a 6. I didn't enjoy doing it at all, but it was pretty bad and they deserved to know what I was thinking. Although some judges say things like, "I don't need to send a comment card in - they know it was bad," I make danged sure they know why I hit them with that. As a judge I feel it's part of my responsibility to help teams get better. And there's something in the Judge's Oath to that effect, if I remember right. ;)

Arlin

QN
05-13-2013, 03:43 PM
In the world of competition BBQ is "average" ever going to win? I say no, and a 6 will ensure just that. In MY mind, above 6 is the better 7, the great but not perfect 8, and the WOW 9.

Really, if there weren't teams out there who can and do produce "WOW" BBQ I'd relax my standards, but they do exist and they do turn in meat like that. So average comp turn-ins get a 6, saving the 7's, 8's and 9's for those who do better.

Like Rookie Dave, I give comment cards for 6's and below. At my last comp I had to rate two of the three scores on a rib below a 6. I didn't enjoy doing it at all, but it was pretty bad and they deserved to know what I was thinking. Although some judges say things like, "I don't need to send a comment card in - they know it was bad," I make danged sure they know why I hit them with that. As a judge I feel it's part of my responsibility to help teams get better. And there's something in the Judge's Oath to that effect, if I remember right. ;)

Arlin

I agree with you. Sounds like you use the entire scale from 2 to 9 per KCBS training and instructions. On the ther hand, there are still judges out there using the 9-8-7 system which can result in scores as described in the original post of this thread.

hamiltont
05-13-2013, 03:47 PM
In the world of competition BBQ is "average" ever going to win? I say no, and a 6 will ensure just that. In MY mind, above 6 is the better 7, the great but not perfect 8, and the WOW 9.

Really, if there weren't teams out there who can and do produce "WOW" BBQ I'd relax my standards, but they do exist and they do turn in meat like that. So average comp turn-ins get a 6, saving the 7's, 8's and 9's for those who do better.

Like Rookie Dave, I give comment cards for 6's and below. At my last comp I had to rate two of the three scores on a rib below a 6. I didn't enjoy doing it at all, but it was pretty bad and they deserved to know what I was thinking. Although some judges say things like, "I don't need to send a comment card in - they know it was bad," I make danged sure they know why I hit them with that. As a judge I feel it's part of my responsibility to help teams get better. And there's something in the Judge's Oath to that effect, if I remember right. ;)

Arlin

That's pretty much how I score as well. 6 gets a comment card EVERY time. The debate over whether judges should be using the 2-9 range or the 6-9 range is a valid one though. Personally I feel I can objectively judge BBQ in a 4 score range (6-9). Using an 8 score range (2-9) IMO is not realistic. I'd like to know how the judges that use the full range of 8 scores do it. What makes it a 5 rather than a 6 or 4 rather than a 5? I'm not trying to stir it up. I'd just like to know... Cheers!!!

bbq.tom
05-13-2013, 04:25 PM
I believe in using the entire scale, but I honestly can state that in the 32 KCBS contests that I've judged so far I haven't seen an entry that rated anything below a 5. Some cooks feel that a 6 or even a 7 is "the kiss of death", but can they honestly say that they've never turned in something that was less than "Very Good" - 8? Critical self-examination might be in order. When it comes to consistency at a table, I agree that pork or brisket being off more than 2 points might point to a problem (can't say the same for chicken and ribs as they can be completely different from one another). I've seen/heard more "strange" comments and scores come from 'cook judges' than either Master CBJs or newbies.
And YES, I give a comment card for EVERY 6 or below and sometimes for a 7 also.

boogiesnap
05-13-2013, 04:27 PM
Average BBQ does suck. In the general population I am an average juggler. I can juggle 1 ball with out dropping it.

Not being a smartazz, just describing average.

juggling one ball is a DQ. that's tossing, not juggling. score? 1.

:icon_smile_tongue:

Arlin_MacRae
05-13-2013, 04:42 PM
That's pretty much how I score as well. 6 gets a comment card EVERY time. The debate over whether judges should be using the 2-9 range or the 6-9 range is a valid one though. Personally I feel I can objectively judge BBQ in a 4 score range (6-9). Using an 8 score range (2-9) IMO is not realistic. I'd like to know how the judges that use the full range of 8 scores do it. What makes it a 5 rather than a 6 or 4 rather than a 5? I'm not trying to stir it up. I'd just like to know... Cheers!!!
Well, I have to take issue with that first part. KCBS is a 2-9 scored society and if you have some judges using the whole range, and others only using the top four, some team is going to get hurt.
The team that gets a really low score from a 2-9 judge is going to score the exact same as a 6 team does on a 6-9 judge's scale.
If you go by the 2-9 rule set, a 6 might be an example of good backyard Q: decent, but maybe not really competition quality. Maybe that rib would get a 6 from the 6-9 judge, too. OK fine, they're equal. But what about the rib that gets a 2 from the 2-9 judge? That 2 (inedible) translates to a 6 on the 6-9 scale, and they are not remotely equal. The 6-rated team is now rated the same as a team who's product isn't even edible!
As long as everyone uses the same system (within the system, I mean) then there's a chance at equality. Otherwise, you might as well compete under two rule sets and the team won't know which they got until the scores come out.

As far as what makes up the 2 to 9 scores, nobody can tell you that. Nobody told me that. The judge has to come up with the divisions on his/her own. It took me a while. And when you're trying like hell to keep from comparing one turn-in to another it's hard! But at the end of the contest; at the end of your 5th or 10th or 30th contest, you have a feel for the numbers. Case in point: my very first judging session. I had a rib that looked bad, but that wasn't the worst part. When I bit into it I couldn't hardly pull the meat off the bone, and what did come was stringy and stuck between my teeth. It was baaaaad. Matter of fact, I just couldn't even take a second bite. I scored it a shaky 3 because I didn't want to hurt the team with my inexperienced rating. (grin) When talking with an experienced judge at the table he said something like, "Dude, If you couldn't put that rib back in your mouth...wouldn't you call that 'inedible'?" Uhhh, yes. BING! From then on I knew what a 2 really was! And so on and so forth up the scale it goes. But remember, every judge has their own idea of what stuff needs to look or taste like. YMMV!!

Sorry for being so long-winded... I seem to be a bit passionate about getting it right when I'm in the judge's chair... :roll:

boogiesnap
05-13-2013, 04:51 PM
a 7 IS the kiss of death in terms of winning. there are too many great cooks that don't get them and win. HOWEVER, the full range is needed in order for the other teams to know ehere they stand against the winners and thus try to improve.

Smoke'n Ice
05-13-2013, 05:00 PM
Several years ago, the judges were instructed to start at 9 and go down. Everyone complained about the high scores so they changed it to start at 6 and go up or down. The newbies do this but the old timers, having had it instilled in their brain by the rep and the teams, that a 7 is a kiss of death so don't use it.

I'm just wondering if we change the starting point to 5 and the judge has to justify why he went up or down and if he goes above a 7, then he writes a comment card or below a 3 requires a comment card.

Does our span go from 7-8-9 to 4-5-6. Just curious.

Oh, I forgot, our new score system can not track comment cards as it would make it to difficult on the programmer and rep so they will go away!

boogiesnap
05-13-2013, 05:05 PM
i can be more specific...

i had a rib program, awesome at home. couldn't replicate in the field over 2 tries. i got 5's 6's 7's and a handful of 8's, if i recall over those 2 events.

i then knew what i was doing wasn't working, retooled the entire cook, and took 8th and RGC'ed. if i had only gotten 7's i may have continued thinking "i just keep hitting tough tables".

smoke on this
05-13-2013, 05:32 PM
OK , it appears most agree that the judges are not on the same page. I think this has been said about 43 times so far. That is my point. What is fair about this to the hundreds of teams each week? How or what needs to be done to correct the largest single problem in competition bbq. My thought is , if you cook bad you have no chance, if you cook good you have a 50/50 chance, the odds of cooking all 4 meats good at the same time is based on your skills and not 100% . End result is the odds are not good with judges not being on the same page.

boogiesnap
05-13-2013, 05:35 PM
OK , it appears most agree that the judges are not on the same page. I think this has been said about 43 times so far. That is my point. What is fair about this to the hundreds of teams each week? How or what needs to be done to correct the largest single problem in competition bbq. My thought is , if you cook bad you have no chance, if you cook good you have a 50/50 chance, the odds of cooking all 4 meats good at the same time is based on your skills and not 100% . End result is the odds are not good with judges not being on the same page.

I still cannot agree with this.

There are too many good teams that win consistently.

RX2006JE
05-13-2013, 06:05 PM
wouldn't the best way to judge is to have one set of judges is all they score is chicken at a contest, then another set just to judge the ribs and so on. but that would have to take 6 extremely hungry eaters for each category. that way every teams entry for each category gets judged by the same set of eyes and mouths. Is that feasible defiantly not especially in a large contest, but that would be the only way that every ones meat gets the same treatment. well really just one judge per category would be the absolute best way. But until then sometimes just gotta scratch your head. I had the same type scoring issue my last time out, it happens when judging is subjective.

sdbbq1234
05-13-2013, 07:12 PM
I still cannot agree with this.

There are too many good teams that win consistently.

AMEN!!

Good teams win; regardless of the judges.

wallace

Bratenmeister
05-13-2013, 07:52 PM
I think the good teams win consistantly because they have (1) perfected a technique that consistantly produces tender product (2) have developed a flavor profile that appeals to a broad base of tastes, and (3) compete in a lot, a lot, of competitions so that the law of averages is in their favor.

No matter how talented a cooking team is, I do not think it is possible for a cook to impress 100% of the population 100% of the time. I suspect that, even for the top teams, on any given Saturday, say a competition with 6 or more judging tables, there might be one table that if its product lands on it might get a perfect or close to perfect score, there might be 3 or 4 other tables that will give high marks, and there might be one or two tables with 1, 2, or even 3 judges that are not impressed. But since they compete in many competitions in a given year, they are landing on the good tables more than the bad ones and, thus, have the law of averages in their favor.

A talented team that competes in only 5 or so contests a year will feel the pain of landing on the table with the unimpressed judges more severly since they are not exposing their product to a broader base of the judging population.

That being said, I think there are things that KCBS can do as an organization to improve the consistancy of the judging. I think, as an organization, it is speaking out of both sides of its mouth to teach the judges in class to use a 2 through 9 scale, but then, on game day, it instructs their reps encurage judges to ignore 2 through 5 and use 6 sparingly. This creates confusion with people who do want to judge fairly and fosters inconsistancy.


AMEN!!

Good teams win; regardless of the judges.

wallace

boogiesnap
05-13-2013, 08:10 PM
I think the good teams win consistantly because they have (1) perfected a technique that consistantly produces tender product (2) have developed a flavor profile that appeals to a broad base of tastes, and (3) compete in a lot, a lot, of competitions so that the law of averages is in their favor.

No matter how talented a cooking team is, I do not think it is possible for a cook to impress 100% of the population 100% of the time. I suspect that, even for the top teams, on any given Saturday, say a competition with 6 or more judging tables, there might be one table that if its product lands on it might get a perfect or close to perfect score, there might be 3 or 4 other tables that will give high marks, and there might be one or two tables with 1, 2, or even 3 judges that are not impressed. But since they compete in many competitions in a given year, they are landing on the good tables more than the bad ones and, thus, have the law of averages in their favor.

A talented team that competes in only 5 or so contests a year will feel the pain of landing on the table with the unimpressed judges more severly since they are not exposing their product to a broader base of the judging population.

That being said, I think there are things that KCBS can do as an organization to improve the consistancy of the judging. I think, as an organization, it is speaking out of both sides of its mouth to teach the judges in class to use a 2 through 9 scale, but then, on game day, it instructs their reps encurage judges to ignore 2 through 5 and use 6 sparingly. This creates confusion with people who do want to judge fairly and fosters inconsistancy.

so you're suggesting because a certain team may have entered 10 contests and won 8 of them they won those because they entered 10? or 10 out of 15, etc. etc.?

i don't think so.

i agree a team needs to compete ALOT to be competitive, but i don't see law of averages in their favor. i saw the same guys and gals every week-end, some always won, some did not.

if you get a 4-7, then a 9, and didn't win, there's a reason. another, better team got all 9's and they did.

we can't all be larry bird. but we can all have fun in the journey.

Bratenmeister
05-13-2013, 08:21 PM
so you're suggesting because a certain team may have entered 10 contests and won 8 of them they won those because they entered 10? or 10 out of 15, etc. etc.?

i don't think so.

i agree a team needs to compete ALOT to be competitive, but i don't see law of averages in their favor. i saw the same guys and gals every week-end, some always won, some did not.

if you get a 4-7, then a 9, and didn't win, there's a reason. another, better team got all 9's and they did.

we can't all be larry bird. but we can all have fun in the journey.

No. The focus of my comment is the judging inconsistency, not the skill level of any teams.

boogiesnap
05-13-2013, 08:23 PM
my point is, there is consistency within the inconsistency.

Gadragonfly
05-13-2013, 08:45 PM
Maybe we need a poll -

As a cook (cooks only reply), would you rather be judged 2 - 9 with the judges being honest, even if they tell you that your product was not up to the challenge on any given weekend.....

or

As a cook (cooks only reply), would you rather the judges consider that you've invested $1,000 and 3 days and only use numbers 6 - 9?

Because in the end, the reps who tell us to keep in mind all that you have invested are only doing so with your best interest in their hearts but apparently it has created confusion in the judging tent and perceived inconsistent scores.

smoke on this
05-13-2013, 09:12 PM
It does not matter which one, just choose one and try to get all on the same page.

Outnumbered
05-13-2013, 10:32 PM
In the world of competition BBQ is "average" ever going to win? I say no, and a 6 will ensure just that. In MY mind, above 6 is the better 7, the great but not perfect 8, and the WOW 9.

Really, if there weren't teams out there who can and do produce "WOW" BBQ I'd relax my standards, but they do exist and they do turn in meat like that. So average comp turn-ins get a 6, saving the 7's, 8's and 9's for those who do better.

Like Rookie Dave, I give comment cards for 6's and below. At my last comp I had to rate two of the three scores on a rib below a 6. I didn't enjoy doing it at all, but it was pretty bad and they deserved to know what I was thinking. Although some judges say things like, "I don't need to send a comment card in - they know it was bad," I make danged sure they know why I hit them with that. As a judge I feel it's part of my responsibility to help teams get better. And there's something in the Judge's Oath to that effect, if I remember right. ;)

Arlin

Arlin, THANK YOU for saying this. I started a very similar thread a year ago that essentially mirrors this topic. Admittedly, my attitude has changed some now that I'm in my second year of competition and I can think back to what I did to get one score vs. another. However, I was told by a KCBS board member on this forum that it is not the job of the judge to make cooks better cooks.

Of course it's not their job and it's not possible. Only I can make myself a better cook...however, if we are going to spend the money to do this and the judges are going to take the responsibility to judge, then it should be their job to defend why they are scoring the way they score and help us understand why we may want to change something.

I liked my ribs this weekend. But I don't count. The judges didn't, and I get that, but I didn't know why. My neighbor, who has the cojones to be honest with me, said my ribs were good, but pretty bland. He tasted sweet, but no real spice in them.

Did I like hearing that? Wish it wasn't true, but it was. So, yeah, I loved hearing it. I now have a direction on what I need to change.

Now if we want to get into statistics and decide how accurate a focus group of one is... :mrgreen:

juggling one ball is a DQ. that's tossing, not juggling. score? 1.

:icon_smile_tongue:

:clap:

Outnumbered
05-13-2013, 10:37 PM
But...put me in the following two camps:

Make every judge use the full scale with no comment cards. If it's average, let me know it is by giving me a 6, not an 8.
I believe the best teams win consistently. Average teams like mine will break through now and again, but the really good teams are there consistently. I don't believe it's possible for them to catch a lucky table for every category at every contest.

Rookie'48
05-13-2013, 11:17 PM
Quote:
Originally Posted by Rookie'48 http://www.bbq-brethren.com/forum/images/buttons/viewpost.gif (http://www.bbq-brethren.com/forum/showthread.php?p=2479512#post2479512)
For me, if I give you a 6 then you're going to get a comment card. Yes, a 6 is defined as "average" but if all you're cooking is average BBQ then you'll never get a call much less a GC or RGC.

My personal feeling is that giving you a 6 means that your food is kinda "suck-y" and I should be willing to tell you why I think that way. If you get two 7s, three 6s and a 8 with no card you'll just scratch your head & say "WTF didn't they like about it?" If you get the same two 7s, three 6s and a 8 with a card from one of the judges saying something like "no meat flavor, only sauce" or "very, very mushy" or "way too much salt" then at least you have a clue as to what the rest of the judges thought about your food.

I think that I owe you that courtesy.

Reply by QN:

So a KCBS score of 6 or "average" barbecue can also be defined as "suck-y" ? Wow, I had no idea. Never imagined something specifically defined as "average" could be considered "suck-y". So, should KCBS make a rules change to the word/definitions associated with each numeric value on the score card? If 6 is suck-y, then 7 is "better than suck-y" and 5 is then some margin of "less than suck-y"? Do you ever give a score lower than 6 since your scale already has that defined as "suck-y"? Here is a public example of a KCBS master judge and board member and how they score entries when they judge. Nothing personal Dave, but I have always had an issue with using the term "average" to define a numeric score for competition barbecue. No one has ever been able to explain what average is. Just a peeve of mine...
__________________

Okay, go back and read the sentence above what you put in bold. If all that you are cooking is "average" you're not going to get any calls - period! And to me not getting any calls sucks big time :mmph:.

And no, KCBS will not give a definition of the word average. What is average to you might not even be close to what's average to me or vise-versa.

Think of the friends or neighbors who compare your stuff to a chain BBQ restaurant. No matter whether they say that it's better than or almost as good as . . . they mean it as a compliment. The fact that you are insulted just proves that your definition of average is just a tad different than their's. They might think that ABC BBQ down by the river is pretty awesome grub (maybe even an 8 or a 9) while you think that it's a 4 on their best day.

So yes, I'm going to stick with my feeling that not getting any calls sucks and that getting 6s means that you won't get any calls.

Bratenmeister
05-13-2013, 11:38 PM
I don't believe it's possible for them [good teams] to catch a lucky table for every category at every contest.
[/LIST]

I don't either. But good teams don't win at every category at every contest.
And there is no denying, in my opinion, that there is some chance at stake,
and a good team will produce a great product but not get the score that it would, on average receive. Is there any team that disagrees? On average, however, they'll take away more trophies the more contest that they compete in. I guess that's my only point.

kenthanson
05-14-2013, 12:13 AM
One thing that gets tossed around on these damned judge bitching threads is the "$1000's of dollars spent". I personally could give a sh*t how much you spent to compete, that was your decision and you gotta deal with it. The judges have one job and thats to decide how good your food was, not how much sympathy they should have for you because you spent money on something. If you want to be judged on how much money you spent, put a hotrod in a f'n car show.

Outnumbered
05-14-2013, 02:17 AM
I agree, Kent. I did not mean to imply money should matter in the judging. All the teams spend a lot. I just want to know why it's a 6, if I get a 6. I would also like to know why I get a 9. That gives me something tangible to work with.

bbq.tom
05-14-2013, 05:31 AM
I agree, Kent. I did not mean to imply money should matter in the judging. All the teams spend a lot. I just want to know why it's a 6, if I get a 6. I would also like to know why I get a 9. That gives me something tangible to work with.

Until judges are required to list a comment on every score (like other organizations require) you will have no idea.
I've proposed numerous times that it would be beneficial for both the cook teams and judges to require judges to justify their scores.
Granted, this would require more time, but turn-in times could be extended to either 45 minutes between each or an hour.
That way it would take the guess out of the scores received.
Just a thought.

BABYGOTBUTT
05-14-2013, 06:38 AM
Here a a couple pictures of the box that I turned in.

boogiesnap
05-14-2013, 07:06 AM
well then. knowing judging IS subjective, i can't see how that box could go from 5-9 in appearance.

i'd be scratching my head myself on that one.

Lake Dogs
05-14-2013, 07:16 AM
Looking at that above, I could understand anything in the 7 to 9 range, but outside of that I really have no clue as to what they were thinking. Sounds to me like you ended up with **** for a table.

Badlands BBQ
05-14-2013, 07:34 AM
That's a 9 to me... Maybe you had a "smoke ring" judge who wanted a pronounced smoke ring... Nice box BGB...

EMTTLC
05-14-2013, 07:50 AM
Great thread. My perspective comes from cooking 34 contests and judging 12. I feel the problem could be addressed in the judging tent. In the current system a judge really doesn't know if he was the proverbial judge #6 or middle of the pack. The conversation after scoring might give him insight, but numbers are rarely discussed. I would like to see the scorecard changed to a "darken the appropriate number" system, where there is no identifiable handwriting. The bottom of each card would have judge identifying information. Prior to turning the score card into the rep, all scorecard would be placed in a laminated sleeve which holds all 6 cards and hides the judge information. This sleeve would the be placed on the table for judges to inspect. No one would know who scored what, but know how they scored compared to the other. The guy who gives all 9's when others are scoring 7's and 8's will most likely see that he is overly gracious, and the guy who gives 4's and 5's when others are giving 6's and 7's should realize that he is being overly critical. This would not be a hard system to implement and I feel that it could be beneficial. Any input would be appreciated, I think I might tweak this idea and see if the board would consider it.

Bratenmeister
05-14-2013, 07:53 AM
That is really an outstanding looking brisket box. Sadly, your brisket landed on a very bad judging table. There was a failure in fair and consistant judging for that box to recieve a 5 or a 6 in appearance.

Outnumbered
05-14-2013, 08:03 AM
Its obvious. Somebody juggled thosenboxes all the way to the turn in table. :-P

Yeah. You got hosed on the 5. The zealot judge.

MAP
05-14-2013, 08:05 AM
Same contest brisket box all 9 one 8 and one 6

QN
05-14-2013, 08:06 AM
Quote:
Originally Posted by Rookie'48 http://www.bbq-brethren.com/forum/images/buttons/viewpost.gif (http://www.bbq-brethren.com/forum/showthread.php?p=2479512#post2479512)
For me, if I give you a 6 then you're going to get a comment card. Yes, a 6 is defined as "average" but if all you're cooking is average BBQ then you'll never get a call much less a GC or RGC.

My personal feeling is that giving you a 6 means that your food is kinda "suck-y" and I should be willing to tell you why I think that way. If you get two 7s, three 6s and a 8 with no card you'll just scratch your head & say "WTF didn't they like about it?" If you get the same two 7s, three 6s and a 8 with a card from one of the judges saying something like "no meat flavor, only sauce" or "very, very mushy" or "way too much salt" then at least you have a clue as to what the rest of the judges thought about your food.

I think that I owe you that courtesy.

Reply by QN:

So a KCBS score of 6 or "average" barbecue can also be defined as "suck-y" ? Wow, I had no idea. Never imagined something specifically defined as "average" could be considered "suck-y". So, should KCBS make a rules change to the word/definitions associated with each numeric value on the score card? If 6 is suck-y, then 7 is "better than suck-y" and 5 is then some margin of "less than suck-y"? Do you ever give a score lower than 6 since your scale already has that defined as "suck-y"? Here is a public example of a KCBS master judge and board member and how they score entries when they judge. Nothing personal Dave, but I have always had an issue with using the term "average" to define a numeric score for competition barbecue. No one has ever been able to explain what average is. Just a peeve of mine...
__________________

Okay, go back and read the sentence above what you put in bold. If all that you are cooking is "average" you're not going to get any calls - period! And to me not getting any calls sucks big time :mmph:.

And no, KCBS will not give a definition of the word average. What is average to you might not even be close to what's average to me or vise-versa.

Think of the friends or neighbors who compare your stuff to a chain BBQ restaurant. No matter whether they say that it's better than or almost as good as . . . they mean it as a compliment. The fact that you are insulted just proves that your definition of average is just a tad different than their's. They might think that ABC BBQ down by the river is pretty awesome grub (maybe even an 8 or a 9) while you think that it's a 4 on their best day.

So yes, I'm going to stick with my feeling that not getting any calls sucks and that getting 6s means that you won't get any calls.

I agree with you. Getting 7s usually means you won't get calls either. Going back to the original post, the primary issue is inconsistent scoring. Until someone invents a machine to score barbecue you are going to just have to deal with scores given by humans who all have their own individual ideas of what looks and tastes average or excellent.

Brauma
05-14-2013, 10:19 AM
Until judges are required to list a comment on every score (like other organizations require) you will have no idea.
I've proposed numerous times that it would be beneficial for both the cook teams and judges to require judges to justify their scores.
Granted, this would require more time, but turn-in times could be extended to either 45 minutes between each or an hour.
That way it would take the guess out of the scores received.
Just a thought.

My initial thought was to agree with this wholeheartedly, but then I thought about all the comment cards I've received over the years. Not one has been useful. Not one!

The one that still tops the list is from the Inaugural Sams Club local event two years ago. We got a comment card that said, "You should not put sliced pork in the box". :loco: Um, yes you can. It's called a money muscle... :doh:

Luckily her score was thrown out (looked like female handwriting). All the other scores were 8's & 9's and we made it to Bentonville.

From what I understand a Table Captain cannot consult a judge who is scoring WAY off from the rest of the table. Best they can do is talk about it afterwards. But then the damage is done.

Rookie'48
05-14-2013, 10:35 AM
From what I understand a Table Captain cannot consult a judge who is scoring WAY off from the rest of the table. Best they can do is talk about it afterwards. But then the damage is done.


Correct, at least in KCBS comps. Talking to the judge about scores is something for ther Rep to do, not the TC.

Kit R
05-14-2013, 10:43 AM
What most judges do not understand is that a 7 is the kiss of death for a team trying to win. What stands out to me is that a judge that scores an entry 977 or 877 had a positive view of the box but did not like the final product for what ever reason and I look very hard at trying to adjust in some way to appeal to all. This judge knows how to count to 9. At the same time there is someone who sees the box appearance as a 6 or 7. For some reason the 6-7 judge never ups the taste and texture score to a higher score than his first number. It is like that is as far as he ever learned to count. I am very supportive of the judges and have a great relationship with many of them. They do a great job. It is the 5-10 % that I have a problem with . They have a direct negative impact on 24 people who just invested $1000 and 3 days of their life for a competition. I really wish these people who pass out the low scores that are inconsistent with the other judges would have to work a complete cook with a team before they could judge again. The day will come when KCBS addresses this issue. In my opinion , this is the single biggest flaw in the whole organization.

From the guy that got one 8 and five 9 s on appearance but two 977s on his brisket at this comp, I say yep. Sadly I didn't get a picture, but who would want to see an 18th place box anyway? :doh:

cottonquick
05-14-2013, 10:48 AM
Would you rather ditch the KCBS scoring system as it stands and go to a comparative system?

carlyle
05-14-2013, 11:21 AM
I like both boxes on this page - and babygotbutt, I think you were the victim of poor judging. Appearance should be the most consistent, but in your case was not. Because we were not in the judging tent, none of us know why.

The reps I know will catch this and talk privately with that judge - can't change what is already on the score card, but judge rehab can start right then.

As an organizer in charge of judges, this is the type of situation that I try hard to avoid
by presorting the judges and having assigned seating. Even with the best prep, however, it can happen. Just needs to be recognized right away and dealt with as best as possible.

Glad you bring this to the forum for discussion.

I do not want to see the system scrapped. I have confidence that the new scoring system will take a step in the right direction at reducing the chances that this continues to happen.

I really want to see continuing education/ refreshers for cbj's . That will help too.