PDA

View Full Version : A different perspective on the new “Score Program.”


Smoke'n Ice
07-16-2013, 07:35 AM
Early on, we had a 3 point scoring system with the judges being told to start at 9 and grade down. Because of concern from cooks and judges that it was unfair not to use the full point system, the BOD implemented a different start point and assigned definitions to each of the numbers. Initially, this had the desired effect of moving more toward a 5 point system.

Gradually, the system, with seasoned CBJ’s, has shifted back to a 3 point system because of places like this board and others that stigmatized the use of a number below 7 as unfair to our “great” bbq turn-in.

The new “score system” points out the judge that use a full 9 point system because they don’t “know” better, and the experienced judge who only use a 3 point system. The real question is “Which one really needs to be educated?”

I know that my bbq turn-in’s do not always deserve a 7-8-9 and am surprised at times when I get them. I place more faith in the new or VIP judge who gives an honest opinion than the seasoned judge who gives me a 9 for dog food.

How do I know which was which at a table? It is simple, look at your old score cards and you can spot the newbie or VIP vs the experienced as the former used a full 5 point system. The new “Score Card” will make this even more obvious to everyone and allow the reps to keep the “seasoned” judges from congregating together based on historical information and the rep can EMPHSIS the point system, starting point and definition and encourage the seasoned judges to consider it as well.

Muzzlebrake
07-16-2013, 09:56 AM
Come to the northeast, there is more than a 3 point system used. In fact at some contests it's not that uncommon to have a 6 somewhere in the score and still finish in the top 3. If you are in the top 3, you are certainly not average.

I also don't think that the food that experienced cooks turn in is average and most of the time it is going to be a 7, 8 or 9. I also think that the margin of difference between an 8 & 9 most of the time is very very small and its not hard to just over or under shoot you meats and they end up as a 7.

All in all I think the judging is much more accurate then we give it credit for and even if we don't want to admit it, there are lots of cooks out there deserving 7's, 8's & 9's. I also think we don't really tend to hear for the folks that the get 5, 6 & 7's consistently. I just don't think those are the people spending as much time on BBQ forums as the 7,8,9 crowd and they also just don't want to talk about it as much.

Outnumbered
07-16-2013, 10:04 AM
I agree with you on the lack of a full scale being used and would fully support the push to get the full scale used.

I guess I don't see that this score reporting impacts that one way or another, however.

Ron_L
07-16-2013, 10:08 AM
This goes beyond the scoring system. It is really about judge training and re-training. My wife and I were certified by the same instructor, but two years apart, and the way that the scoring system was described was completely different. Without re-training or recertification for judges there is no consistency.

Vince RnQ
07-16-2013, 10:14 AM
I have always believed that a judge who only uses 7, 8 or 9 is just as dangerous as a judge who only uses 5, 6 or 7.

boogiesnap
07-16-2013, 10:18 AM
i'd have to agree with sean in regards to the northeast. i know for a fact that judges here are not afraid one bit to hand out 5 and 6's for food that i would consider to be pretty good(not even mine mind you).

Ron_L
07-16-2013, 10:23 AM
i'd have to agree with sean in regards to the northeast. i know for a fact that judges here are not afraid one bit to hand out 5 and 6's for food that i would consider to be pretty good(not even mine mind you).

Regional differences are also an issue. Honestly, I've seen pictures of boxes from other regions of the country that were category winners or even 180s that would even get a 6 in appearance around here. But, as long as there is consistency within a single competition, that doesn't matter as much to me. I'll probably never cook a competition in the Northeast or the Pacific Northwest, so comparing the scores there to the scores in the midwest doesn't really do much for me. But, it may to teams who travel the country.

Muzzlebrake
07-16-2013, 10:50 AM
I agree with you on the lack of a full scale being used and would fully support the push to get the full scale used.

So are you saying there should be more 3's, 4's and 5's?

tugbone
07-16-2013, 11:31 AM
We just competed in troy comp and scoring was all over the place.

Rich Parker
07-16-2013, 11:34 AM
In the upper midwest it is not uncommon for presentation to see plenty of 6's and 7's and most likely you will be under 14 total 9's out of 24 presentation scores. When I went down to Gadsden, AL, I got 21 total 9's in appearance which is unheard of up here.

INmitch
07-16-2013, 12:11 PM
i'd have to agree with sean in regards to the northeast. i know for a fact that judges here are not afraid one bit to hand out 5 and 6's for food that i would consider to be pretty good(not even mine mind you).

That's a no chitter! @ Lake Placid I know on cow the first three judges gave me 989, 855, 999 with the other 3 somewhere in between. I did get a comment card for 755. The thing that was disturbing is the judge wrote 755 on the coment card and I didn't have a 755 on my score sheet. I had an 855. Did the judge write the wrong score on the comment card? Or did the wrong numbers get entered?

Outnumbered
07-16-2013, 12:25 PM
So are you saying there should be more 3's, 4's and 5's?

Bit of a leap, I'd say. I would, however, encourage full scale use. If 6 is the, mid point, that should be the score for the average entry. Right now are you sure the 8 is just OK, or is it really exceptional? Depends on the judge and you still really don't know that under this new system.

Q-Dat
07-16-2013, 02:47 PM
IMHO as soon as a judge has judged enough BBQ that his or her standard for what is good BBQ goes up more than a little, that judge needs to either stop judging, or take a long break from it.

Some cooks complain about judges being taken from the general public at IBCA comps because most folks don't really know what good BBQ even is, so they end up being too generous with their scores. I think its just as bad to have judges who have judged so much that they consider themselves a connoisseur of what BBQ should be.

Now I have no doubt that there are folks who could judge 40 contests per year and never let it affect their standards, but there are many who would.

bbq.tom
07-16-2013, 03:57 PM
How often do you look at the COMPLETE list of competitors at a contest and their scores? You will see PLENTY of scores in the 140s, 150s and low 160s, MEANING that judges are giving scores lower than just 7-8-9.


As a judge, it seems that MOST of what I get at the table is at least above average and better!
Maybe I'm just lucky to be judging great teams, but believe me, I've had below average (and worse) barbecue and I know the difference! I've only been judging for 5+ years (60+ contests), but I've eaten and reviewed over 500 Q-joints in the USA. Generally speaking, competition barbecue is at least above average compared to barbecue in general.

bbq.tom
07-16-2013, 04:12 PM
Maybe the "Average" score should NOT be "6", but should be "4". That way you would have only 3 scores "below average" or worse, and 5 scores "above average" or better. Would truly seperate the scoring spread a bit more.

BogsBBQ
07-16-2013, 04:16 PM
Come to the northeast, there is more than a 3 point system used. In fact at some contests it's not that uncommon to have a 6 somewhere in the score and still finish in the top 3. If you are in the top 3, you are certainly not average.

This is a good point. When we won in NH, we had 6s. We also had two negative comment cards...in two different categories. :doh:

Smoke'n Ice
07-16-2013, 06:59 PM
How often do you look at the COMPLETE list of competitors at a contest and their scores? You will see PLENTY of scores in the 140s, 150s and low 160s, MEANING that judges are giving scores lower than just 7-8-9.


As a judge, it seems that MOST of what I get at the table is at least above average and better!
Maybe I'm just lucky to be judging great teams, but believe me, I've had below average (and worse) barbecue and I know the difference! I've only been judging for 5+ years (60+ contests), but I've eaten and reviewed over 500 Q-joints in the USA. Generally speaking, competition barbecue is at least above average compared to barbecue in general.

In Rio Rancho several weeks ago, I turned in chicken which I did not like. It had a flavor that was, for want of a better word, wild. I have used free range chicken, certified organic all year and have noted the same flavor at least twice before. I plan on making a change this coming weekend but don’t know what yet as business got in the way. Anyway, my scores were 998, 887, 878, 977, 999, 876 for a total of 160.5714. I can assure you that the 876 was a generous score that I would not have given. Appearance scores I don’t have a problem with because of the “DAMN I’M GOOD” syndrome :biggrin1: but the rest, in my humble opinion were too high. Yes, we are our own harshest critic but it was that bad. Even with these scores it placed 17th out of 42 teams and probably should not have been.

The point I’m trying to make is the other categories came in at my expectation, but the chicken was out of line. I know that if I get 7’s in any category other than appearance, I’m out of the money, so giving a 7 to make me feel good is just wrong, give me the 4 or 5 it deserved and I can learn from that.

CBQ
07-16-2013, 07:54 PM
I’m out of the money, so giving a 7 to make me feel good is just wrong, give me the 4 or 5 it deserved and I can learn from that.

Come on up to NEBS land and our judges will be happy to school you with a bunch of 4's. :becky:

They aren't afraid to use the whole scale, but I have to say they generally get it right. We saw our share of those 4's and 5's when we started, and now not so much. We got better and the scores show it.

It is a regional difference, we saw our scores pop up when we hit the road and went to The Royal, Sam's National, and The Jack last year, but those higher scores didn't translate into a higher finish. It was just regional differences in scoring.

CBQ
07-16-2013, 08:03 PM
Right now are you sure the 8 is just OK, or is it really exceptional? Depends on the judge and you still really don't know that under this new system.

KCBScore still won't tell you what the judge was thinking, but since it DOES tell you the average score for that judge, you may gain some insight into that "8", knowing if it's close to the judge's average or not.

You could have hit a table that was all bad/all good, so as always you probably still need at least 3 contests to draw a conclusion about where you really stand.

Muzzlebrake
07-16-2013, 09:12 PM
Bit of a leap, I'd say. I would, however, encourage full scale use. If 6 is the, mid point, that should be the score for the average entry. Right now are you sure the 8 is just OK, or is it really exceptional? Depends on the judge and you still really don't know that under this new system.

I didn't mean on your sheets, I meant in the overall scheme of things. As I alluded to before, I think there may be more low scores given than we think. At out last 2 contest in Lake Placid and Troy 11/28 and 11/32 teams scored under 600. You can rest assured these teams had plenty of 6's, 5's and even 4's. I tend to think these aren't the folks that you are going to see spending time in online forums tring to hone their craft. On the other happened I think it's reasonable to think that many experienced cooks that devote a lot of their life to improving their cooks, turn in food that is better than "average" the majority of the time.

Outnumbered
07-16-2013, 11:41 PM
^^^Sean, I believe we're on the same page. My point is that when you have TCs telling judges, as was the case with me in the one contest I've judged, that anything below a 6 requires a comment card, you are likely going to get judges who score nothing below a 6. No matter how bad it is, you'll see nothing below a 6 so they don't have to fill out a comment card. That doesn't mean they're wrong, it just means they don't want to have to say, "Hey, Mr. Cook, that entry was just plain bad."

I was told the cooks spend a lot of money and time to do their entries so you need to score high as a result. If a judge doesn't hear that or if they want to use the scale that's given to them, they may give a 6 for an entry that is average...or they may give a 6 because the entry is horrible.

I've gotten a few 6s at contests, and those are always along with more 9s. My only question to that is , "How am I supposed to know if that 6 was a horrible entry according to that judge, of if it was average."

In the case of the new score reporting, I think I can get that. I just don't believe this new reporting system will keep judges from using a 3 point scale.

I hope I'm wrong, however. If, as was mentioned in the other thread, we get a tracking system and some education to train judges on how they should score, I may be. In that case, I will gladly wear the egg on my face.

chromestacks
07-17-2013, 10:15 AM
We just competed in troy comp and scoring was all over the place.


It was for us too. A lot of the teams that have previously won did not score well. I do not believe it was coincidence. Last year we were 6th in Brisket, 8th in pork, 15th in ribs, 25th (?) in chicken and 14th over all. This year we were at the bottom of the pack in every category. Table of death? We were on table 6 two times! many of the table 6 teams finished at the bottom. For us chicken was probably scored accordingly (never seem to get it dialed in, tried a new technique) but, my pork is always top 10 CONSISTANTLY! It finished DAL! Once again....Table 6.....I am not going to change a thing except maybe skipping this contest in the future for a few years