Analyzing Scores
 

So, I've started down a path of analyzing our scores for the year (and last year to check for trends) based on the individual judges scores, as opposed to the larger number. I'm analyzing 9,8,7, etc from each judge for each contest we cooked. I'm wondering if anyone else has done this, and any tricks or insights people have taken from it.

The numbers, naturally, are averaging out into the 8.xxx for everything, which is to be expected. Obviously higher is better, but I'm trying to find some insights on if anyone looks at this this way, or if I am (probably) getting too deep and not seeing the forest for the trees.

What do you guys do to evaluate scores? I know there's regional differences that keep a comparative standpoint from being useful, but just some generalities would be nice.

Man, winter is kicking in before I'm even done with this season!

FWIW, all I have now is a blunt averaging of the scores, and then plugged into the KCBS weights to give me the average score from a judge, and then a line multiplied by 5 to give me an "average score" when you don't drop the low scoring judge.

It's not complete, and not robust enough to draw conclusions, that's for sure, but it's interesting.

I will email you my spread sheet if you want...it is pretty insane but it gave me some useful information.

I would love to see your spreadsheet, if you don't mind. I'm a sucker for a good spreadsheet. I'll pm you my email address.

If possible I would also like an email. Thanks!

I've done similar analysis in the past. The one thing it did was drive me crazy! It basically shows is that you need 9s to do well. What makes one judge give an 8 vs. different person giving you a 9? I couldn't find that answer. One example I had was the last contest of 2018 for me. I looked at my score sheet and out of all the numbers on that page, there were only two 7s. Everything else was and 8 or 9. Sounds good but was 15th out of 47 teams.

I looked at your last contest. If one single judge had changed their taste score from an 8 to a 9, you would be GC instead of RGC. It is literally that close of a game most weeks. Things are decided by one or two people giving an 8 instead of a 9.

This gives me a headache...…..:cry:

Yeah, we had agonized over the 8s we got in taste for that reason. But, we lost to a more than worthy opponent in Brad, and we needed that 180 in chicken. We've struggled with it for a while, so getting that score was awesome.

I agree with the sentiment that it's about 9s. Especially on taste/tenderness, which also go hand in hand. a 699 is basically the same score as a 989, which is something that we've been trying to keep in mind.

I realize it's mainly an exercise in futility, but it's something that I'm curious about and trying to analyze our scores better.

I don't know guys... As a CBJ and a cook, I'm not sure how much information you can really get from the numbers. Unless a judge actually is able and willing to take the time to provide you with specific feedback, there's not really too much you can get from just a number.

Especially on Taste/Tenderness.

For example, let's say I get a rib turn in and I bite into it: Perfectly done from a tenderness aspect but the flavor is way off. It tastes like I just licked a saltlick. I score it a 6.

Unless I write a comment card that says "Waaaay too salty".... How is the cook going to know why (s)he got a 6? What are they going to adjust?

Now, I try to write as many comment cards as I can, but things move fast in the tent and you just don't have the luxury of being able to really help the cook very often.

So really, what useful information can you glean from just the raw numbers?

Maybe it won't tell me much. Just trying to come up with ways to up my game, and the way my mind works, I just try to go to stats.

You might see if some of the local teams are still having a early spring practice cook. They had CBJ's judging and then giving the team direct feed back on why they scored the entry the way they did. I never did one, but heard a lot of good things about the results.

I did one with Keith Sisk in 2018. It was really good experience and lots of good feedback. A comment card from every judge for every entry, plus, you got your comment card handed to you by the judge and they explained their score and you got to ask questions. It definitely upped our game.

There are enough unknowns that it can become a rabbit hole in a hurry.

I'd focus on a couple of things. How did you do in relation to the other entries on your table? What did teams known to be consistently strong, in that category, do? I would NOT assume that just because a judge has a lower scoring average that they are a low scoring judge. A judge that's using a greater range is going to have a lower mean score than the 8-9 judge. (I'd love to see standard deviation shown along with their scoring mean) Was there an entry on the table that scored significantly lower than the others? It's possible that entry negatively impacted subsequent entries due to creosote, excessive seasoning, or a list of other possible reasons.

You are dealing with a lot of variables, plenty of unknowns, and a small data set. That being said, I think you can discern some clues but need to be cautious to not manufacture something that isn't there.

Clear as mud?

Scoring sensitivity
 

Ap Ts Tx 1 score 5 scores 4 meats
9 9 9 36.0000 180.00 720.00
8 9 9 35.4400 177.20 708.80
7 9 9 34.8800 174.40 697.60
9 9 8 34.8572 174.29 697.14
6 9 9 34.3200 171.60 686.40
8 9 8 34.2972 171.49 685.94
9 9 7 33.7144 168.57 674.29
9 8 9 33.7028 168.51 674.06
8 8 9 33.1428 165.71 662.86
7 9 7 32.5944 162.97 651.89
9 9 6 32.5716 162.86 651.43
9 8 8 32.5600 162.80 651.20
8 8 8 32.0000 160.00 640.00

Don't know if this will help anybody else, but it opened my eyes a bit.
We all know that Taste is weighted more than Texture and Texture more than Appearance, but until I made this table it really didn't hit me.
A 170 gives you a good chance to be in the top 10 and a 175 or more gives you a chance to win. In order to get 170, Taste has to be a 9 and Texture has to be an 8 or 9. You can have a 6 on Appearance and still have a chance to place.
Might be obvious to most competitors, but it made me realize, looking back at my scores, that I needed to get better on Taste and more consistent on Texture before anything else.