A little lite number crunching

[ag80n]

We have decided to try to crunch some numbers, beyond just looking at the raw scores, to see if we can figure out where we might improve. I will outline below what I have done so far but I am also interested in suggestions for improvement or other ways I might look at the data.

What I have done so far:

Created a percentage rank for each contest’s overall placement and each meat and graphed it. The closer to 1st place the higher the percentage would be (a 1st place would be 100%). This number doesn’t really show much except whether we are moving into better positions or out. It does not show us if the opposition was tough or if our scores actually improved. The one thing this helps with is seeing if the 21st place out of 45 teams was any better placement than the 18th out of 40 teams.

The next calculation actually seems to be more useful. I took the top 10 teams’ scores overall and for each meat and averaged them. Then I subtracted our score from the average for an offset. What this showed me was how far out of the middle of calls we were. It mitigates somewhat the difference in high scoring contests vs low scoring ones. Let’s say the pork average for a contest was 172 and we were at 164 for an 8 offset but the next contest the average was 169 and we were at 163 for a 6 offset. Even though the score looks worse the offset is better and shows that we probably gained a little at what was just a lower scoring contest.

The finale task is to take appearance, taste and tenderness scores, drop the drop score, and create an average for each. I then will graph those averages with the scores to see what caused a score to go up or down. We have notes on recipes, timelines, and box pics that we can go back to to see just what we did.

Maybe this will help us. We have already noticed one thing in that we have actually trended up a little overall even though we didn’t feel that way.

Thoughts?

 

[Blanton]

Sounds like a lot whiz bang math to me and overly complicated. I don't consider my scores againest others, only myself.

I look to be in the 170's. If not, where did I loose points. First place to look is appearance. Less than all 9's and I am looking to correct next time.

After that, tenderness. One judge a 7 and I know it was a good turn in, oh well. More than 1, tells me something.

 

[kenthanson]

I think for your offsets of particular categories you would then have to take that number and find it to be a percentage of the higher score. The 6 on the lower scoring contest might look better than the 8 on the higher scoring contest but if the percentage that it is if the higher of the two scores then it might intact still be a worse result. Does that make sense?

 

[didisea]

Yes, it can help to see how you are doing relative to other teams, but ultimately, it comes down to cooking 3, 2, 1 BBQ, meaning you need to be shooting for 3rd-1st place in every meat at a contest.
It is good that you have recipes, timelines, and photos, because to me focusing on those things are more important. Consistency is one of the most important things to have. Analyzing where I've screwed up in contests usually involved trying to do something out of the norm, and caused a problem with the whole flow of the contest.
Also, planning for contingencies, like losing power, wind, weather, etc. is sometimes critical, because the teams that deal with adversity best can save their cook.

 

[enasnidx]

Interesting data analysis. Like Jeff, I hold myself more accountable to my own standards than comparing to others. I like to tinker with my data some as well, but what I look for is trends in my own scores. I have a spreadsheet that I enter my scores from each judge for every category. I have a different tab for every comp and a tab for a running average for the year. Here I can look to see why I may have done better or worse in T, T, or A in any given category at every comp and try to identify what may have influenced that. This helps me put focus on the areas of greatest opportunity. For example if I see a trend that my average tenderness score for brisket is down, I know what I'm working on first.

 

[hedge]

Damn you guys love your spreadsheets.

 

[ag80n]

I am not so much interested in comparing with others as seeing my own growth or failure to grow. By using the top ten average my hope is to remove the over or under scoring that can happen so I get a truer view of my own performance. For example in ribs the average top 10 score usually falls around 174-175 but we had one were the average was 171. Our score was lower that day but apparently so was everyone else's. If we had just looked at ours we might have been overly depressed.

The final destination is to get to the same place you go for Didisea. We want to be able to be more consistent. When you only cook 5-6 contests a year that can be hard to do without a little number crunching.

Hedge - I really dislike math and Excel but this wasn't too bad to put together because the formulas were easy and the data was all on the KCBS website. Mostly copy and paste. I almost enjoyed it- almost... no not really :shock:

 

[timzcardz]

We have decided to try to crunch some numbers, beyond just looking at the raw scores, to see if we can figure out where we might improve.
.
.
.
.
Thoughts?

My thought is that I don't need to crunch numbers to figure out where we might improve.

Anything that isn't consistently 9's is something that we might improve.

 

[Roo-B-Que]

You can also look to see how you did at your table. If you won your table then you did something right.

 

[Jorge]

You can also look to see how you did at your table. If you won your table then you did something right.

This is much more important than what your final category score was. How did you do in relation to those on your table. After that you can look to see how your table scored in comparison to others. When doing that you need to look at what the table did for all four categories to determine whether or not you might have landed on a table that scored high or low, out of the norm. If it was way out of wack one way or the other you know that you probably weren't as good, or bad, as your scores indicate.

Learn what you can, but don't get lost in the numbers looking for something that isn't there or believing in something that the numbers just don't support. It can be a great tool, or a great way to start chasing your tail.

 

[ag80n]

Learn what you can, but don't get lost in the numbers looking for something that isn't there or believing in something that the numbers just don't support. It can be a great tool, or a great way to start chasing your tail.

Yeah, I think I may have given the impression that we are digging real deep. That ain't what we are trying to do here ("lite number crunching"). Just, with our comps spread out so much, it's hard to see trends because we don't see them week to week. We may see that we have a low score but without looking back historically we don't know where we started going wrong. The danger is, because of the time between cooks, there are variables that shift in between cooks for us. I had to find a way to adjust for that if I could. With this I can say "oh here is where our chicken started going down in presentation or taste - did we change something?"

The idea of looking at tables is something we have done but ya'll have given me some new ideas. Just wish the reports were all digital. I don't intend to hand type anything.

Thanks for all the helpful comments!

 

[Funtimebbq]

When I read the original post, my first thought was that for any consistent analysis, the judges have to be the same. Of course that is not possible. But what is, is for KCBS to use the data it has to create tables of judges who's averages are as even a possible. I know, you're all say, there he goes again.
All that is required is for an organizer to submit their judge's membership numbers to KCBS and for someone there to run a program that will average those judges historical scoring so tables can be assigned as equally as possible. But they won't even do that for the Sam's Club contests where they already know the judges months in advance. I don't get it and maybe the Reps. on the KCBS BoD can explain it. From what I read in the Jan. BoD minutes, they are the people with reasons it won't work and stopping this from happening.

 

[shares]

My CalBQue site has a KCBS score summary tool for free (as in free, not even ads). You type in KCBS score numbers, it's optimized - fast and easy as possible.

Then, you can mash up your scores with several options to help connect the dots. Copy/paste into your spreadsheet to compare with previous contests. Maybe some find it useful, I welcome feedback too.

 

[Pappy Q]

If you can unlock the mystery of KCBS scoring and determine any consistent useful information.....well, good luck.

 

[EngineeredBBQ]

Not to try to discourage you but I used to look at the numbers a lot too. Then I realized r that you can score a 180 one week and a 167 the next with the same recipe, same meat so the numbers don't necessarily mean much. (If you do that too much you might have a polarizing flavor but low scores happen sometimes, my chicken was like that last year some loved some apparently didnt'). Now that l just look at table position and overall positions to gauge how I am doing.

 

[ag80n]

Not to try to discourage you but I used to look at the numbers a lot too. Then I realized r that you can score a 180 one week and a 167 the next with the same recipe, same meat so the numbers don't necessarily mean much. (If you do that too much you might have a polarizing flavor but low scores happen sometimes, my chicken was like that last year some loved some apparently didnt'). Now that l just look at table position and overall positions to gauge how I am doing.

I get what your saying and that is what we do most of the time and will probably do from here on out. This is just maybe a one time thing. We had noticed our scores slipping and couldn't figure out where the holes were. I think we have found several which I will post in just a few.

 

[ag80n]

So here's what we have learned from graphing scores and comparing pics and recipes from the different cooks to the numbers they produced. Most of this is stuff we all know but sometimes fail to see until we look back over time:

1. we have really let our box prep slip. We thought they looked good but comparing the good boxes from a year and a half ago when we were often all 9s in appearance to now tells a different story.

2. my pork tenderness scores had slipped because of a slight change in how I was finishing them and its obvious that it happened when I changed although I couldn't see it looking at just one comp.

3. pork appearance was also possibly affected by a change in trimming

4. overall taste and tenderness scores have been going up but these little increases were masked by the issues with appearance

Although we are still looking at things, we have a general idea of what has been going on. The good thing is we didn't find any major issues with anything. Just little attention to detail stuff and tweaks to make.

 

[enasnidx]

I'm probably abnormal in that I actually enjoy the number crunching stuff. I use bbqdata.com for quick reference to my average scores and comp results, but I still like to mess around with my spreadsheets. I don't lose sleep or make game plan changes over a few points here and there that I may be off, but I enjoy seeing my progress over time and of course watching for trends that may need to be addressed over the course of the season.