MMMM.. BRISKET..
The BBQ BRETHREN FORUMS.  



Our Homepage Donation to Forum Overhead Welocme Merchandise Associations Purchase Subscription Amazon Affiliate
Go Back   The BBQ BRETHREN FORUMS. > Discussion Area > Competition BBQ

Notices

Competition BBQ *On Topic Only* Discussion regarding all aspects of Competition BBQ. Experiences competing or visiting, questions, getting started, Equipment, announcements of events, Results, Reviews, Planning, etc. Questions here will be responded to with competition BBQ in mind.


Reply
 
Thread Tools
Old 07-15-2013, 11:33 PM   #76
G$
is Blowin Smoke!
 
G$'s Avatar
 
Join Date: 01-01-05
Location: Southern Arizona
Default

Quote:
Originally Posted by gettinbasted View Post
There was also a "money" table at wellsville. 3 of the 4 category winners came off of table 8. I'm guessing we won't be arguing to have those judges reeducated!
So.... If the best four turn ins happen to arrive at the same table, wouldn't we expect it to be, what you have termed a "money" table?
G$ is offline   Reply With Quote


Thanks from:--->


Old 07-15-2013, 11:35 PM   #77
G$
is Blowin Smoke!
 
G$'s Avatar
 
Join Date: 01-01-05
Location: Southern Arizona
Default

Quote:
Originally Posted by Vince RnQ View Post
So Kelly, as a Contest Rep (and a damn good one), what do you think of the idea of sorting tables based upon a CBJ's historical scoring average instead of experience level?
How many contests are needed to make that number statistically meaningful?
G$ is offline   Reply With Quote


Old 07-16-2013, 01:21 AM   #78
CBQ
is Blowin Smoke!
 
Join Date: 01-11-09
Location: Cumberland, RI
Default

Quote:
Originally Posted by BigZ View Post
This is going to be a newby comment so bare with me, (I have never been in a competition yet, next month will be first) with out knowing what the "old" sheet was I see that the new one can help you/kcbs figure out which Judge you don't want to have or that needs more training. Since you don't get to choose your table and the same Judge's don't go to all of the comps, how does this "help" you as a cook to improve with out the comment cards? I can see knowing your score can tell you what area you were good or bad in though.
We don't really know what, if anything, KCBS will do with regard to using KCBScore to change judge behavior. Maybe nothing. What it DOES do is give the cooks a lot more information about their performance. Did you have a problem with your product, or did you hit an unlucky table? While the new system won't give you a conclusive answer, it gives you more insight into how you really did and what the judges might be thinking.
__________________
Chris
Rhode Hog BBQ
"The Most Dangerous Team in BBQ" - Jon Vallone
KCBS, KCBS CBJ, NEBS
CBQ is offline   Reply With Quote


Thanks from:--->
Old 07-16-2013, 07:40 AM   #79
gettinbasted
is One Chatty Farker
 
Join Date: 09-14-12
Location: Springfield, MO
Default

Quote:
Originally Posted by G$ View Post
So.... If the best four turn ins happen to arrive at the same table, wouldn't we expect it to be, what you have termed a "money" table?
It is absolutely possible that the "money" table had the top entries in each category and that the "table of death" had terrible food across the board. At an 8 table contest, however, I would consider it unlikely. With only 5 or 6 entries hitting each table per category, any conclusion drawn can be dead wrong.

It is what it is. We are all aware that a good or bad table can exist when we turn in our food. That is BBQ, and as Boogiesnap pointed out, it will even out over the course of a contest and a season.
__________________
Gateway Drums - THE Insane Can Posse - gettinbasted.com
gettinbasted is offline   Reply With Quote


Old 07-16-2013, 08:06 AM   #80
gettinbasted
is One Chatty Farker
 
Join Date: 09-14-12
Location: Springfield, MO
Default

Quote:
Originally Posted by G$ View Post
How many contests are needed to make that number statistically meaningful?
I don't advocate any kind of judge training program based on their score history. G$ makes the perfect point as to why. There just won't be enough data. Judge reeducation is a dangerous path to go down.

BBQ judging is subjective. I have received a 180 and a 160 on essentially the same piece of meat at two competitions. The truth was probably somewhere in the middle. I would love to have that 160 be a 170, but I don't want to give that 180 pin back!
__________________
Gateway Drums - THE Insane Can Posse - gettinbasted.com
gettinbasted is offline   Reply With Quote


Thanks from: --->
Old 07-16-2013, 08:29 AM   #81
BMerrill
Full Fledged Farker
 
Join Date: 02-17-09
Location: Ayden, NC
Default

At Greenwood, SC, out of the top 10 overall teams, only Lotta Bull hit the same table twice.
BMerrill is offline   Reply With Quote


Old 07-16-2013, 08:55 AM   #82
musicmanryann
is One Chatty Farker
 
Join Date: 06-11-08
Location: Ames, Iowa
Default

I'm with Vince. At our contest this weekend there was clearly and bad table and a great table (from a cooks perspective ;-)). Fortunately for ours most, if not all, hit every table once. I have been saying for a couple of years now that sorting judges based on how they actually score, rather than on the number of contests they have competed would be a much more reliable method for equalizing the scoring across tables. You have judges that score high, score low, and score middle of the road no matter their experience level. Experience does matter a great deal, but there are still bad judges with a lot of experience and good judges with little experience.

It seems this new scoring system would open the door for tracking and sorting judges based on their variance from the mean. The question is whether or not the new system is currently keeping track of judges' scoring across multiple contests.
__________________
Big T'z Q Cru Championship BBQ Team
CBJ #50801
6 WSM's
2 Weber Kettles
FEC100
2 Backwoods Fatboy
2 Not-so-Ugly Drum Smokers from Gateway BBQ Store
[URL="http://s674.photobucket.com/albums/vv102/Ryannewstrom/Jambo%20--%20The%20Orange%20Monster/"]Jambo -- The Orange Monster[/URL]
[URL="http://www.bigtzbbq.com/#!__main/links"]www.bigtzbbq.com[/URL]
Join our [URL="http://www.facebook.com/home.php?ref=logo#/group.php?gid=101606657515&ref=ts"]facebook group[/URL]
musicmanryann is offline   Reply With Quote


Thanks from: --->
Old 07-16-2013, 09:34 AM   #83
Muzzlebrake
Babbling Farker
 
Muzzlebrake's Avatar
 
Join Date: 09-22-06
Location: Pleasant Valley NY
Default

I'm going to use this data to help me make adjustments to things within my control. This past week for instance I know my ribs and pork tanked. I didn't really like either one but it didn't think they were as bad as they placed. Looking at the scores I can now see my ribs hit a low scoring table and the pork just tanked on its own merit.

I would have thought the judges scoring statistics would be very useful to them just as Tigerpaw mentioned.
I'm not sure that we need to seat judges according to their scoring averages to even out the tables though. In our contest this weekend, the top 10 overall I think 7 of the teams had hit the low scoring table at least once so it seems to have balanced out in the end. On the other hand, the GC didn't hit it once and won by 10 points, 2-4 all hit that table and were only separated by 5 points.

Who knows? I think more data is definitely need before any adjustments are made. The rest of the year should give you a good sample range to work with.
__________________
Sean Keever

"What sort of people are these charcoal masters? They behaved badly and were unconcerned with appearances. Their hair was long and unkempt and their clothes were wrinkled and old. They drank beer to and from the crab house and they made rude noises while we cooked." Tao of Charcoal
Muzzlebrake is offline   Reply With Quote


Old 07-16-2013, 09:41 AM   #84
mobow
is one Smokin' Farker
 
Join Date: 12-11-09
Location: Marshall MO
Default

You also have to keep in mind that the low table may be low or it may be accurate. The high table may be accurate or it may be inflated. More information can be good but it can also increase the odds of a faulty conclusion. Keith
__________________
WSM 22.5, MCBJ, In The Key Of Que
mobow is offline   Reply With Quote


Thanks from: --->
Old 07-16-2013, 09:45 AM   #85
Outnumbered
is Blowin Smoke!
 
Join Date: 09-18-11
Location: Smithville, Missouri, United States
Default

Quote:
Originally Posted by G'pa Herb View Post
I was a table captain at Holbrook, and I believe that the CBJ % was less than 65%. There were at least 3 VIPs at most of the tables. On my table, there were 3, so to help the teams and scoring, I held a mini judges class before the judging began, and then again before each meat turn in. The non CBJs scoring was right in line with the CBJs on my table, +/- a point here and there.

The new score sheet is very revealing, judges are no longer anonymous, and names could be easily placed to tables and seat numbers by those in the judging area.

My advice would be to not use Holbrook as a baseline for the new scoring system because of the low CBJ percentage. Just for the record, even with only 5 tables, the Table Captains took the time to make sure no table judged the same teams entrys twice.
I commend you for that and this is not a critique of how you handled. And thank you for handling it this way as it seems to be the best way to do it. Just using it to illustrate my point.

Assume that the non-CBJs score a point lower than the rest of the table for whatever reason. They throw the lowest out, so we can assume it's the low score, so you have two non-CBJ who score a point lower on across the board on taste from a purely arbitrary reason (at least theoretically here). You now have lowered the scores from that table by 5 points per category, per team. That's a 20 point swing for all 4 meats and can mean the difference between a GC and middle of the pack finish for overall.

Of course, the same can be said for the other extreme as well. Maybe you did a middle of the road cook, but you hit the right table and you mediocre stuff got GC.

But as gettin'basted says, this new system at least gives you the information to know if your entry was bad or if your luck was.

Thanks to KCBS for making this change. I think it's great.
__________________
Greg Vincent

When I die, my goal is to have a visible smoke ring when the coroner cuts me open for the autopsy.
Outnumbered is offline   Reply With Quote


Old 07-16-2013, 09:49 AM   #86
Alexa RnQ
is Blowin Smoke!

 
Join Date: 09-12-07
Location: the Ninth Ring of Hell, cleverly disguised as Phoenix
Default

Quote:
Originally Posted by mobow View Post
You also have to keep in mind that the low table may be low or it may be accurate. The high table may be accurate or it may be inflated. More information can be good but it can also increase the odds of a faulty conclusion. Keith
I think tracking across all categories and an accumulated body of several contests will reduce the faulty conclusion. A judge may land on a lucky table that gets all great entries, or on a poor table that gets all scary entries, but the odds of that happening in all four categories in contest after contest over time are pretty remote.
__________________
.
Don't practice until you get it right; practice until you can't get it wrong.
Don't overthink or undercook.


[url=http://www.rhythmnque.com]Rhythm 'n QUE[/url]
Alexa RnQ is offline   Reply With Quote


1 members found this post helpful.
Old 07-16-2013, 09:57 AM   #87
bignburlyman
is one Smokin' Farker
 
bignburlyman's Avatar
 
Join Date: 09-10-03
Location: Great Bend, Kansas
Default

With all the talk of now being able to see more data and more of the tables scores, and how this may help balance the judges better, no one has mentioned the 800 pound gorilla in the room---the judges will most likely never see any of the score sheets. How will this help any judge to realize they are too harsh or too generous unless KCBS taps them on the shoulder 5 contests down the line and say "Your scores do not fit the mean average of the other judges"?

There are a lot of great judges out there, and there are a few total dooshbags, most of my contests judged are in Kansas, and I know a couple of judges that I prefer to not sit at the same table with them. Now expand that couple of judges nationwide and there really are quite a few that are not that great. Someone once told me to remember that even though medical doctors are mostly very well respected, half of them graduated in the bottom 50% of their class.
__________________
David
MCBJ & CTC #5112

Traeger Texas 075
Blackstone Tailgater


Family: Where Life Begins, and Love Never Ends.

bignburlyman is offline   Reply With Quote


Old 07-16-2013, 10:25 AM   #88
Slamdunkpro
Babbling Farker
 
Slamdunkpro's Avatar
 
Join Date: 02-27-07
Location: Northern VA
Default

I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.
__________________
Aporkalypse Now Competition BBQ team
Thanks to our sponsors: Wusthof Trident USA, Creekstone Farms, Myron Mixon Smokers & The BBQ Guru
MABA 2014 Team of the Year
Spicewine Tandem - "The Beast"
, 2 Vulcan V33's & 1 V9
Brinkman modified upright, Weber Kettle(s); Meadow Creek Pr-24, Super Stealth Mode black & Night Vision Red Thermopens
-----------------
MABA - (Mid Atlantic BBQ Association) President
KCBS Certified Judge - #23289
Slamdunkpro is offline   Reply With Quote


Thanks from:--->
Old 07-16-2013, 10:31 AM   #89
gettinbasted
is One Chatty Farker
 
Join Date: 09-14-12
Location: Springfield, MO
Default

Quote:
Originally Posted by Slamdunkpro View Post
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that from awards since if you consistently get scores over the judge's mean average you should be getting walks.
Agreed. Just another thing to discuss over a beer after awards.
__________________
Gateway Drums - THE Insane Can Posse - gettinbasted.com
gettinbasted is offline   Reply With Quote


Thanks from:--->
Old 07-16-2013, 10:45 AM   #90
Muzzlebrake
Babbling Farker
 
Muzzlebrake's Avatar
 
Join Date: 09-22-06
Location: Pleasant Valley NY
Default

Quote:
Originally Posted by Slamdunkpro View Post
I guess I'm in the minority who think that all this table info is as useful as the history boards on roulette wheels. It's fun to use to play "what if" but otherwise statistically irrelevant. Regardless of what the score sheet says your table history at any contest is always N=1. In other words your history starts from scratch every new contest. Different judges, different table combinations different tastes. This is especially true if the judge average score is also N=1 (just the history for that contest) or if this a combined score average across all 4 meats.

If the judge's average score is cumulative then what does that really tell you? A judge averages 30.2 and you got a 32.5 from them. OK so your flavor profile hits well with that judge, but you already knew that's from awards since if you consistently get scores over the judge's mean average you should be getting walks.
So you don't see any value in knowing that the product you turned in scored where it did due to it hitting either a good/hot table or bad/cold table. I think everyone has had that turn in they thought was great that tanked or sucked and they walked and wondered why. Granted I don't think there is much you can do about it but I see value in at least knowing that.
__________________
Sean Keever

"What sort of people are these charcoal masters? They behaved badly and were unconcerned with appearances. Their hair was long and unkempt and their clothes were wrinkled and old. They drank beer to and from the crab house and they made rude noises while we cooked." Tao of Charcoal
Muzzlebrake is offline   Reply With Quote


Thanks from:--->
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


Forum Custom Search: Enter your Search text below. GOOGLE will search ONLY the BBQ Brethren Forum.
Custom search MAY not work(no display box) in some configurations of Internet Explorer. Please use compliant version of Firefox or Chrome.







All times are GMT -5. The time now is 04:28 AM.


Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
2003 -2012 © BBQ-Brethren Inc. All rights reserved. All Content and Flaming Pig Logo are registered and protected under U.S and International Copyright and Trademarks. Content Within this Website Is Property of BBQ Brethren Inc. Reproduction or alteration is strictly prohibited.
no new posts