PDA

View Full Version : Want to know your Table Number?


Brewmaster
05-10-2011, 09:15 AM
I was talking to a friend of mine about the Scoring Detail we receive after a contest. He had an idea, I thought was a very cool idea, and I want to know what you all think.

Some of you are more familiar with the judging process than I am, so I don't know if it is even possible. He thought it would be nice to see the table number our turn-in made it to.

I thought this would be very beneficial in determining if a certain judge scored each turn-in low or if they just didn't care for my entry. I also think it would kind of neat to see who's turn-in was on the same table.

What are your thoughts?

Cheers,
Nate

mobow
05-10-2011, 09:21 AM
The table would be fairly easy to include if the program allows it. However, we are not numbered as judges at that table so you would not be able to follow my score from one entry to the next by where it is listed on anyones sheet. I might be entered as judge number one for chicken and judge number 5 for pork.

Brewmaster
05-10-2011, 09:29 AM
The table would be fairly easy to include if the program allows it. However, we are numbered as judges at that table so you would not be able to follow my score from one entry to the next by where it is listed on anyones sheet. I might be entered as judge number one for chicken and judge number 5 for pork.

I think that would be something easily changed, were you would be Judge 5 on Table 6 for the whole contest. Again I'm not real familiar with the data entry portion of the scores, but don't you agree?

mobow
05-10-2011, 09:32 AM
yes, seems like it could be done if they added that info. keith

Southern Home Boy
05-10-2011, 09:42 AM
Yeah. I like that idea. That way, I can find out who scored my turn-in low and then go "talk some sense into them". :tsk:

I think that pretty much defeats the purpose of a "double blind" judging system, don't you? :confused:

JayAre
05-10-2011, 09:50 AM
Yeah. I like that idea. That way, I can find out who scored my turn-in low and then go "talk some sense into them". :tsk:


I suspect this is why we will never have any idea who actually judged our stuff...

DawgPhan
05-10-2011, 09:54 AM
You should just start cooking FBA. they actually put together the best scoresheet in bbq. You can actually learn something about your bbq from those scoresheets.

Fatback Joe
05-10-2011, 09:58 AM
You should just start cooking FBA. they actually put together the best scoresheet in bbq. You can actually learn something about your bbq from those scoresheets.

I'll second that. I started in FBA and when I moved to KCBS and saw my first score sheet, I thought there was a mistake/missing info. KCBS sheet isn't too helpful.

Brewmaster
05-10-2011, 09:58 AM
I suspect this is why we will never have any idea who actually judged our stuff...

I really don't want to know who judged my entry.

I want to know if a judge was constant in scoring across all the categories. Or if it was really my entry they didn't like.

I think this information would give the cook a better idea if they needed to change something or if they got on a bad table.

Big Sticky
05-10-2011, 10:04 AM
No one wants to know the judges name just the table number and judge number(1-6). I think the amount of info you could glean about your BBQ would be impressive.
You could see if one table consistently gave higher scores.
You could see if a judge consistently gave low scores across categories.
You could more quickly decide if changes were for the better or worse.

Hot Wachulas
05-10-2011, 10:16 AM
I third the FBA scoresheet. The judges are given a unanimous numbers so you dont know who they are.
You can see what teams were on your table, How the table did overall and they try to make it so you get different judges for each meat category. It is very helpful. It lets you know if your on the right track or not. I've been 11th in chicken twice but still had the best score on my table. Just the luck of the draw but at least I know I dont have to start changing things up. You can also see everyone else's scores so you can see where your getting beat at. M@

Big Sticky
05-10-2011, 10:22 AM
I third the FBA scoresheet. The judges are given a unanimous numbers so you dont know who they are.
You can see what teams were on your table, How the table did overall and they try to make it so you get different judges for each meat category. It is very helpful. It lets you know if your on the right track or not. I've been 11th in chicken twice but still had the best score on my table. Just the luck of the draw but at least I know I dont have to start changing things up. You can also see everyone else's scores so you can see where your getting beat at. M@

I've never competed in a FBA comp, but this sound very similar to the idea that is being put forth here.

Fatback Joe
05-10-2011, 10:23 AM
I third the FBA scoresheet. The judges are given a unanimous numbers so you dont know who they are.
You can see what teams were on your table, How the table did overall and they try to make it so you get different judges for each meat category. It is very helpful. It lets you know if your on the right track or not. I've been 11th in chicken twice but still had the best score on my table. Just the luck of the draw but at least I know I dont have to start changing things up. You can also see everyone else's scores so you can see where your getting beat at. M@


I always felt like if I won my table, I did all I can do. LIke you said, it let you know if you were on the right track and just hit a tough table.

jbrink01
05-10-2011, 10:52 AM
No one wants to know the judges name just the table number and judge number(1-6). I think the amount of info you could glean about your BBQ would be impressive.
You could see if one table consistently gave higher scores.
You could see if a judge consistently gave low scores across categories.
You could more quickly decide if changes were for the better or worse.

Better yet, this info would allow KCBS to follow a judges performance and rate them. Which, I thing is needed in that we as cooks fund the bulk of this endeavour and deserve to have the BEST judges, not just judges.

carlyle
05-10-2011, 11:23 AM
With KCBS, a different table judges each of your categories.

So if you know what table you had, you would have to find the other 5 teams that had that table? Right?

KCBS has in the works a plan to track judges. Not implemented yet. That idea needs a chance to work first. Before a team who is angry and in the heat of the moment
have a chance to track down a judge, who by the way is only identified by their number.

Just read some of the above comments, this could get ugly fast. That would not be good for KCBS or BBQ.

The double blind barrier of judges not knowing what team the entry is from and the cooks not knowing who judged the entry needs to be preserved. The integrity of the system KCBS devised and preventing intimidation of anyone requires some kind of
buffer.

One practical note: teams don't get the judging sheets til after the awards ceremony which is a couple of hours after judging is done. At least at our contest, the majority of judges have already left for home by then.

While the idea of getting better feedback about an apparent low score is a good one, the unintended consequences of what you suggest would be bad for BBQ. IMHO.

Brewmaster
05-10-2011, 11:54 AM
Thank you for your opinion, but I have to disagree.

You would not need to find all the other teams and what table they had. It would be printed on the score sheet right after Judge #6 score. There could be a column that would list the table number.

If it works for FBA, why would it now work for KCBS? Maybe KCBS can lean something from FBA.

And why can't you still make it double blind. They change the team numbers. Why can't they change the judge's number?

As I said before, I don't want to know who the judge is or even talk to them. I want to know if they consistently scored low at my table and across all the categories. Or if they didn't care for my entry and I need to make some changes.


With KCBS, a different table judges each of your categories.

So if you know what table you had, you would have to find the other 5 teams that had that table? Right?

KCBS has in the works a plan to track judges. Not implemented yet. That idea needs a chance to work first. Before a team who is angry and in the heat of the moment
have a chance to track down a judge, who by the way is only identified by their number.

Just read some of the above comments, this could get ugly fast. That would not be good for KCBS or BBQ.

The double blind barrier of judges not knowing what team the entry is from and the cooks not knowing who judged the entry needs to be preserved. The integrity of the system KCBS devised and preventing intimidation of anyone requires some kind of
buffer.

One practical note: teams don't get the judging sheets til after the awards ceremony which is a couple of hours after judging is done. At least at our contest, the majority of judges have already left for home by then.

While the idea of getting better feedback about an apparent low score is a good one, the unintended consequences of what you suggest would be bad for BBQ. IMHO.

Bentley
05-10-2011, 12:04 PM
And why can't you still make it double blind. They change the team numbers. Why can't they change the judge's number?


Pretty sure KCBS is a single blind system folks, the IBCA ticket system is an example of a double blind.


All I wanna know is do I need to reactivate my concealed weapons permit when I come to judge?

riblette
05-10-2011, 12:09 PM
KCBS has in the works a plan to track judges. Not implemented yet. That idea needs a chance to work first. Before a team who is angry and in the heat of the moment
have a chance to track down a judge, who by the way is only identified by their number.

Just read some of the above comments, this could get ugly fast. That would not be good for KCBS or BBQ.



I’m a cook. I know a lot of my fellow cooks and I would like more feedback on what the judges were thinking. I don’t care who the judge is…don’t have any interest in “setting them straight”, but I would like to know why they scored the way they did. For example when I’m given a “6,6,6,” by a judge, and two sets of “9,9,9”s on the same box all I can take away is confusion.

IMHO, comment cards are way underutilized. I received ONE last week. Comments: “dry”. Our team number was on the card and that’s it…didn’t even tell me what the meat category was. When we took to back to the KCBS rep it was simply torn up and we were told “there’s nothing we can do about it”. Seems like if there was a way to provide more feedback there would be less interest in knowing who the actual judge was.

I have heard there is some interest in gathering data on judges for the purposes of evaluation and education by KCBS, and I know this data can be fairly easily captured and compiled if there is a desire to do so. I’ve also heard there has been resistance in the judging community.

Brewmaster
05-10-2011, 12:18 PM
Thank you for the correction. Like I said I'm not all that familiar with the data entry portion.

Pretty sure KCBS is a single blind system folks, the IBCA ticket system is an example of a double blind.

Hot Wachulas
05-10-2011, 01:19 PM
I'm pretty sure the judge numbers for FBA are random and even the judge doesnt know what number they are.

Southern Home Boy
05-10-2011, 01:58 PM
I'm pretty sure the judge numbers for FBA are random and even the judge doesnt know what number they are.

That could work. But I think that's about the only way I would be comfortable with it.

Let's face it, the BBQ world is not that large. All the competitors know some judges and all judges know at least some competitors. I can just see a couple of friends talking about a comp afterwards:

"Hey, what table were you on? Did you see what was going on at table 6? I got some really weird scores..."

"Yeah... There was some FNG who didn't know his arse from a hole in the ground. The table captain had to talk to him like three times about his scoring. Gene was on that table and I could tell he was getting PO'd."

"Really? Gene and I play poker next Friday. Hmmm..."

Ok, so is that conversation really that much of a stretch? Personally, I don't think so. Would Gene "give up" the FNG? Would our competitor do anything about it even if Gene did?

Probably not, but why take the chance?

Brewmaster
05-10-2011, 02:06 PM
I'm pretty sure the judge numbers for FBA are random and even the judge doesnt know what number they are.

Sounds like a good system.

Wrench_H
05-10-2011, 02:13 PM
I'm with David, the comment cards seem like another good way to get us some info but aren't used. We turned in terrible chicken, knew it was terrible and knew it was very salty. We got 3 comment cards all confirming what we knew. However those were the only comments we got. It would have been great to get some comments on our brisket and ribs which we didn't know what was good/bad about.

I also agree with the OP that some sort of comparison of how each judge scored (without any knowledge of who that judge was) would be nice. Particularly since, as mentioned by someone else, there are 9-9-9's and 6-6-6's on the same turn-in box. It leaves you wondering if you have a perfect brisket or a piece of shoe leather. This would particularly help a newcomer like myself who is trying to learn as much as possible from the scoresheet.

Bentley
05-10-2011, 02:36 PM
I will give you my perspective as a 9 year judge & competitor. I would love to give more detailed feedback to teams, just as I would like to receive more when I compete, the time constraints make it virtually impossible.

We are instructed to judge all meat before we fill out a comment card. We are not allowed to take notes, a no-no, and if I am wrong on this, someone please correct me! So I have to judge all the meat and remember what was wrong with piece 1 and write about with no notes. So I have a 30 minute window to judge the meat, which if I am doing MY scoring correctly takes just about all of that. When do I get the time to give you feedback? Do I rush my judging on the outside chance I give a score under 6 (any score under 6 I feel obligated to articulate to the team why I gave it) and lessen my effective as a judge?

Teams will have to decide if they wanna lobby to have judging times increased, cuz 30 minutes is not enough to do what I need to do as a judge properly and still give you feedback that will benefit you!

Brew-B-Q
05-10-2011, 02:42 PM
I don't think you would need to know the table number, judge number, or teams at your table in order to make the score sheets more useful. What if they added a row under each category on the front page. In the new row, they list the average of the other five scores that each judge handed out to the other teams at your table.

For example,
Judge 1**Judge 2*Judge 3**Judge 4*Judge 5*Judge 6
(8 8 8 )**(6 6 6)**(8 9 8 )**(8 8 8 )**(7 8 8 )**(6 6 6)
(8 7 7)**(8 8 8 )**(7 8 8 )**(6 6 6)**(8 8 7)**(6 6 6)

This would protect the identity of the judge, but also give you an idea of whether you got a judge that just didn't like your food (judge 2), or a judge that didn't care for anything that day (judge 6).

Sorry for the formatting, but hopefully it makes sense.

I realize this would still leave many questions unanswered, but it would let you know if you were the victim of a low scoring judge or whether your food just wasn't that good that day.

Big Sticky
05-10-2011, 02:54 PM
I do not think that judges are going to fill out comment cards no matter how long they have to judge. This I think gives the teams a way to actually make heads or tails of what the judges think of your food.

Hot Wachulas
05-10-2011, 03:00 PM
I don't think you would need to know the table number, judge number, or teams at your table in order to make the score sheets more useful. What if they added a row under each category on the front page. In the new row, they list the average of the other five scores that each judge handed out to the other teams at your table.

For example,
Judge 1**Judge 2*Judge 3**Judge 4*Judge 5*Judge 6
(8 8 8 )**(6 6 6)**(8 9 8 )**(8 8 8 )**(7 8 8 )**(6 6 6)
(8 7 7)**(8 8 8 )**(7 8 8 )**(6 6 6)**(8 8 7)**(6 6 6)

This would protect the identity of the judge, but also give you an idea of whether you got a judge that just didn't like your food (judge 2), or a judge that didn't care for anything that day (judge 6).

Sorry for the formatting, but hopefully it makes sense.

I realize this would still leave many questions unanswered, but it would let you know if you were the victim of a low scoring judge or whether your food just wasn't that good that day.

Thats exactly what the FBA sheet looks like. Plus it shows those scores for every competitor. So you can look for all the teams that had judges 1-6 on say "chicken" and see how you placed on the table and how they placed as well. Seeing other teams scores broken down really helps put things in perspective. Score sheets aside, the 1hr between turn-ins is my favorite.

Big Sticky
05-10-2011, 03:04 PM
No one wants the names of the judges.
All they need is to have table 1 judge 4 as an example.
I do not know how to put it on the sheets but I'm sure it would not be that difficult.

carlyle
05-10-2011, 03:46 PM
Sounds like KCBS should check in with or talk with FBA.

On the surface this sounds simple to do and is something everyone should want.
However, you have to have software that is able to spit out the info you want in the way you want. In all likelyhood KCBS would have to change software then train everyone to use it.
In the judges area, after scores are entered and the order of finish determined, the process to get each team their own packet of info to return to them takes a while to do. With printing, copying, collating and stapling, more than one person is occupied with this important task. And we are on the clock as well. The deadline is to have everything ready for the awards ceremony that everyone is anxious for.


I think some computer changes are in the works, but I don't know what changes or where in the process they are. Or if they even address this concern.

I sympathize with and agree that cooks need better feedback from judges so they can improve their product or at least understand the scores better.

As an organizer, I want this info too. I get some in real time from our reps.
But something in writing would be better and would help me guide the selection process for future contests.

pop's smokin hot que
05-10-2011, 09:51 PM
If we do that, why not tell us judges the team we are judging? KCBS is a double blind contest as it should be. You do quality work you get quality scores.

Brewmaster
05-10-2011, 10:34 PM
You lost me here.

If we do that, why not tell us judges the team we are judging? KCBS is a double blind contest as it should be. You do quality work you get quality scores.

riblette
05-10-2011, 10:52 PM
You do quality work you get quality scores.

So, you're suggesting all judges are equally capable and there's no room for improvement? Sweet! I feel better now that that's cleared up! :grin:

HarleyEarl
05-10-2011, 11:09 PM
Sorry for the length of this, but after mulling it over for a while, I just had to post.

To be honest, I really don't want nor do I care which judge is judging my food - I prefer the blind approach. Also since KCBS will be looking at their scores and identifying outliers, that's good enough for me. HOWEVER, I would love to see which table scored which six teams so I can see if I knocked it out of the park for my table or tanked at the table. Say if the top six teams in chicken all came from table 1 and my chicken was at table 2 and I had the best of table 2, and I came in 15th, I would be able to know I just got the bad luck of the draw where my turn in ended up at. Such be life! If you talk to enough teams during a contest, you'll inevitably hear something about catching a good or bad table - this would just confirm that. There's really nothing we can do about the human element and no judge in the world can taste every team's entry and even if they could, there would still be inconsistency.

I've also seen quite a few comments about the comment cards. I would love to see more of them. I've done 5 contests this year and only received one comment card for a tenderness score on ribs for a 6 (the other judges gave me 8's & 9's). The comment was simply "over cooked". Guess what, all 6 judges nailed it - I knew the judge who gave me the 6, got a hold of the the second slab which was a bit over cooked. Fortunately I have not received any 5's (yet, but it's bound to happen though) and have only received a few other 6's and would have like to know what caused a them to vary from the other judges scores. I can see the point of the judges not having enough time in the 30 minute window, but until you've been on the other side cooking, you can't even imaging how much work a team puts into that same 30 minute window to present the turn-in boxes to the judges, so that argument really doesn't hold a heck of a lot of water with me - the cook teams are professionals and so should be the judges and give us useful feedback if you don't care for what is presented to you.

Enough ranting...

Big Sticky
05-10-2011, 11:19 PM
To find out what teams a judge had at his or her table all that he or she would have to do is look at a score sheet after the contest.
If random numbers are assigned to a judge even after the contest a team would not know who the judge is unless the judges said I was judge whatever on table whatever. I do not understand why judges feel they need to be in the witness protection program.

The only reason I want this info is so I can better my Q. I would be able to through out scores that are consistently low from one judge if they consistently scored low for all at the table or even across categories.

Not all judges score the same.

CaptTable
05-11-2011, 06:54 AM
First, the present scoring system does not allow for entering the table number anywhere.

Second, KCBS does not use judge assignment (Judge 1, Judge 2, etc.) It is purely random. Therefore, "Judge 1" in chicken, may be "Judge 5" in ribs.

Third, I am not a huge fan of comment cards. I wasn't in the beginning, then I was, and now I'm not! Sounds wishy-washy, doesn't it. However, as a rep, I think they could become very useful if, and this is a huge if, KCBS REQUIRED the judge to comment on any score given of 5 or less. I know many cooks think it should be 6 or less, but a 6, or "average" score doesn't, necessarily mean anything was wrong with it, just nothing was great about it. Plus, some cooks feel their product is always a 9, so comments should be for anything less than a 9. Trust me, judging would take forever!

Sorry for the long post,

Phillip

CaptTable
05-11-2011, 06:59 AM
And, FBA score sheets ROCK. I've never worked or been to a FBA contest, but love their results packages!

Phillip

JayAre
05-11-2011, 08:33 AM
When I first read this topic, I thought, yeah, great idea give us more information to digest so that we can intelligently adjust our products, but in the end, it really wont change anyting, no matter what you do there will always be inconsistant scores, that is because nobody has invented a tasting machine that would consistantly judge base on a given criteria, until that happens humans will judge based on their prefereance...we all have had that time where we scratch our head trying to figure out what was wrong with our stuff when in reality, there probably wasnt anything wrong, it just didnt wow that judge. (unless you turn in raw chicken, which seemed to be a trend this weekend)

by the way Phillip, it was nice to see you this weekend (for the most part, next time how about not stopping by at 12:15!!!) :redface:

musicmanryann
05-11-2011, 08:47 AM
I am for anything that will efficiently facilitate teams getting more and judges giving more feedback, while at the same time preserving the integrity of the current blind system.

Brewmaster
05-11-2011, 09:01 AM
I agree, that it's hard to determine when you need to make changes to your entry or if you really need to tweak something.

But lets say your turn-in gets to a table with 3 of the top ranked teams. And all three of those teams score top 3 at that table and your scored 6th. You still get a very respectable score and hit top 10 in the contest in that category.

Now with this scenario and the way things are scored now, you would think you were probably the best at your table, but really have no idea.

If the scoring was changed you would know there is room for improvement.

This is just one benefit I see from this system.

Really I am more interested in seeing a judge scores me a 6 and doesn't score another 6 the whole time. Then I know I've got a problem and not a low scoring judge.


Anyway, I just want to say thanks for everyone's opinions. This is what this forum is all about. :thumb:

Cheers,
Nate

When I first read this topic, I thought, yeah, great idea give us more information to digest so that we can intelligently adjust our products, but in the end, it really wont change anyting, no matter what you do there will always be inconsistant scores, that is because nobody has invented a tasting machine that would consistantly judge base on a given criteria, until that happens humans will judge based on their prefereance...we all have had that time where we scratch our head trying to figure out what was wrong with our stuff when in reality, there probably wasnt anything wrong, it just didnt wow that judge. (unless you turn in raw chicken, which seemed to be a trend this weekend)

by the way Phillip, it was nice to see you this weekend (for the most part, next time how about not stopping by at 12:15!!!) :redface:

Yakfishingfool
05-11-2011, 09:03 AM
Any given day any skilled team can take GC, Any given day, any given skilled team can be screwed over by bad judging. I disagree with the above comment: quality work does not always produce results. With tthe time/money and heart given to these events/competitions to not have some way to at leaet remove outlying judging seems crazy. I'll leave it KCBS or a bettter organization, to come up with a solution. With no comment cards teams must kind of drift in the ether to find the right things needed. Or take a winning teams Competition class. S

ParkAvenue_2
05-11-2011, 01:12 PM
I think that pretty much defeats the purpose of a "double blind" judging system, don't you? :confused:

Agree with this.

ParkAvenue_2
05-11-2011, 01:15 PM
Third, I am not a huge fan of comment cards.

From a team's perspective, I don't think the comment cards are used liberally enough.

In the last several years, we've probably ony received comment cards 2 or 3 times. Many times, we've speculated about why a score was low or high in a particular area, but it would be better to have the written feedback from the judge so that our speculation doesn't take us off down the wrong trail in trying to adjust something.

Big Sticky
05-11-2011, 01:28 PM
How would this effect double blind?

Bentley
05-11-2011, 07:05 PM
KCBS competitions are not double blind.

JayAre
05-11-2011, 07:17 PM
n.
A testing procedure, designed to eliminate biased results, in which the identity of those receiving a test treatment is concealed from both administrators and subjects until after the study is completed.

double-blind dou'ble-blind' (dŭb'əl-blīnd') adj.