KCBS judge seating changes

[GrillBillie_D]

Just saw the board meeting quick notes:

Early 2017, it was determined that judges would be seated via the Shuffle. This was instituted nationwide. Mid-year, Mark Gibbs developed a method of seating of judges that was based on their scoring averages divided evenly among the number of tables to be utilized at a contest. This program was Beta tested across the country for several months and it was determined that it was very successful. It was rolled out to ALL reps in January of 2018 and mandated to be used for all Master Series contests.

I am in favor of this change but this is not the end of scoring issues. It may help a little at contests but there are still other issues. The fundamental issue of judge training differences still needs to be addressed.

[Slamdunkpro]

Originally Posted by GrillBillie_D

Just saw the board meeting quick notes:

Early 2017, it was determined that judges would be seated via the Shuffle. This was instituted nationwide. Mid-year, Mark Gibbs developed a method of seating of judges that was based on their scoring averages divided evenly among the number of tables to be utilized at a contest. This program was Beta tested across the country for several months and it was determined that it was very successful. It was rolled out to ALL reps in January of 2018 and mandated to be used for all Master Series contests.

I am in favor of this change but this is not the end of scoring issues. It may help a little at contests but there are still other issues. The fundamental issue of judge training differences still needs to be addressed.

No offense to Mark but this is putting a band aid on an arterial wound. YOU CANNOT AVERAGE RANDOM EVENTS. It'll look good for a bit but the TOD's and TOA's will remain until, as stated above "The fundamental issue of judge training differences still needs to be addressed."

[Gary Tomato]

They used the new seating system at the KCBS masters event here in Melbourne earlier this month.

I'm a KCBS judge myself so I'm interested in the opinions of competitors, could you explain what the "fundamental issue of judge training differences still needs to be addressed" is/are?

[Fat Freddy]

Originally Posted by Gary Tomato

They used the new seating system at the KCBS masters event here in Melbourne earlier this month.

I'm a KCBS judge myself so I'm interested in the opinions of competitors, could you explain what the "fundamental issue of judge training differences still needs to be addressed" is/are?

Not the only one but a big issue a lot of people have including many of the judges is the way they were instructed. Some were instructed to start at a 9 and work down from there, others to start at a 7 and work up or down and yet others at a 5 and work up. That is one major issue

[QN]

Originally Posted by Fat Freddy

Not the only one but a big issue a lot of people have including many of the judges is the way they were instructed. Some were instructed to start at a 9 and work down from there, others to start at a 7 and work up or down and yet others at a 5 and work up. That is one major issue

There is not supposed to be any predefined starting score number. Each entry is supposed to be scored as it is, on its own merit. Many years ago the standard was to start at 9; everything was considered to be a 9 until you saw it and then actually sampled it. This resulted in a LOT of 180 scores. At some point it was decided to define 6 as the starting point and score up or down based on each entry. I think the intent was to reduce the number of 180 scores.

[ModelMaker]

Really I don't care where they seat me. My scoring patterns are not set by how I score past events and past entrys but are more caused by the entrys provided to my table on that one day. I react to the quality of each contests entry.
I also contend there will always be a TOA and TOD it just has to have a winner and a loser.
As far as
"The fundamental issue of judge training differences still needs to be addressed."

All CBJ training classes have for several years been taught from one power point presentation. You can't be more consistent than that. All that's left is to recertify judges trained before the programs beginning.
Mandatory recetification? I'm good with that. It can be done online in a hour...
I've been to 7 CBJ classes taught by 4 or 5 different instructors and I don't see a problem with current training procedures.
Ed

[Ron_L]

Originally Posted by Fat Freddy

Not the only one but a big issue a lot of people have including many of the judges is the way they were instructed. Some were instructed to start at a 9 and work down from there, others to start at a 7 and work up or down and yet others at a 5 and work up. That is one major issue

Agreed!

Originally Posted by QN

There is not supposed to be any predefined starting score number. Each entry is supposed to be scored as it is, on its own merit. Many years ago the standard was to start at 9; everything was considered to be a 9 until you saw it and then actually sampled it. This resulted in a LOT of 180 scores. At some point it was decided to define 6 as the starting point and score up or down based on each entry. I think the intent was to reduce the number of 180 scores.

While that may be true to day, it was not always true. When I sent to my first CBJ class we were taught to start at a 6 and score high for food that was above average, etc. My wife went a couple of years later and she was taught to start at a 9 and score lower as required. Both classes were taught by the same instructor, but the content had changed.

Without some sort or re-certification program to bring every one up to today's standards, there are judges who were taught differently and approach the table differently.

Originally Posted by ModelMaker

Really I don't care where they seat me. My scoring patterns are not set by how I score past events and past entrys but are more caused by the entrys provided to my table on that one day. I react to the quality of each contests entry.
I also contend there will always be a TOA and TOD it just has to have a winner and a loser.
As far as
"The fundamental issue of judge training differences still needs to be addressed."

All CBJ training classes have for several years been taught from one power point presentation. You can't be more consistent than that. All that's left is to recertify judges trained before the programs beginning.
Mandatory recetification? I'm good with that. It can be done online in a hour...
I've been to 7 CBJ classes taught by 4 or 5 different instructors and I don't see a problem with current training procedures.
Ed

It's been a while since we've run into you at a competition, Ed, but I don't think senior judges like you (and I mean that in terms of judging experience, not age ) will change how they score based on seating. But, it may affect newer judges, and may help to even things out, but only time will tell.

[rob g]

I still think the problem is the fact KCBS will not define what is "average" bbq. The score card dictates that average bbq rates a 6 and then moves up or down from there. Is average everything you have sampled in your life or only those meats eaten at a judging table? I think a good start would be to define average as competition average. This would even out people's life experience to some point.

[ModelMaker]

Originally Posted by rob g

I still think the problem is the fact KCBS will not define what is "average" bbq. The score card dictates that average bbq rates a 6 and then moves up or down from there. Is average everything you have sampled in your life or only those meats eaten at a judging table? I think a good start would be to define average as competition average. This would even out people's life experience to some point.

Competition Average????
So every new certified judge starts out with absolutely no idea what "competition average" is?
Do you want to use the meat offered in the class to set the new judges average for all future contests?
After one contest do they know what competition average is then?

I've always held that 10 judges have 10 ideas of what average is, kind of what the word average means.
Do you think there is such a swing in the word average that contests are won or lost because of it? A new judge that is from small town Iowa's exposure to BBQ is going to be dramatically different than a new judge from downtown Kansas City.
Average moves constantly.
Only experience will set what a judge determines average to be.
Ed

[gettinbasted]

Originally Posted by Ron_L

It's been a while since we've run into you at a competition, Ed, but I don't think senior judges like you (and I mean that in terms of judging experience, not age ) will change how they score based on seating. But, it may affect newer judges, and may help to even things out, but only time will tell.

If the goal is to even things out, we are all in big trouble!

On a side note, I wonder how they determined it was a success? More scores bunched to the middle of the pack?

[rob g]

Originally Posted by ModelMaker

Competition Average????
So every new certified judge starts out with absolutely no idea what "competition average" is?
Do you want to use the meat offered in the class to set the new judges average for all future contests?
After one contest do they know what competition average is then?

I've always held that 10 judges have 10 ideas of what average is, kind of what the word average means.
Do you think there is such a swing in the word average that contests are won or lost because of it? A new judge that is from small town Iowa's exposure to BBQ is going to be dramatically different than a new judge from downtown Kansas City.
Average moves constantly.
Only experience will set what a judge determines average to be.
Ed

You are correct that only experience will determine what average is. As new cbj's get a few contests under their belt then their idea of average will settle out if they are directed to base their opinions only on what they have eaten at a contest table. KCBS still needs to try to define average so judges have something to base their opinion on and try to develop a standard that can be used at every contest.

While we are told not to compare sample 1 vs sample 2 judging by its very nature is comparative. We are comparing the piece of meat we have in front of us to our individual definition of average or hopefully a defined standard.

Until somebody can put a piece of meat in front of you and say this should be scored a 6/7/8 based on published KCBS standards you will not get consistency across the board. Standards need to be defined and that is easier said than done.

[D&D BBQ]

I would say you definitely can get more consistency in terms of appearance scoring as ALL of the judges are looking at the same box but yet you’ll get scores ranging from 6-9. That shouldn’t happen. Yes, that lone 6 is thrown out, but the remaining 7 kills you when the others gave it 8-9. It’s the same dang box, the scores should be done within 1 point! Now for taste and tenderness, you’re going to get some natural variability as palates differ and what is the right amount of heat for a spicehead is too much for someone with acid reflux. You’re also going to get different tenderness scores because the meat may differ even if cooked to the same temperature. That’s going to happen no matter what and is part luck and skill of the cook.

As far as people’s palates go, the only way you can even come close is for KCBS to finally determine what “average barbecue” is. Is it your neighbor’s backyard bbq chicken that’s a little too charred and covered in SBR or is it the local bbq joint’s consistently decent stuff? Everyone has different life experiences but I would think the natural tendency is for more experienced judges to go with what they’ve tasted in previous comps for comparison’s sake (unless instructed otherwise). As others also referenced, ALL judges either need to start at 9 and go down or start at 6 (average) and go up or down... Of course, then you gotta come up with a universally understood definition of what “average barbecue” is for that 6 starting point to work. Good luck with that...

Reminds me of an old PJ O’Rourke joke about the social sciences which he says boils down to 3 rules:
1. People do lots of things.
2. We don’t know why.
3. Test on Friday.

[JS-TX]

I think a starting point is a good thing. But starting at 9 will inflate scores as stated above. If anything a starting point of average ~x (6?) is acceptable. To define average is a little tough because flavor is so subjective. Somebody's average is somebody's perfect. How common are scores of 5 or below? But at the end of the day, aren't all boxes subject to the good/bad table dilemma? If so, then outcome will always vary a bit... IMO.

[Scottie]

I went through the class on 2000 or 2001. Started at 9. Have never had any new training or certification.... That is my problem with KCBS judging and until everyone is on the same page, these changes are just a band-aid

[ModelMaker]

Originally Posted by Scottie

I went through the class on 2000 or 2001. Started at 9. Have never had any new training or certification.... That is my problem with KCBS judging and until everyone is on the same page, these changes are just a band-aid

Jeez Scottie, where you been, I haven't been yelled at in a long time...
Have you ever considered sitting in on a class that would be free to you?
Have you taken the 50 question quiz on the KCBS website to see if your current on rules and practices?
I know you want to be the best judge you can be.
Ed