I split this off.
Great basic thread and I am learning a lot!
But, here is a more detailed and specific question for our Judges and Experienced Competetors.
Please bear with me- this is complicated.
I was a Ladies Gymnastic Judge (USGF) in the 80's for 3 years.
Sounds like a similar system for training and certifying Judges.
Classes, video, study, pracitce, and a "killer" written test.
Start at the lower levels and, with "experience, a track record, and more testing", move up in the classes and authority.
The exact details are not important- the concept is. There are rules, training, and evaluation before you become a "Judge".
We (the USGF and FIA) had a final scoring system based on the judges being "In Range".
Basically, all judges must be "in range" with each other and the "head judge". The range depended on the score. For example, for "head judge" score of 9.5 to 10- we had to be within .1 or a "conference" was called.
This acceptable range went up to .5 points at a 7 score from the head judge (or something like that). The numbers are not important.
If a conference was required, the judges discussed (in private but face to face) just what they had seen to generate the score.
Normally a matter of "viewing angle" or rules interpretation.
I will save you the details.
The bottom line was--if the Judges observed or scored things outside of an acceptable range--It had to be resolved before the score for the girl was posted.
Lots of subjectivity--seems just like judging good Q
But, I learned that Judges almost always (but they are not perfect) tried to do the right thing, within the rules, to render a valid and accurate score!
Human evaluation is subjective by nature and can never be reduced to totally objective terms.
If so- hire a computer!
I have seen remarks (in some BBQ rules) about throwing out the lowest scores to resolve ties, etc. I saw some comments in the origonal thread about spices, texture, presentation, etc. that raised my curiosity!
Other than that, I have no practical experience. Help me out here.
So- finally- here is/are the questions for the guys who have been there!
From meet to meet, are the judges normally in agreement on scores?
How about within a meet if they are way off- is anything done?
If a judge is "way low" or "way high", is the score resolved?
If a judge seems to "favor" or "dis-favor"some item, i.e. presentaion or spice leve, is action taken on the spot to equalize or normalise the scoring.
I am not advocating adoptance of rule changes in any way shape or form!
I am just curious about how the current system really works.
DF and I am entering that world as comtestants and appreciate any feedback on how the system works.