Context and tl;dr – I spent two hours writing an “essay” for a topic that I felt that was interesting and not for academic purposes, but ended up realizing that a fundamental assumption was flawed. It’s still an interesting topic nonetheless and I might go back to finish the job since I can probably work my way around the flawed assumption. At the very least, I had quite a bit of fun writing this. This was probably more productive than the time I tried to write an excel spreadsheet that could solve Sudoku’s using Solver (I know it can be done! I just need to fiddle around with it some more!)
I apologize for the poor formatting and limited editing of this spur-of-the-moment article.
Z-Scores and Solver: How Business School has changed the way I think
When I was still a high school student taking the International Baccalaureate (IB) program, I wrote an essay on the Martial Arts and how difficult it was to quantify “good” and “bad” art (for the record, I was writing a TOK essay). I argued that although certain moves were easy to quantify (i.e. “the amount of spins on a tornado kick”) a Wushu form was difficult, if not impossible, to grade using quantitative means.
Now, several years later in the midst of my University career, I still carry a passion for the Martial Arts. However, I am not merely a competitor in the sport but also an event organizer and assistant director of an upcoming martial arts tournament, the 32nd International Can-Am Martial Arts tournament. Oh boy, if only I could go back in time and give my old “me” a quick talk.
Contrary to my arguments in my high school essay, martial arts forms are commonly graded quantitatively by judges. These scores are produced when several judges (commonly 3-4) observe the performances of various competitors and grade each one. These scores are then averaged, and the competitor with the highest score is awarded a gold medal (followed by a silver medal, bronze, etc). How does this work? Well, there are four judges and by averaging the scores of the combined judges we can have a more “accurate” quantitative number to judge the competitors by. Of course, these “numbers” are arbitrary; they serve only to measure a competitor’s performance against someone else in the same bracket. We certainly cannot compare a performance of an 18-year old martial artist to that of a 3-year old, or grade a performance with a quantitative number that would be universally recognized (unless that score was between 1 and 7 ;D). In fact, it is a urban tale that the first competitor will almost always be given a relatively “low” score so that the judges can grade future competitors higher if they are more impressive than the first (that is not to say that the first competitor will never place first, it just means that you’ll seldom see a superbly high score of, say, 9.98 on the first competitor so that the judges will have room to work with. Judges rarely give out 9.98 as a score at all).
I am rambling on and on about the process of scoring competitors fairly. The point that I’m trying to make is that scoring can be quantitative, but only when placed side by side with a performance with the same bracket.
Now on to the main topic.
^You see this formatting right here? I would never do that in an academic essay. Pfffft.
Today I attended yet another meeting for the upcoming tournament which I am volunteering for. Today we were presented with an interesting problem. That problem is the inspiration for my sudden urge to write a giant article about martial arts, scoring, analytical statistics, and Excel.
The problem is: How do we fairly award the “Best All-Around” and “Grand Champion” trophies to the competitors that deserve it, WITHOUT having the top competitors compete again, and WITHOUT bias on the part of judging.
Some context is needed here.
Every year our tournament hosts a rather large subcategory of competition events under the Wushu category. These range from hand forms and short weapons to flexible and long weapons. Each of these categories are subdivided into beginner, intermediate and advanced categories. Each of these categories are further divided into age categories (14-, 16-17, 18+, etc.). Each of THESE categories are even FURTHER subdivided into male and female categories. With all these categories, the Can-Am boasts over 800 events. That’s a huge number! Each event is awarded a gold medal, silver medal and bronze medal winner. And on top of that, we have large awards known as “Best All-Around” and “Grand Champion” awards. And this is where the problem starts.
The Best All-Around trophy exists to recognize the competitor with the best accumulated scores in a variety of Wushu forms. In previous years, a competitor would have to compete in five different events in order to quality for this prestigious prize (hand form, short weapon form, long weapon form, flexible weapon form, and an optional form). This award was pretty easy to designate to the deserved competitor in each year because, well, not many people can do that many forms in a day without keeling over and dying (or: not many people competed in 5 events). However, in recent years we have lowered the criteria for this award so that eligible competitors need only to compete in three events to qualify (hand form, short weapon form, and long weapon form). Although this means that competition for this prize would increase, we overshot our expectations and found that WAY TOO MANY competitors were eligible for this award, making it very difficult for judges to identify the competitor who truly deserved this award. On top of that, we have so many events! It is also common knowledge that people in the “advanced” category gain exactly one more “point” than those in the “intermediate” category, and the same goes for the “beginner” category (For example, competitors in the “advanced” category typically receive between 8.0-9.0 or 9.0-9.9 depending on the event, whereas competitors in the “intermediate” category of the same event typically receive between 7.0-8.0). Knowing this, competitors who compete in three “advanced” events will almost always beat the accumulated average of competitors who only compete in “intermediate” events. Is this really fair? Will the “Best All-Around” champion always be the one who competes in “advanced” categories? What if he’s the only competitor in that category, making it easier to gain a better score than if he was to compete with, say, ten more competitors? Does that make this situation a Simpson’s Paradox? Will I ever stop asking rhetorical questions?
And there is yet another problem: the Grand Champion trophy. This trophy is awarded to the competitor with the best score. This trophy is meant to be given to an individual who may not excel at a wide range of forms, but is astronomically amazing in his speciality. You may already have questions. Chris! How do you determine the “best score”? Didn’t you just say that there were hundreds of categories? How do you compare a specialist in a long weapon like Spear to a specialist in a short weapon like Straightsword? Isn’t that, well, form-ist? You form-ist person!
Well, of course, we had a system to determine the Grand Champion winner. It was quite simple: the receivers of a gold medal in an “advanced” event were eligible to compete, if they wish, for the Grand Championship trophy. All eligible competitors would then gather at the end of the day and present their forms again, one by one, in front of a strict panel of judges who would then announce the winner of the tournament.
Note the phrase that eligible competitors could compete for the grand prize “if they wish.” Now, why would a competitor NOT want a run for the great granddaddy of all tournament prizes? Two simple reasons: One, it is typically easy to tell if someone is miles above you in skill level, so some competitors wouldn’t want to waste their time vying for an award they know they won’t get (sorry for the harshness). Second, and more realistically, THEY’RE TIRED.
The flaw of the Grand Champion trophy is that you have to go through every single event in order to produce all the eligible contestants. And, in such a huge tournament, that takes time. Lots of time. This means that a competitor that competed at noon would have to wait ten hours or more in order to compete for this little trinket. No thank-you, da-yo. Besides, if you already gave your all in your event early on in the day, waiting ten hours to compete again with the same form is crippling to the competitor. This process is inefficient, and is slightly unfair to those competitors who have to compete in more than one event.
Although this system is somewhat redundant and ugly, for 31 years this system has seldom run into problems and has been tried and true. Why? Because the winners were almost always fairly obvious. Fortunately, every year there would be few Best All-Around entries to choose from, and recognizing the most prominent competitor was always easy (this guy beat everyone else by a landslide in all 5 events!). The Grand Champions were, realistically, fair because the eligible athletes were fairly graded by all the judges. So, in theory, these processes do work.
But.
This protocol is not very efficient under high capacity.
Over the years the Can-Am tournament has grown steadily from the small “West-Coast Championships” it once was, to the “International Can-Am Martial Arts Championships” it is now. More and more competitors are flocking to this tournament, and the martial arts community is only increasing in size. If we relax the criteria for Best All-Around, it will become difficult to spot the athlete that is truly “best all-around.” And of course, the more competitors there are the longer the tournament takes, which means that the Grand Championship competition will probably take place even later than before. In theory, these practices are still viable and can be used to host a decent tournament. But what if we have too many powerful competitors? What if the winner isn’t so clear? What if we run out of time for the Grand Championship? What do we do? What SHOULD we do?
It’s simple. We kill the Batman limit the eligibility of the competitors. That is the easiest way to ensure efficiency in a tournament environment. Just make the criteria harder to achieve, and then we can shake off all the ineligible competitors.
But that’s not what I want.
Regardless, what I want does not necessarily dictate to what should be done. But my opinion stands that I believe that anyone, any competitor, should be eligible for the Best All-Around and Grand Champion trophies. Although it may be slightly counter-intuitive that a “BEST” all-around and grand “CHAMPION” awards should not be awarded only to those in “advanced” weapons forms, we must keep in mind that there are many, many variables apart from mere qualitative categories like “advanced” or “intermediate.” These categories include age, form style, and maybe even sex (if you’re sexist, but I’m not). What I’m trying to say is that if you have an award that is restricted to only “advanced” competitors, you’re biasing that award to competitors that have more experience (say, a competitor that is 18 years old as opposed to a 13-year old that is in the advanced category because he arbitrarily chose that category when he signed up). Likewise, if you have an age-restricted award (say, a grand championship award limited to ages 14 and under, or 18 and over) you’re neglecting the fact that these competitors are all competing in different styles of martial art. Basically, if you only base the award on one variable, you’re ignoring the rest of the variables. And that creates some sort of bias.
Now, usually this isn’t a problem. Large trophies are almost always given out correctly because judges usually rule the best competitor, and their word is final. But for the sake of efficiency, fairness, and stability of the competitive scene I must suggest something else.
My goodness, it’s been 2000 words in and I’m finally talking about the concepts stated in the title of this work. This a terrible “essay” and I should feel terrible.
At any rate, recall the very, very beginning of this article when I mentioned that Wushu as a sport can only be quantitatively described if competitors are compared with others in the same category. This means that each “category,” no matter how different the individual athletes are, is a group with a common variable. If subdivided correctly, such a group would include all the “important” variables such as age, sex, style, experience, you name it, and it could have it. Another important aspect would be that such a group MUST have at least two individuals otherwise it would not be a group.
Now, given the fact that we have one category with at least two different individuals in it, we can derive some average or “mean” from the quantitative scores of these individuals. From this, we can easily deduce the standard deviation of this set of numbers, and from that we can derive the Z-score of each individual in their specific category (given that X is a measurement variable having a normal distribution). Coincidentally, many of the most popular events have 20+ individuals, nearing 30 individuals with which we can assume the Central Limit Theorem…
At this point I realized that the Central Limit Theorem could not be applied to many of the less popular events, which typically had less than 30 people in it. This means that normal distribution cannot be assumed, which meant that Z-Scores are pretty much useless in such a situation. In an ideal world where normal distribution could be assumed, I would have wanted to take the Z-Scores of each event so that I could compare them with each other without bias; I would have wanted to figure out some easy way to do this, and then automate the process using Excel and Solver. In theory, it should not be too difficult to write a spreadsheet that could easily chug out Z-Scores for each event which I could then compare with, making Best All-Around and Grand Champion a much easier task to undergo (and less biased, as well!).
I very disappoint. But mostly because I’m too tired to continue typing a non-academic essay this late in the night.