HomeMade Rating Systems
The Washington Post released a new "top passer" ratings system for high school quarterbacks today.
The formula (click here to see the graphic that explains it) is fairly complex, involving touchdowns per attempt, interceptions per attempt, a modified touchdowntointerception ratio, completion percentage, and yards per attempt. There are some "weird multipliers" in there to convert the scores into an easiertounderstand 100point scale.
The NFL has a similar scale, but with a maximum score of 158.3, which confuses a lot of people.
For high school football fans, the new rankings for the top 20 high school quarterbacks put Tyler Campbell from Good Counsel in Olney in first place with a 97. Bryn Renner of West Springfield is the highest rated Fairfax County QB, ranking 6th with a 93.6 rating. My high school, I'm sorry to say, has not made the list. (A full list is online as well.)
For math people, what do you think of this formula? Does it make sense?
And have you ever devised your own rating system for anything? Can you share it with the group?
I've never created a formula like this, but my colleague Jay Mathews developed a simple way to rank high school according to the rigor of its classes. The Challenge Index is both controversial and extremely influential.
By
Michael Alison Chandler

November 12, 2008; 11:33 AM ET
Previous: My First Parent Conference 
Next: Review, More Review, Some Algebra II
Posted by: bob9999  November 12, 2008 1:29 PM  Report abuse
Hi – This is Jeff Nelson, who put together the formula. Michael Alison Chandler asked me to write a little bit about how I came up with it, so here goes:
To start, I am no math wiz. Luckily, this formula isn’t that complicated, mathematically.
Once I determined how much I wanted to weigh the categories, and what onfield results should produce the maximum values in the formula (and that was easily the most agonizing aspect of this process), it was simply a matter of figuring out the proper equations.
For instance, if completion percentage is out of 10 points, and I want someone completing 70 percent of his passes to get the full 10 points, what equation works?
In this case, I started with 0.7 x ? = 10. Then I divided both sides by .7 and the answer was 14.2857143. Or more simply, 14 2/7.
Therefore, for the formula, I take someone’s completions divided by attempts to get their completion percentage, and then multiply it by 14 2/7. (Basic, I know.)
The most challenging category was the modified TDtoINT ratio. This was out of 5 points and I wanted someone with 0 TDs and 0 INTs to start in the middle, getting 2.5 points. Then if they had more TDs than INTs, they would get more points. And if they have more INTs than TDs, they would get less points.
So I figured out how to make 1 TD and 1 INT = 2.5 points: (TDs per attempt)/(TDs per attempt + INTs per attempt) x 5.
The only problem, of course, was that zero screwed it all up. So that’s when I consulted the George Washington University statistics department, figuring there was some complicated way to solve the problem. But a GW professor had a simple suggestion: adding 0.1 to everyone’s TDs and INTs. (See: “The Zero Issue” in our graphic: http://www.washingtonpost.com/wpdyn/content/graphic/2008/11/12/GR2008111201132.html)
This worked.
Once I had the equations for each part of the formula, I plugged in the numbers and looked at the results. And I liked them. And as I said, I thought the math was relatively easy.
Posted by: jeffnelson00  November 12, 2008 2:32 PM  Report abuse
Hi Jeff,
Any such system is bound to be subjective, so take my comments for what they're worth: one man's opinion.
1) In the first column, there should be no reason to add 0.1 to the TDs per game figure. There is also no need to add 0.1 to the attempts per game, as this is strictly a passer rating system, and a QB with 0 attempts would not be rated.
2) There is no reason to add 0.1 to the INT per game in the second column.
3) The caps ("most possible points") seem to be based on the fact that you are using a base 10 numbering system, rather than anything related to football.
4) I would not add anything to the statistics to avoid the divide by zero issue. The way to avoid that is to us logic; IF some denominator is 0, THEN use a default value, ELSE use some formula.
As I said, this all is merely my opinion; your mileage may vary.
Posted by: BradJolly  November 12, 2008 8:52 PM  Report abuse
The problem of coming up with a simple score to represent several difference scores is a very common. Lets say we have three scores: x,y and z. Now, there are many possible functions that can take several numbers and give you a single number. Our score s, could be computed like this:
s = sin(x+y)^z
(This is not a very good function to use.) Of course there are an infinite number of such things that we could construct. These types of functions have complicated behavior. To make this simple, one typical thing to do is to ask that the function be linear. This means that we use functions that look like this:
s = w0*x + w1*y + w2*z
The task is then to figure out what numbers (sometimes called weights) w0, w1 and w2 should be. Jeff had great instincts in going with a formula that looks like this. Using expert knowledge to pick the weights is certainly a valid way of going about things.
In a more formal science or probably even a business setting we would be worried about things like what does the score mean in some objective sense. Does it help us predict the probability that a team will win? This is where more math can help us by, say, helping us choose weights that have the greatest chance of predicting to what extect a particular quarterback helps a team win. However, real life is extremely messy. Sometimes the mathematics is up to the challenge but sometimes it isn't.
Posted by: mathlete  November 13, 2008 12:23 AM  Report abuse
In a Popper sense, the formula can't be falsified, so it's just an opinion decorated with a lot of math mumbojumbo, kind of like putting Christmas lights on your house.
Posted by: jimward21  November 13, 2008 7:26 AM  Report abuse
A football comment, not a math comment. If you look at the top rankings listed in the table in the article, one thing jumps out: the TDInt ratio is key. When doing a formula like this, it is often worth it to see if the results come out as intended. If not, then tweaking the formula might be the answer. Was it your intent that the TDInt ration have this much of an effect on the rankings? I agree it is an excellent addition, but I'm wondering if it is not rated too highly.
Posted by: Dougmacintyre  November 13, 2008 11:43 AM  Report abuse
Dougmacintrye  I do see what you're saying  that when looking at the Top 20, the more TDs and the less INTs a passer has, generally, the better he'll do. But completion percentage is still a big part of it (otherwise Mike Thomas would be higher) and so is yards per attempt (otherwise Billy Cosh would be much higher).
Overall, TDs and INTs have a significant impact on the rating (15 out of a possible 45 points), and the modified ratio is part of that, but hopefully its impact isn't too great.
Posted by: jeffnelson00  November 13, 2008 12:26 PM  Report abuse
BradJolly  I added 0.1 to the TDs in the "TDs per Attempt" category because for single games, I want someone throwing 0 TDs in 10 attempts to be differentiated from someone throwing 0 TDs in 30 attempts. I added the 0.1 to the Attempts in that category just so that the 0.1 in the TDs was accounted for (because theoretically, if someone is starting with 0.1 TDs, they need to have thrown 0.1 passes.) Maybe that wasn't absolutely necessary, but I don't think it creates a problem.
I added the 0.1 in the "INTs per Attempt" category because, again, I wanted to differentiate between someone throwing 0 INTs in 10 attempts and someone throwing 0 INTs in 30 attempts.
As far as the caps go, we decided on 10 in the "Yards per Attempt" category because 10 yards gets a first down. And if someone is getting a first down per attempt, that's outstanding.
We chose 1 per 10 in the "TDs per Attempt" category from looking at quarterbacks' numbers throughout the past few seasons (and not just the best quarterbacks, but the average and below average too). And it seemed those who throw 1 TD per 10 attempts are the elite.
Posted by: jeffnelson00  November 13, 2008 12:37 PM  Report abuse
The comments to this entry are closed.
In my opinion, this formula isn't very related to mathematics, even though it does use arithmetic in finding the ratings. This is a group of sports writers deciding on a method of comparing quarterbacks. The sports writers are choosing the stats that they believe should be included in the rating and then giving these stats a weight based on their perceived importance. The weights are based on the writers' knowledge of the game. Other experts may have differing opinions about the relative importance of the stats. The worth of this rating will be determined by how helpful it is to the writers in preparing their stories and how much Post readers like using this rating to compare local quarterbacks. It can't be determined to be "correct" in some mathematic sense.