Elo Ratings System
NAQT's current team rankings are computed by an algorithm similar to that used for the Elo Ratings in chess.
There are two types of ratings, preliminary and established. Teams that have played 19 or fewer games on NAQT questions have preliminary ratings, while those with at least 20 games have established ones. Broadly speaking, established ratings should be given a relatively high level of trust, as 20 games should be enough to determine the team's ability. Conversely, preliminary ratings are based on relatively few games (almost certainly from only one or two tournaments) and may fluctuate wildly.
Every team starts with a rating of 1200 which remains constant until it has played three games. At that point, the team's rating changes to the average rating of its opponents (when it played them) plus 400 times the quantity wins minus losses divided by games played. Thus a team that had beaten a 1400-ranked team and lost to a 1600 team, a 1200 team, and a 1000 team would have a rating of (1600 + 1400 + 1200 + 1000)/4 + 400*(1-3)/4 = 1300 - 200 = 1100.
Once a team obtains an established rating (by playing its 20th game), its rating will change more slowly. The change in a team's rating as a result of playing a game becomes 32 times the difference between the "actual result" and the estimated probability of a win, given the teams' ratings. The "actual result" is considered to be 1.0 for win and 0.0 for loss. The chance of winning is given by 1/(1 + 10d/400) where d is the difference in teams' ratings. So if a team with 1600 rating defeats a team with a 1400 rating, the 1600 team will earn 32(1 - 1/(1 + 10200/400)) = 7.7 points, while the loser will lose 2.8 points. Thus the most that a team can earn from a single game (defeating a team ranked hundreds of points above them) is 32; conversely the most one can lose (by being defeated by a team ranked hundreds of points lower) is 32. Defeating a team with the same record earns 16 points.
All ratings are reset at the beginning of the school year.
NAQT does not take into account which players are on which team. NAQT considers all "A teams" from the school to be the same entity. If a school sends only one team (and therefore doesn't need to label its team), that team is considered equivalent to its A team at other tournaments.
NAQT currently treats teams with alternative names (e.g., "Larry," "Moe," and "Curly" rather than "A," "B," and "C") as distinct teams. At some point in the future, this will probably change and such teams will simply be mapped to "A", "B", and "C" in the order of their finish.
Because NAQT is rarely able to load a year's results in chronological order, NAQT recomputes its ratings from the beginning of the year whenever data from a new tournament is available. This may mean that teams' ratings change without their having played additional games (because teams they played may have had additional results loaded thereby changing their ratings).
Not all tournaments are included; some tournaments never report results, and others report incomplete results that can't be loaded. NAQT intends to make a good-faith effort to load complete tournament results promptly, but may not always be able to.
NAQT includes the results of all official games, including tiebreaker games. Scrimmage matches are not counted.
NAQT does not currently make any decisions (including HSNCT wildcard applications and seeding) based on these ratings.
NAQT does not, as of April 2009, posit that these ratings are a true measure of the best teams in the country. We look forward to gathering more data, offering more statistical analyses, and sharpening our algorithms over the years, but for now, having the highest rating should not be officially reported that NAQT believes a team to be the best in the country and/or the likely winner of the HSNCT.
We welcome questions about these ratings and would love to hear people's opinions on how to improve them. Please don't hesitate to contact us at results@naqt.com.