January 2, 2011
December 31, 2010
December 22, 2010
College football decides its champion in a unique way that has become somewhat controversial because every other major sport in America uses a playoff. Over time, the sizes of those playoff systems have expanded, making college football stand in ever sharper contrast.
College football crowns its Bowl Championship Series (BCS) champion after pairing the top-ranked two teams in a single game. The top teams are determined largely by expert polls with some input from computer algorithms. The team ranked third often has a semi-legitimate case that it deserved an opportunity to play in the championship game, especially since the BCS formula has been repeatedly tweaked. The issue of fairness is a common attack thrown at the bowl tradition by playoff agitators. But fairness is impossible to measure. Or is it?
Any playoff system requires a cutoff that leaves a single team out. The wider the net, the more arbitrary that cutoff becomes (requiring ever more complicated tie-breaker rules). The result is that any playoff introduces another kind of unfairness. An 8-team playoff gives an arguably weaker team the chance to defeat a squad that was much better during the regular season. That may make for enjoyable entertainment, but it is definitely unfair in its way. The argument is that a playoff cheapens the regular season and all its games.
Professional football in the NFL uses a 12-team single elimination playoff to determine its champion. During the most recent Super Bowl, a team with a 9-7 regular-season record (Arizona Cardinals) played and nearly won. That kind of finale happens because with 32 teams in the NFL, more than one-third make the playoff cut.
And consider professional basketball in the NBA and hockey in the NHL. Sixteen teams are included in the NHL's Stanley Cup playoff - selected from just 30 teams. Likewise, in the NBA, 16 playoff teams are chosen from 30 in the league. Literally below-average teams make the playoffs in those sports every year. Is that fair?
Wide bracket playoffs reward casual play during the regular season. Instead of striving for excellence, the smarter strategy is to avoid injuries, especially during the late season games. Such a structure is one way of defining greatness, but is it the only way?
These kinds of arguments can be had in any sports bar in America. But now college football is under assault, with President Obama suggesting a playoff, and Senate hearings grilling the BCS as "un-American." Now that's a low blow. With all the sports statistics available, it's about time somebody took a look at what's fair using quantitative analysis.
The Fairness Index
Being number crunchers by training, we decided to create the Fairness Index. Our index measures the regular season record of a league's champion against its top team. Let's say the champ has a 12-4 record, while the top-seeded team went undefeated during the regular season, 16-0. That's easy. The fairness index for that league in that year is 75 percent. In another case, the top team might have a 15-1 record and go on to win it all, so the fairness index would be 15/15 or 100 percent.
How do you think the fairness index for pro football compares to college? If you think a playoff is fair, then you probably think they're about the same, right? Not even close.
97.2 % BCS era
96.3 % Pre-BCS era
Professional U.S. Sports
96.6 % Basketball (NBA)
92.6 % Baseball (MLB)
91.6 % Football
* The Fairness Index measures the average ratio of the champion's regular season record to its team with the best regular season record. For example, the average NFL champion has 91.6 percent as many wins as the team with the best record that year. Each of the professional average includes only the years with the current number of playoff teams.
The fairness index is much higher in college football (97.2 percent) than in the NFL (91.6). The higher level of fairness for college ball was true before the BCS, but is even more true today. Gnash your teeth all you want, but the one thing an NCAA football playoff would not be is fair. By our estimate, it would be about 5 percentage points less fair. Translation: the odds of the best team winning the championship would be 5 percentage points lower.
Critics will point to the fairness index for basketball, which at 96.6 percent is roughly the same as the BCS. The NBA seems to prove that a playoff does not mean a low fairness score. But wait, these are different sports, and the playoff design is critical. Three reasons why the NBA index is deceptively high: its playoff follows a best-of-seven format for each series, better teams enjoy home-court advantage, and scoring is frequent (thus less subject to luck rather than talent). The better question is whether the fairness index would rise or fall outside of a playoff. If only there was a way to test that.
But there is!
Fairness Declines as Playoff Bracket Expands
Consider the problem of "playoff creep." As you increase the number of teams in the playoffs, you increase the likelihood that the best team will not win, since they will face more chances to be upset by an inferior team. Most plans for a college football playoff imagine a small number of qualifying teams, maybe four, maybe eight. That way the regular season would still matter to a great degree, and the top teams would not have to face as many playoff games, pivotal injuries, and possible upsets. Realistically, it is unlikely that the bracket would remain small. History says so. Look at the NCAA I-AA (now FCS) football playoffs. Look at the bloated NCAA basketball playoffs. Look at the NBA and NHL mentioned above. Over the decades, they all suffered playoff creep so severe that the regular season is now little more than a pre-season.
But no case of playoff creep is clearer than Major League Baseball. Once upon a time, the Pennant Race was as hallowed and glorious as anything in Sport, and it meant simply finishing the 162-game season with the best record. Winning the World Series was icing on the cake, sure, but it wasn't the cake. Only the team with the very best record in the regular season won the Pennant. Period.
Until 1969, the best team in the American League won its pennant, same for the best team in the National League. That was what determined the two - two! - teams that made it to the World Series. And winning the pennant wasn't just some kind of cheap semi-final for the Series, it was an achievement all its own. From 1969 to 1993, the same logic applied to four divisions. Then in 1995 they converted to a full-blown 8-team playoff, complete with wild cards.
Guess what happened to fairness? Playoff creep in Major League Baseball is associated with a clear decline in the fairness index, from an average of 97.5 percent up until 1968 (the highest in our data), to 95.5 percent when the playoff was introduced, then 92.6 percent when the playoff bracket was expanded after 1984.
A College Football Playoff?
The playoff creep that occurred in those sports is almost certain to hit college football, and with it, a decline in fairness. Again, lower fairness scores with bigger playoff systems means without a doubt that more teams with worse records will get crowned champion than before. It's happened in other sports, which the data proves.
The consequence is that regular season records will not matter. The difference between the BCS and NFL average fairness index is equivalent to one less win in a season. One way to interpret that is that a single loss will essentially have no consequence on a team's chances to be recognized as champion. In other words, every game in the regular season will be not only less important than it is now, but unimportant. There is no question that a playoff would reduce the importance of the regular season.
Famous games such as the series of Florida State-Miami games in the early 1990s would be changed from elimination games in the National Championship chase to warm-ups to possible future play-off matchups. Those games would still have appeal to fans of those teams but would lose their appeal to much of the nation.
College football is unique, though many wish to change it and end its traditions. College football has a more important regular season than any other sport. And because of that, college football has rivalries that maintain their intensity while those of other sports fade. The one knock on college football has long been that its championship crown just isn't fair. But now you know the real story.