March 27, 2013 by Charlie Eisenhood in Livewire, Opinion with 27 comments
This week’s penultimate USA Ultimate college rankings are now posted. While for the most part the rankings look about right, there are some marginal discrepancies that range from odd to outright unfair.
Look at the cutoff of the Open Division at #18: Northern Iowa gets in to earn a fourth bid for the North Central. But they sit just nine points ahead of Texas A&M, a tiny advantage that could literally have been decided by one or two scores in a game.
Now, no disrespect to Northern Iowa, but who have they beaten? They got some strong wins at Huck Finn but they don’t have a single win — not even a single game! — against a top 25 team.
Texas A&M, on the other hand, has a win against USAU #8 Central Florida, and has played close against many of the country’s best teams. Their strength of schedule far exceeds UNI’s.
A&M has one bad loss — to Georgetown at Centex — but they beat them later in the weekend by a bigger margin.
Anyone looking at this situation would say that A&M deserves to be ranked higher than Northern Iowa. But not the ranking algorithm.
Now, you’re never going to have a perfect algorithm. Teams know at this point that wins matter, as do score differentials.
But it seems like we could reduce situations like this by changing the bid allocation process away from a purely mathematical computer approach to a blended approach like the Bowl Championship Series in college football.
There, a coaches poll, a sportswriter/player/expert poll, and a computer ranking get weighted equally to determine the BCS rankings. That means that human input accounts for two-thirds of the rankings, and data-driven algorithms determine the rest.
USA Ultimate’s rankings need some human input. I believe there is enough coverage of the sport at this point that journalists, coaches, and experts could put in ballots that would help balance out some of the marginal “mistakes” that the rankings seem to make.
I threw this idea out on Twitter and got mixed feedback.
John Cassidy said, “Yeah but football is on tv and a lot easier to see all teams where in ultimate it is impossible.” Jeff Rathburn echoed that sentiment: “despite the gr8 work you do, there is nowhere near enough coverage to make informed votes that affect bids to Nats.”
But I think that, for the most part, a wide range of regional representation would smooth out any discrepancies. Just looking at this example above, does anyone put UNI above Texas A&M? Probably not, if they’re being honest, and that might make the difference for A&M getting that last bid.
I don’t think there should be an expectation that people need to have seen every team on tape to be able to make a solid top 25 ranking. Rathburn also said that you’d need way more information about rookies playing in games, key injuries, etc. to make good decisions about relative team strengths.
While that would be great, and would possibly make a difference in rankings, I don’t think most coaches and players who vote in the BCS rankings are sitting down and watching hours of tape about teams across the country in a different conference. They’re looking at results — game scores and margins — just like the computer.
There would obviously need to be some work done to get the right structure set up to make this work. But a human element in the USAU rankings is critical to the fairest distribution of bids.
