May 4, 2013 by Charlie Eisenhood in News with 18 comments
We reached out to USA Ultimate about the seeding decisions in the Southwest Region, which have been scrutinized by teams and onlookers. Jeff Kula, the National Open College Director, and Richard Dana, the Manager of Competition and Athlete Programs, sent Ultiworld this detailed statement about the seeding process.
***
While there certainly isn’t a need to “defend” any of the decisions, USA Ultimate appreciates the opportunity for transparency in the process. Obviously there’s not enough time to rehash every seed from every region in excruciating detail, but given the specifics of the situation we’re happy to answer these questions to shed some light on the process and implementation.
Let’s start by describing the seeding method in general, as it will help to provide a context for understanding the rest of the statement. For seeding, decisions are made “from the top” based on CC results, meaning:
For the 1 seed, compare Stanford, ASU, SDSU [the three conference winners]. Once Stanford was seeded #1 …
For the 2 seed, compare LPC, ASU, SDSU. Once LPC was seeded #2 …
For the 3 seed, compare Davis, ASU, SDSU …
This process continues until all teams are seeded. To make these comparisons, the USA Ultimate rankings were used along with the seeding matrix.
Simply looking at this matrix provides head-to-head results and RRI. Additionally, by clicking on a square (where 2 teams meet), you can instantly compare their entire seasons, including common opponents, tournaments, head-to-head, dates, scores, etc.
This is done for every region and every team comparison across the country. This seeding method is best because it provides a high level of consistency and is as objective as possible. Any other system instantly breaks down because you’re required to compare too many teams, resulting in many circular comparisons that require a high level of subjectivity, or scenarios where head-to-head arbitrarily factors in depending on which 2 teams you choose to look at. What if 2 or 3 conferences have significant upsets? How does one rectify the seedings and determine who to bump up or down? The answer is … there is no answer, at least not one that can be applied uniformly across the country. The current seeding system provides consistency, and which teams enter the comparison is completely within the teams’ control based on performance on the field (i.e. at CCs).
There’s a lot of weight put on the regular season these days, which is great in terms of building excitement and allocating bids in a reasonable way. And it is an important factor in the specific cases of comparing teams across conferences when it comes to seeding. But once the Series starts, the general rule to determine which teams are compared takes precedence. Regular season means nothing if you get eliminated at Conferences. This explains both the Conference finish rule (determineswho to compare) and the importance of the regular season (how you’re comparing).
On to your questions:
UW: Why is SDSU so low? They have really played well in the second half of the season and appear, to us, to have earned a better seed.
USAU: SDSU is seeded 8th because, when comparing them to teams using the method above, that’s where they fell. SDSU ended up 8th based on individual comparisons. In looking back, they’re 9th in RRI and 10th in the USAU rankings, so this shouldn’t be too shocking.
Bottom line … seeding is meant to be reflective of the season up until this point, and not in any way predictive of what might happen. Just because a team is peaking in April and May, that shouldn’t override the rest of the regular season. (Sidenote: The rankings already take into account a decay function that rewards recent results.) The regular season is what a team uses to build its resume, and the Series is what a team uses to advance to the next level (though it does still factor in to seeding).
Perhaps the system tends to punish a favorite for losing more than it rewards an underdog for winning. There is nothing wrong with that … that’s what makes it a qualifying tournament. Doing the opposite would be punishing all the other teams who built resumes during the regular season AND took care of business in the Series.
Peaking in April/May is recommended for every team. But if you do it at the expense of the regular season, your team will need to qualify through the Series from a lesser seed. And there’s nothing wrong with that.
UW: A number of teams have been frustrated with this iteration of the seeding. Why has that been ignored, when it is a part of the seeding process (team input, that is)?
USAU: In the guidelines, team input is to be considered for seedings. There is no specified amount of time for this, but it should be reasonable. In the case of the Southwest, the initial mix-up between Arizona and ASU caused the first iteration to be somewhat meaningless, so things were delayed from there. That’s not a good excuse, just part of the explanation.
However, it’s important to realize that time for team feedback is counterbalanced with a need to get things out quickly. While teams deserve the right to give input, they also deserve the right to plan and strategize as long as possible heading into the weekend. Depending on the circumstances it can become a fine line, whether it involves unfinished events, waiting for team confirmations, or any number of other issues that can delay initial seedings.
The timing involved with the final iteration of seeding was not ideal. USA Ultimate has heard this concern and will use this and other communications to improve both this and other processes and programs in the future. In the SW, the arguments that have ensued would not have altered the seedings. A great deal of debate went in initially and nothing has been presented that wasn’t previously considered. However, team input is a critical element, and sometimes it does help shed light on seedings that we may have otherwise missed. No doubt that we’ll continue to try to allow for this part of the process, regardless of the circumstances, and improve in collecting it in both a timely and inclusive fashion for all teams.
UW: We believe we understand how Davis ended up at #3 (with head-to-head wins over ASU). But how does the seeding process deal with the fact that Davis has lost twice to Arizona (and more recently)? Is that simply not considered?
USAU: It’s simply not considered, at least not directly. The team’s results are still considered as a whole, so the losses to Arizona factor into rankings, common opponents, etc., and thus into the team evaluation. But since Davis and Arizona are never compared (see above), these results are not considered in terms of “head-to-head”.
In short, we need consistency. CCs can play out in too many ways to count, so once you start arbitrarily choosing which head-to-head results matter and which don’t, you produce a system that creates more problems than it solves. Does head-to-head only matter because Arizona was a top team in the region? What if they were the 12th best team? What if Arizona was a top team, but they’ve additionally gone 0-2 vs. the 12th best team? Now throw in upsets in 2-3 conferences, and you see how it spirals out of control.
Our job is to enforce a system that can be consistently and fairly applied regardless of team, region, results, etc. This method achieves just that, and fortunately, it is also more objective than suggested alternatives.
