Ultiworld globe logo

DIII Algorithm Trouble

by in Editors' Blog with 12 Comments

In the comments on our post about changing the ranking algorithm, Dan C. vented some frustrations about the bid allocation in the DIII division.

The completely computer generated algorithm has proven frustrating on the DIII level as well, especially with a general lack of non-region play in the Metro East and New England division. Do they deserve 3 bids a piece when hardly any of those teams have played outside their regions? I disagree. Looking at last years Nationals results the NE and ME region teams placed T-7th, T-9th, T-11, and T-13 where as NC and NW teams (who both faced more diverse competition throughout the season) placed 1st, 2nd, T-5th, T-5th, T-7th, and T-9th. I guess more than anything I’m a NC player who is disgruntled that the region is currently slated to receive just 1 bid when they have 3 very deserving teams in GOP, St. John’s, and St. Olaf (teams who have all played many non-regional games) while a two seemingly less deserving regions (in terms of non-regional play) in ME and NE are earning several bids a piece. Maybe similar results at Nationals this year will sway USA Ultimate to make changes to the bid allocation system. Here’s to hoping…

About Charlie Eisenhood

Charlie Eisenhood is the editor-in-chief of Ultiworld. You can reach him by email (charlie@ultiworld.com) or on Twitter (@ceisenhood).

View all posts by Charlie Eisenhood →

  • Mitch

    DIII teams just don’t travel as much as D1 teams, on average. I recall looking at a score reporter seeding chart for nationals a year or two ago and there was I think 3 games total between all 16 teams where as the D1 seeding chart had dozens, maybe even 100 games total for 20 teams. Right now, I think DIII bids should be either equally allocated or something simpler along those lines until there is enough connectivity in the algorithm through increased play.

  • BD

    I’m not sure I really agree that the algorithm is failing the NW and NC, didn’t each of those regions receive 3 bids last year? Surely we don’t want to go back to a system where each region is allocated bids on LAST YEAR’s results instead of the regular season from this year…

    • digdog

      The main problem seems to come from teams not playing very many games. For example Rice played at d3 warm-up then didnt play again untill the last weekend of the season, in some last min scrimmage. Seems to me they cost the SC a bid by not playing.

  • http://www.facebook.com/rthompson Ryan Thompson

    Part of this is that those ME and NE teams have only played one tournament – most are slated to play this weekend. Wentworth won’t, and they’ll lose the NE a bid. Those extra games from this weekend will help more accurately place those teams.

    But NC and NW teams haven’t really done themselves many favors with their records this year. D-III Warmup actually seems to have worked against them, since these D-III teams have had fewer chances to actually play against D-I teams and slot in higher in the rankings.

    The bid cutoff right now is 1162, which is #124 in the full rankings.

    Part of the problem is also that the middle of the rankings are much less accurate than the top – maybe Texas A&M and Stanford should be in over Whitman and UNI, but maybe not. But the middle section of the rankings is rife with uncertainty.

    What team is better – North Texas at #39 or Illinois at #60? They’re 134 ranking points apart – the same as Florida State at #9 and Northern Iowa at #18.

    • Dan C

      I agree that NC and NW teams do not have as impressive looking records as NE and ME teams and, in that, they have not done themselves any favors. However, I think that their non-regional play should hold more weight in the algorithm. Or perhaps a new algorithm should be developed for DIII teams because they face more difficulties in attaining non-regional games (this is to address Mitch’s comment).

      We don’t want to go to a prestige system that rewards teams based on their previous season’s results, but my point about the NW and NC teams being stronger than NE and ME holds true to the results at this years DIII warm-up. All NC and NW teams placed in the top 6 while all NE and ME teams placed in the bottom 5 (to address BD’s comment). It’s frustrating that DIII wam-up results may work against those teams that participated because I thought it was an incredible tournament that offered DIII teams something they’ve struggled previously to get: a chance to see other National contending DIII teams early in the season.

      • Steve

        I don’t think Warm-up is working against those teams. I think it’s simply that the ME/NE teams are insular, and thus the few connections they have to the “outside world” tend to have more variance. Some years those will push the entire group down, some years they push the entire group up.

        This is an argument for capping the number of bids that a single region can receive.

  • Nathan

    Another problem with the rankings is the region strength vs team strength issue. Right now the South Central has five teams in the top 19, but only one bid to nationals. That’s not ideal. The problems the computers have are at the margins for DI teams, since they’re constantly playing each other and out of region games. Relying on the computer for DIII teams that rarely travel outside their region is considerably worse.

    These aren’t problems that are easily solved though. The human element may help more at the DI level, where there is more news coverage, available games, but it doesn’t do as much for DIII teams. For example, Ultiworld, which is the premier Ultimate site for college rankings still has Claremont ranked #2. Though they had a very good DIII Warmup tournament, they placed five spots behind Occidental at TiV. Not saying Occidental should be ahead of them, but either Occidental (unranked) is considerably better than Ultiworld says, or Claremont isn’t nearly as good. It’s gotta be one or the other. Meanwhile, they move St. John’s down six spots, after a Chicago Invite that was considerably more impressive than their performance at Warm Up. This might be due to their bad time at Huck Finn, but they were missing their top players for that tournament, something the “human element” is supposed to account for that the computers can’t. I’m not saying they’re doing a bad job, they do a great job. No matter how you rank DIII teams, since there will always be teams that are difficult to place. It’s just easier to tear down rankings from the human side, because they’re more subjective, and aren’t as easily explained.

    My point is, not only is there no perfect solution, I don’t even know if there’s even a halfway decent solution. Maybe somebody that’s better at math than me can come up with a better formula, that also somehow accounts for teams not leaving their region. But I don’t think that’s possible, so I think we’re stuck with what we’ve got.

    • Dan C

      Well stated, Mr. Jesson.

      I would argue that an even bid allocation system is a halfway decent solution because it allows for the best teams from every region to make it to the National tournament. For instance, say those regions currently allocated a single bid have the two potential National finalists. With the current system, the regionals finals would essentially be the National championship in this scenario as only one of these two dominant teams make it to Nationals. However, having two bids from each of the 8 regions disables this (quite outlandish and very theoretical) possibility from happening. My feeling is that the current state of DIII Ultimate needs to be treated differently than DI due to obstacles that DIII teams face. The even bid system provides a solution that is equal for everyone. Arguments made against this system parallel arguments made about the current system: unproven regions are earning multiple bids. The even bid allocation system limits these “unproven” bids to 1 as opposed to 2 or more.

      As a proponent of wholistic thinking, give me some more arguments against a even bid allocation system for DIII.

      • Nathan

        I definitely see the advantages in the system you’re proposing, but as you stated it has some drawbacks as well. For example, if we implemented this system this year the DIII SE Region would get two bids. Right now Georgia College is the top ranked DIII team in the SE at #42, while Rhodes is second at #66. With an even bid allocation system for DIII you would be lowering the overall quality of play at DIII Nationals. For a Division that is striving for recognition, this could potentially hamper that progress. However, I think it’s fair to say that your idea does make it more likely that we end with the best team winning it all at the end of the day.

      • anon

        except that there are 10 regions. Are you suggesting an increase to 20 bids to Nationals then?

        • Dan C

          Wow, I’m dumb for saying there were 8 regions. Sorry for that. But I suppose a reasonable solution to that would be to combine DIII regions that make sense geographically or that make sense in terms of number of competitive teams. For instance combining GL with either NC or OV (probably with OV based on geographical distance), combining SE with AC, or SW with SC or NW (creating a gigantic region that may cause problems for transportation). I think the overarching argument I’m making throughout all of this discussion is that DIII Ultimate is a different brand than DI and needs to be treated as such. Whether this means constructing new regions, adjusting the algorithm to account inter-regional play differently, or changing the bid allocation system is up to USAU. I’m just advocating a call to change.

  • http://www.facebook.com/jesse.moskowitz.1 Jesse Moskowitz

    I love this dialogue. Having followed the DIII world all season I can’t tell you how frustrating it is to distribute tangible value to DIII results. There are a number of problems I’d love to address when I sit down and have the time. Thanks for bringing up this issue. As many have said, hopefully we can reach a point where the algorithm can work practically to serve our unique division of Ultimate.

Find us on Twitter
Find us on Facebook