By The Numbers: Predictions for 2016 College Women’s Rankings

Building out a predictive numerical model for ranking college teams.

January 13, 2016 by Scott Dunham in Analysis with 5 comments

Creating expectations for the college ultimate season has always been a precarious task, fraught with challenges.

While the relationship between statistical analysis and ultimate is still in its infancy, it seemed sensible to try to bring a bit more science to the predictions process, with the idea that a new perspective might shed light on what we can look forward to come spring, as well as what is important to team development. Also, what fun is a surprise performance without an expectation to outperform?

The following is an attempt to use past performance with adjustment for expected changes in personnel to set the bar for 2016.

A year ago, as a tool to help make preseason predictions for the college women’s 2015 season, I used statistics from the 2014 College Championships to calculate what fraction of goals, assists, Ds, and turnovers could be attributed to players that were returning the next year. Also, seeing evidence for how additions of outstanding freshmen (and transfers and grad students) had an immediate impact, I compiled those numbers as well. Along with some ad-hoc weighting, I used this data to make some preseason predictions. The approach worked reasonably well, but was unsatisfactorily arbitrary, as I just picked weighting factors that seemed reasonable.

This year, I conducted the same analysis of the 2015 Nationals statistics, but realized I could use the 2014 data to generate objective weightings. Because they were available and seemed most relevant, the factors which I included were:

1. Power rating for each team. To keep it simple, I just used the final end of season rating.
2. Fraction of goals and assists represented by players that were returning the next year. Some data on Ds and turnovers is available, but was taken inconsistently and thus seems unreliable. For teams that did not make Nationals, I made estimates and thus the predictions for these teams are expected to be less reliable.
3. My measure of the strength of experienced incoming players. Here, there is of course some subjectivity. My scale is that 1.0 represents the addition of a strong U19 worlds player or the equivalent. Thus, a team adding a junior worlds player plus another that stood out at YCC might get a 1.5 in this category. I included any new grad students or impact players returning from injury that I was aware of using the same scale.

For the 2014 data, I used linear regression to calculate the weighting factors for each of the above.¹ The first interesting result of this regression was that the weighting factor for goals returning (A) was nearly zero (actually, very slightly negative). A simple explanation for this result could be that it is a lot easier to develop receivers than throwers. However, I would suggest that another component is that the most effective cutters are throwing as well as receiving threats. Those that are dual threats end up with a large number of scoring opportunities (especially in proportion to their touches), and so may be actually over-weighted in the assists returning factor. This may compensate for the obvious positive impact of strong cutters who do not have a significant number of assists.

Because the weighting for goals returning was so small, I neglected it in the model used for forward-looking prediction (A=0). The resulting parameters are B (assists)=542, C (new players)=93, and P0 (a measure of the ‘break even’ point)=0.75.

Putting these numbers into perspective, losing players accounting for 10% of the previous year’s assists can be expected to give a drop of 54.2 points (B*0.1) in power rating , and a team should bring back more than 75% of their assists (or add the equivalent in skilled rookies) to expect improved results for the next year. It is notable that the ratio of C to B is 0.17. This suggests that adding a junior worlds level player is equivalent to losing more than 1/6 of the team’s total assists and can be expected to raise a team’s power rating by 93 points. There were only a few players at nationals who significantly exceeded this level of impact for their teams in 2015 (e.g., Alika Johnston, Angela Zhu, Kelsey Fink, Han Chen, Lucia Childs-Walker, Julia Bladin, Emma Kahle, Stephanie Williams, Stevie Miller, Jennifer Corcoran, Corinne Dunwoody).

Using the extracted parameters, I made predictions for the results of the 2016 college women’s season, as shown in the table below.

The analysis gives a clear ordering of the predicted semifinalists, with Oregon the overwhelming favorite to repeat and British Columbia their top rival. Because of Stanford’s outstanding results in 2015 and strong recruiting class, they show up at #3 despite losing players who accounted for a majority of assists in 2015. Next is Dartmouth, boosted by their top recruiting class. The two teams predicted to compete with these four for a semifinal berth are Whitman and Central Florida, who are essentially tied for 5th in predicted 2016 power rating. Ohio State and UCLA are also nearly tied in predicted ranking and complete the set of predicted quarterfinalists.

Virginia is next at #9, but after that many of the predicted rankings are very close. Throughout the rest of the list, teams tend to be bunched in groups of 2-4 around the same projected PR. For example, Middlebury, Wisconsin, and Pittsburgh at #17-19 are separated by a mere six points in projected PR. The list was extended beyond #25 to the top 27 as Florida State and Michigan were so close to California, all just behind Minnesota. However, given the lack of statistical data for non-Nationals teams, the results in this range should be considered highly speculative.

Compared to Ultiworld’s subjective Power Rankings, this analysis ranks Stanford (3 v. 5), Ohio State (7 v. 10), Colorado College (12 v. 19), Tufts (15 v. unranked), Middlebury (17 v. unranked), and Wisconsin (18 v. 23) notably higher while Pittsburgh (19 v. 13), Texas (20 v. 9), Kansas (21 v. 14), and Michigan (27 v. 18) rank significantly lower in my scale.

If we take these rankings as given, the predicted bid allocations are NW 6, NE 3, SW 2, SC 2, NC 2, OV 1, SE 1, AC 1, GL 1, ME 1. Since Pitt at 19 is nearly tied with Middlebury and Wisconsin, the 2nd NC bid or 3rd NE bid could easily shift to OV or to SC, which has Texas and Kansas at #20 and #21. The SW with Davis at #22 and California at #25 would also seem to have a significant chance to pick up a third bid. With Tufts at #15 in addition to Middlebury at #17, the 3rd NE bid seems the most tenuous in the initial projection, followed by the 2nd NC bid and the 6th NW bid, with WWU and Victoria at #14 and #16.

It will be fun to see how these predictions, based as they are on limited data and a simple model, do compared with subjective predictions which include Fall tournament results. However, even before the season begins there are a number of takeaways from this analysis:

1. Crunching the numbers gives what seems (at least to me) like a pretty reasonable prediction for the upcoming season. With more statistics (and statistics for multiple years) becoming available in the future, the use of numerical models for predictive ratings seems promising.
2. There is a clear set of elites in the College Women’s field, but once you get past the top 8 or 9 teams, there is a tremendous amount of parity. The battle for bids in 2016 should be an exciting one.
3. Having effective throwers appears to be the dominant key to success in college women’s ultimate.
4. Bringing in skilled rookies (or grad students) has a huge impact. This suggests that one of the most effective methods of long-term development may be building and supporting (and then recruiting from) girls’ ultimate programs in the areas schools draw matriculants from.

I only compiled the data for college women. If someone else wants to compile the data for men (or club), I would be happy to work with them to generate a similar analysis.

Disclosure: My daughter is a freshman at Stanford University playing for Superfly.

1. This was the regression used:

   PR2015 = PR2014 + A*(GR-P0) + B*(AR-P0) + C*NP

   where PR2015 and PR2014 are the end of year USA Ultimate power ratings, GR and AR are the fraction of returning goals and assists, and NP is my estimate of the strength of new players being added for 2015. A, B, C and P0 are fitting parameters to be determined by minimizing the mean squared difference between the 2015 PR calculated from the equation above and the actual 2015 season results. ↩