Better Box Score Metrics: The AUDL’s Offseason Data Upgrade

The AUDL updated their slate of stats for 2022. Does it help us better understand players and teams?

April 6, 2022 by Paul Würtztack in Analysis with 0 comments

The AUDL may have added three new teams, but the real offseason news is that the league revamped its player stats page and now offers some very welcome new data and functionality.

One year after changing the ultimate stats landscape by bringing us individual throwing and receiving yardage for every game, the AUDL has, among other enhancements, added offensive possession totals for each player, as well as the option to present every stat on a per-possession basis.

As said often in this column, per-possession rates can offer a fairer basis of comparison than per-game or per-point rates, given the disparities in offensive opportunities between O-Line and D-Line players, or players on efficient and inefficient teams. The AUDL site does not yet offer “block opportunities,” the defensive version of possessions that is needed for calculating block efficiency rates, but it would seem like an easy addition.

Before continuing with player stats, I will mention that there have also been upgrades to the team stats, including some additional efficiency metrics and the ability to quickly compare a team’s own output with what its opponents did against them. It’s in the team stats that one finds the reasons for winning and losing, and are far more consequential in that regard than individual stats. The reason I spill more ink on individual stats is that the team stats are essentially “right” while player stats have typically been terrible. It’s the attraction of problem-solving.

In this vein, returning to player stats, the AUDL has added “OEFF,” which is the offensive efficiency of the team when that player is on the field. I don’t think this helps solve the problem it is designed to solve, which is to help capture “winning” qualities that we don’t or can’t measure directly, such as the ability to unlock an offense through more creative throws, drawing the best defender, or even “leadership.” One problem is that OEFF is entirely a reflection of the line-quality one plays on, particularly since it is only giving the “on” half of the on/off, “with or without you” numbers. Even within a team, it is difficult to ascribe meaning to the OEFF value. For example, in 2021, every AUDL team’s O-line was more efficient than their D-line, typically by about 5 percentage points. If the best player on a team were to play every point (theoretically, no fatigue involved), their OEFF would be lower than the OEFF of his teammates who only played O-line, for reasons that have nothing to do with their ability.

In addition, for offense, it is highly unlikely that one is going to pick up a signal for “intangibles” that is anywhere as significant as the offensive activity that we already measure directly. The best potential use for a stat like this is where we don’t have robust direct measures, and that of course is for defense. I’m skeptical that we can glean very much good information without the ability to control for a number of factors, but if we’re going to test drive it, defense would be the best place to start.

Now if I mention that what the AUDL calls OEFF is what other sports refer to as +/- (or more specifically, the + half of OEFF), you may know where this is headed—back to me again beating that not quite dead horse. The problem is not just that the AUDL includes ultimate’s +/- stat (Goals+Assists+Blocks-Turns) in its menu of 25 columns of data, it’s that it is the one used as the default for sorting the player list. If you’re going to present the data with the inference that it is sorted based on “quality,” you have to use something better than +/-.

The traditional +/- statistic is horribly biased against handlers since throwing is about five times as risky as catching. If you filter the main player stats page for the 2021 season, the front page of the top 20 players includes just two who threw for more yards than they caught. Compare that to what we actually think about the value of good throwers: six of the 14 First Team and Second Team All-AUDL were majority throwers.

However, as I also discussed in my +/- column, an easy and effective option is to use NetStats, which adds hockey assists to +/-, not because HAs have special significance, but because they improve the sampling of overall activity and help balance out the higher risk of throwing. And it just so happens that the AUDL is now providing hockey assists as a column of data, so it’s an easy step. If one were to sort based on NetStats, you would find eight majority throwers among the top 20, consistent with the fact that they represent about 43% of all players.

A more specific way to show the bias is to use a Handler Index (HI), the portion of a player’s total yards that are throwing yards. The average HI for the top 20 in +/- is only 33%, while the top 20 in NetStats averaged a much more balanced 46%. Yes, NetStats diminishes the relative value of a block, which could be perceived to be unfair to D-Liners, but let’s not kid ourselves: +/- is not fair to D-liners either. The top 20 in +/- played an average of 8% of their points on D-line. (That Eric Brodbeck is on this list playing 40% D-line is remarkable). The best way to be fair to D-liners is to specifically account for the structural imbalance, whether through possession-based stats or other means.

Having a couple of columns for a Handler Index and D-Line Index is not only helpful for eyeballing the biases of any particular stat, it’s also a quick shorthand for any player’s role on a team. Here’s what it looks like for the all-AUDL first team:

From this, one can quickly see that, for example, Osgar and Williams, while both hybrids, are hybrids of a different stripe. Even if not of interest to the AUDL, I intend to offer these indices in our 2022 EDGE summaries.

As for other items on my on my AUDL wish list, I have three for the moment.

1. The ability to filter the data so that one can see regular season stats only, rather than regular season combined with postseason. It’s great that one of the new functions is to view all the stats on a per-game basis, which can help control for extra playoff games, but extra games against the best teams makes unbalanced schedules even more unbalanced and can introduce new biases that are unfair for players on those playoff teams.
2. In the same way that hucks are tracked as a certain type of pass attempt, I would love to see similar tracking of Hail Mary passes at the end of a quarter; not because they are an interesting stat—they’re not—but so we can remove the incomplete ones from turnover totals. They’re not turnovers. They do not prevent the thrower’s team from scoring, nor do they give the other team an opportunity to score. They are simply incompletions, with no ill effect. Counting them as throwaways distorts actual throwaway numbers, again at the expense of throwers called upon to sacrifice their efficiency numbers for the good of the team.
3. Wind data for each game. Even taken with just a phone at the beginning of each half or quarter. EDGE stats are based on the premise that the value of turns is related to their scarcity, and nothing affects scarcity like 20 mph winds.

In the end, it’s not about having more stats; it’s about having stats that better and more fairly reflect what it means to play well. The AUDL’s contributions to better data in the past two years have been transformative, and hopefully the women’s semi-pro leagues will soon offer similar ways to engage.