Better Box Score Metrics: The Most Impactful Offseason Moves

Plus projected win totals for the season.

April 27, 2022 by Paul Würtztack in Analysis with 0 comments

With the AUDL about to start up again, it’s time for a quick review and preview. Here are today’s questions:

1. Which player movements in the offseason were the most significant, in terms of the relative impact on former and future teams?
2. Considering all the movement together, as well as realignment, how many 2022 wins does a “player-movement” model predict for each team?

Now isolating the effect of any one player on a team’s past performance, much less future performance, is squishy and requires a lot of hold-your-nose assumptions. Still, what’s the point of an analytics column if you don’t want to be stupidly speculative?

To answer the questions, I’ve evaluated the movement of 124 players, including those switching teams, retiring, coming back into the AUDL after not playing in 2021, or out for the season with an injury.¹ I’ve relied on the currency of EDGE_100 (E100), a measure of a player’s productivity per possession and block opportunity, which I’ve set against the background of the relative efficiencies of the old and new teams.²

The principle at work here is akin to that of the “replacement player,” although instead of indexing players to a uniformly low benchmark, I’ve set team-specific replacement levels relatively high, on the theory that any one player has to be very good to move the team needle, and if an added player is not as good as the ones already on the field, he won’t play enough to make a discernible difference. Dividing the team-level EO100 (offensive efficiency) and EB100 (block efficiency) by eight seemed adequate for this purpose. If a player’s combined E100 value was greater than the replacement value, that player was presumed to be in heavy rotation and the positive difference was added to the team’s 2021 scoring differential. If the player’s E100 was less than the threshold, the value was not counted rather than subtracted. In this exercise, EB100 values (in the absence of any better measure) represent the prevention of scores to the opponent.

In reality, the holes created by departing players are often adequately filled through the development of players already on the team, or the addition of rookies. I do not attempt to measure this directly, but do modulate the effect of departures at the team level to account for it. Two other issues required some creative accounting:

1. Lapsed veterans. A number of teams are adding AUDL vets who did not play in 2021. Without yardage totals in their stat sheet, I cannot directly assign them an E100 value. However, I have estimated their E100 values using NetStats, as described here.
2. New teams. I have no basis for projecting the baseline competitiveness of the three new teams. Given that there are no player departures, only arrivals, it seemed necessary to set a low baseline, commensurate with a lower-tier team. At the player level, the baseline Replacement Values for expansion teams were set equal to those for the Aviators. In the model for projecting team wins, the baseline mean and standard deviation for goal and goals allowed for the expansion franchises were set to equal to Philadelphia’s 2021 season.

Beyond this, it’s naturally safe to assume, as I have, that there will be no changes in roles, team tactics, injuries, players missing games, and other variables that could change the fortunes of a team’s season.

As an example of how this shakes out, let’s consider the high-profile move of Antoine Davis from Atlanta to New York. Atlanta’s replacement level EO100/EB100 split was 8.2/1.8. In 2021, Davis had a split of 9.0/5.7. Although he was slightly above replacement level for offense, a function of having both elite scoring ability and the second most turns on the team, his block efficiency rate was Atlanta’s second highest.

A player’s effect is also scaled according to how much he actually played relative to other players on the team. Davis played a high percentage of Hustle points last year, indicating that he played a sizeable role in setting the Hustle’s team efficiency. In contrast, Dallas’s team efficiency was marginally affected by Chris Mazur’s high efficiency rates, simply because he played so few of the season’s points. His departure therefore is not expected to make as big a hit on Dallas’s overall efficiency and scoring differential.

All told, losing Davis is estimated to cost Atlanta 0.62 in goal difference per game. And the effect of Davis on the Empire? For a player’s future team, I set equal expectations: 20 points played for every game, in the same ratio of OL/DL as the prior season. New York’s EO/EB100 split was 8.9/1.7, meaning that the impact of Davis joining New York is slightly less than leaving Atlanta; his net benefit to NY is 0.44 goals per game. The logic is that a great player can only do so much for an already great team, and that he would have a relatively larger impact on a weaker team. Had Davis decided to turn south to Tampa Bay instead of north to New York, he would have been projected to improve the Cannons score differential by 1.28 goals.

So which player movements are expected to have the biggest effect? We’ll break it down into several categories:

• The Goodbye list—those who have created the biggest hole on the teams they’ve left;
• The Welcome list—players likely to have the biggest impact on a new team; and
• The “Fortune Shifters” list—those players with the largest combined Goodbye + Welcome effect.

The Goodbye List

Starting with the Goodbyes, Table 1 shows the 40 with the largest estimated effect on their former team’s projected average goal differential. While many teams are represented, one can feel the pain for teams like Los Angeles, Oakland, Toronto, Montreal,³  Dallas, and Boston.

Table 1. The effect of a leaving player on his former team’s projected average goal differentia. The Goal Differential Difference is a function of the difference between the player’s offensive and block efficiency (EO100, EB100) and the team’s replacement value (RV) as well as the proportion of team points played in 2021.

Focusing at the top, Harris, McDougall, Kerr, Poitte-Sokolsky, and Halkyard were not just key players; they each had the highest E100 scores on their respective teams and played a ton. They will be leaving crater-sized holes, and the excitement of any new season is to see how well the team is able to shovel them full. In all, 81 players were estimated to cause a sting by not suiting up for their 2021 team in 2022.

The Welcome List

Without the ability to evaluate “rookies” who had not previously played in the AUDL, 54 player movements were estimated to make a positive impact on the new 2022 team (Table 2).⁴ Given the premise of the exercise, most of those on this list are going to teams that did not make the playoff in 2021. Those going to expansion teams tend to be concentrated in the top half of the list, a function of setting a low initial replacement value for those teams. Players going to 2021 playoff teams tend to be concentrated at the bottom of the list. Again, there’s only so much one can do in joining a good team that is already pushing the ceiling of the sport’s efficiency range, except perhaps fill the hole of a departed, which is why New Team replacement values are adjusted to account for the loss of departed players and are typically smaller than the RVs in Table 1.

Table 2. Projected benefit of players to their new team, measured in average game goal differential for the team. Projections are based on each player playing 20 points per game in the same proportion of OL/DL as in their most recent AUDL season. New Team RV EO100 and RV EB100 represent the adjusted team efficiencies after accounting for departing players.

The Fortune Shifters

Consider an Oakland vs. Salt Lake game in this coming year. It’s projected that Salt Lake will improve its scoring margin by 3.1 goals just from luring Jordan Kerr away from Oakland (Table 3). This is rare leverage; only six players generate better than a two-goal swing, including Drew Swanson (Chicago to Seattle), who does so based on his prodigious block rate. Table 3 shows the top 10 in combined effect of leaving one team for another, although Cam Harris makes the list just on the impact of his not playing this year for Toronto.

Table 3. Players with the largest combined effect (measured in projected goal differential) of leaving one team for another.

Win Predictions

What is the net effect of all the movement? To get an estimate, I summed up each team’s credit and debits in terms of offensive and defensive point differences.⁵ For each, I added them to their 2021 per-game average, split between home and away games. Using the new expected goals and goals allowed totals, along with the standard deviation of their 2021 averages, I simulated the 2022 schedule 25 times. The average of those runs result in the team’s expected win total for the season. Again, the team projections are based on the assumption that all players will play all games.

As an example, let’s look at whether it’s possible that Detroit did enough in the offseason to likely end its losing streak. In 2021, its average home score was 14.2 – 26.5 and its average away score was 16.7 – 26.8. The glimmer of hope is that there was very high variation (StDev=5.47) in what their opponents scored against them at home (remember 34-10?). This means if they can narrow the expected point differential a bit though player acquisition, enough stars may align for a win or two.

The main additions of Chris Walsh from Indy and Johnny Bansfield, Nathan Champoux, Jake Steslicki, and Mark Whitton from their past, coupled with no significant personnel losses, gives more hope. Walsh in particular had a very strong 2021 with an E100 rate of 11.7. This would make him a productive addition to any team, but going to the Mechanix is reason for a parade, given that their replacement value of 4.0 is half that of the Empire’s 8.9. Combined, these five additions to Detroit are expected to add 1.5 goals to the Mechanics score while reducing opponents scores by 1.56. This 3.06 swing is the largest in the league, except for the expansion Shred and Summit and reduces their average margin of loss from 11.1 goals to 8. That still seems like a lot, but recall that they came close last year without the new pieces, and the high variance/stars align factor in the model could result in predicted wins. Let’s hit go and see what happens.

Detroit Wins!

The streak ends. Detroit scored at least one win in 19 of the 25 simulations. (Pittsburgh was the most frequent, but not only, victim). As shown in Table 4, the simulations predict anywhere from 0-4 wins for the Mechanics, which is actually a better forecast than that for the Cannons, who will be competing with no big-name acquisitions and the loss of Andrew Roney and Bobby Ley in a division full of solid teams.

Table 4. Model output for the effect of player movement on projected team wins. Projected wins are the average of 25 simulations of the season’s schedule; “low range” and “high range” show the highest and lowest win totals of the simulations.

At the other end of the spectrum, Carolina, DC, and San Diego are projected to have the best regular seasons at 10-2, and add to them Chicago and New York as teams that ran the table at least once in the season simulations. Every 2021 playoff team appears to be in good position to return to the playoffs, except one: Dallas. The combination of losing so many top players and joining a highly competitive division results in a projected three-win drop compared to 2021.

But Dallas doesn’t suffer the biggest fall. That would be Montreal, whose main sin was to rejoin the East and play Boston, New York, and DC for half their games. Treating their 6-2 Canada Cup record as a 9-3 full season record, the Royal are projected to win only five games in 2022. Madison and LA are projected to each drop two games more than they did in 2021.

If we exclude the expansion teams, the hot-stove league award for gaining the most projected wins goes to Pittsburgh, where the incoming combo of Ian Engler, Mike Pannone, and Noah Robinson⁶ (if they actually play) more than makes up for the departure of Thomas Edmonds. The Thunderbirds are projected to win four additional games this year. Counterintuitively, Indy is the next biggest gainer, even though they are battling for division wins with Pittsburgh.

Carolina’s four losses in 2021 (all by one goal) were more than would be expected from their overall point differential, and the cosmos evens things out this year with an expected additional two wins, despite the season-ending loss of Allan Laviolette to injury. Seattle is the only other team expected to improve by at least two wins, although that is heavily Rehder-dependent. A more sophisticated version of this exercise might value equivalent gains by a group of players more highly than by an individual, on the basis that the gains might be more stable across all games of the season.

In the battle of the expansion teams, the model proves to be a complete cop-out, projecting each team with a 7-5 record. On the other hand, it’s hard to look at the comparable lists of pick-ups (often at the expense of division rivals) and easily pick out the most impressive grouping. Given that each franchise has a well-developed local talent pool to build on, the projections seem reasonable, even if boring.

One of the premises of this exercise is that it considers a player’s efficiency to travel intact to another team. But a player’s efficiency is also a reflection of the team around him: handlers benefit from better cutters and vice-versa, the opportunities to be productive are greater if teammates are not turning it over excessively, and so on. One of the opportunities of “The Yardage Era, Season Two” now upon us is that player movement may be a particularly good way to help estimate the magnitude of the team effect. This is something I’ll probably come back to in the future, even at the cost of reminding us that I made some season predictions I may wish were long forgotten.

1. I’m grateful for Ruffner & Cohen’s two “Swing Pass” podcasts this week for the thorough review. Thanks also to Alex Rubin. ↩

2. In order to easily take advantage of the AUDL’s new official possession data, I have simplified the EDGE-O calculation by assigning a league-wide value of 0.51 goal equivalents for every turnover rather than using game-specific values; the difference in outcomes is minimal. I have also slightly shifted some of the weighting from the “scoring premium” to yardage. This is because the scoring premium is there to account for a theoretical “nose for the goal” ability, in which a player scores much more frequently (or infrequently) relative to their yardage than other players. A full year’s worth of data showed that that variation declined with increasing yardage, and that therefore we should reduce the weighting of the premium. This had the side benefit of simplifying the equation to EDGE-O (Goal Equivalents) = (0.007*total yards) + (0.2*scores) – (0.51*turns). We can call it the Bond Rating. EDGE-O100 is EDGE-O per 100 possessions. EDGE-B = 0.51*blocks. ↩

3. Although rostered, the expectation is that Sascha Poitte-Sokolsky will miss significant amount of the season. For most players, this was not considered, but given his outsized influence on the Royal, it seemed prudent to reflect this in his case. ↩

4. Given that Portland’s rookies included two World Games alternates and a U24 national team member, it seemed excessively distortive to not include any values for Raphy Hayes, Jack Hatchett, and Leandro Marx in the database, so all three were assigned proxy E100 values in the high, but not the highest, range. ↩

5. At the team level, because I can only account for incoming players who previously played in the AUDL, the cost of all departures from a team is divided by three in order to balance the overall league effect of comings and goings, and to account for the fact that players already on the team often adequately fill the hole of a departed player with more development and playing time. ↩

6. Several players had relatively small samples sizes (e.g. Nethercutt) or data from no more recent than 4 years ago, but Robinson was both; his huge offensive efficiency rating has been reduced by 20 percent to avoid overweighting. ↩