Better Box Score Metrics: The Most Impactful Offseason Moves

Plus projected win totals for the season.

April 27, 2022 by Paul in Analysis with 0 comments

With the AUDL about to start up again, it’s time for a quick review and preview. Here are today’s questions:

1. Which player movements in the offseason were the most significant, in terms of the relative impact on former and future teams?
2. Considering all the movement together, as well as realignment, how many 2022 wins does a “player-movement” model predict for each team?

Now isolating the effect of any one player on a team’s past performance, much less future performance, is squishy and requires a lot of hold-your-nose assumptions. Still, what’s the point of an analytics column if you don’t want to be stupidly speculative?

To answer the questions, I’ve evaluated the movement of 124 players, including those switching teams, retiring, coming back into the AUDL after not playing in 2021, or out for the season with an injury.¹ I’ve relied on the currency of EDGE_100 (E100), a measure of a player’s productivity per possession and block opportunity, which I’ve set against the background of the relative efficiencies of the old and new teams.²

The principle at work here is akin to that of the “replacement player,” although instead of indexing players to a uniformly low benchmark, I’ve set team-specific replacement levels relatively high, on the theory that any one player has to be very good to move the team needle, and if an added player is not as good as the ones already on the field, he won’t play enough to make a discernible difference. Dividing the team-level EO100 (offensive efficiency) and EB100 (block efficiency) by eight seemed adequate for this purpose. If a player’s combined E100 value was greater than the replacement value, that player was presumed to be in heavy rotation and the positive difference was added to the team’s 2021 scoring differential. If the player’s E100 was less than the threshold, the value was not counted rather than subtracted. In this exercise, EB100 values (in the absence of any better measure) represent the prevention of scores to the opponent.

In reality, the holes created by departing players are often adequately filled through the development of players already on the team, or the addition of rookies. I do not attempt to measure this directly, but do modulate the effect of departures at the team level to account for it. Two other issues required some creative accounting:

1. Lapsed veterans. A number of teams are adding AUDL vets who did not play in 2021. Without yardage totals in their stat sheet, I cannot directly assign them an E100 value. However, I have estimated their E100 values using NetStats, as described here.
2. New teams. I have no basis for projecting the baseline competitiveness of the three new teams. Given that there are no player departures, only arrivals, it seemed necessary to set a low baseline, commensurate with a lower-tier team. At the player level, the baseline Replacement Values for expansion teams were set equal to those for the Aviators. In the model for projecting team wins, the baseline mean and standard deviation for goal and goals allowed for the expansion franchises were set to equal to Philadelphia’s 2021 season.

Beyond this, it’s naturally safe to assume, as I have, that there will be no changes in roles, team tactics, injuries, players missing games, and other variables that could change the fortunes of a team’s season.

As an example of how this shakes out, let’s consider the high-profile move of Antoine Davis from Atlanta to New York. Atlanta’s replacement level EO100/EB100 split was 8.2/1.8. In 2021, Davis had a split of 9.0/5.7. Although he was slightly above replacement level for offense, a function of having both elite scoring ability and the second most turns on the team, his block efficiency rate was Atlanta’s second highest.

A player’s effect is also scaled according to how much he actually played relative to other players on the team. Davis played a high percentage of Hustle points last year, indicating that he played a sizeable role in setting the Hustle’s team efficiency. In contrast, Dallas’s team efficiency was marginally affected by Chris Mazur’s high efficiency rates, simply because he played so few of the season’s points. His departure therefore is not expected to make as big a hit on Dallas’s overall efficiency and scoring differential.

All told, losing Davis is estimated to cost Atlanta 0.62 in goal difference per game. And the effect of Davis on the Empire? For a player’s future team, I set equal expectations: 20 points played for every game, in the same ratio of OL/DL as the prior season. New York’s EO/EB100 split was 8.9/1.7, meaning that the impact of Davis joining New York is slightly less than leaving Atlanta; his net benefit to NY is 0.44 goals per game. The logic is that a great player can only do so much for an already great team, and that he would have a relatively larger impact on a weaker team. Had Davis decided to turn south to Tampa Bay instead of north to New York, he would have been projected to improve the Cannons score differential by 1.28 goals.

So which player movements are expected to have the biggest effect? We’ll break it down into several categories:

• The Goodbye list—those who have created the biggest hole on the teams they’ve left;
• The Welcome list—players likely to have the biggest impact on a new team; and
• The “Fortune Shifters” list—those players with the largest combined Goodbye + Welcome effect.

The Goodbye List

Starting with the Goodbyes, Table 1 shows the 40 with the largest estimated effect on their former team’s projected average goal differential. While many teams are represented, one can feel the pain for teams like Los Angeles, Oakland, Toronto, Montreal,³  Dallas, and Boston.

Table 1. The effect of a leaving player on his former team’s projected average goal differentia. The Goal Differential Difference is a function of the difference between the player’s offensive and block efficiency (EO100, EB100) and the team’s replacement value (RV) as well as the proportion of team points played in 2021.

Name	Former Team	EO100	EB100	RV EO100	RV EB100	% Team Points Played	Goal Differential Difference
Cameron Harris	TOR	13.3	4.4	6.9	1.6	0.59	-1.8
Sean McDougall	LA	8.3	7.3	4.9	1.4	0.65	-1.6
Jordan Kerr	OAK	10.2	6.4	5.8	1.6	0.58	-1.5
Sacha Poitte-Sokolsky[1]	MTL	12.9	3.3	6.7	1.7	0.53	-1.5
Tanner Halkyard	BOS	11.9	8.4	7.7	1.6	0.45	-1.3
Thomas Edmonds	PIT	11.1	3.1	5.9	1.4	0.5	-1.2
Mike Mackenzie	TOR	11.3	2.7	6.9	1.6	0.59	-1
Garrett Martin	SEA	9.5	4.3	5.1	1.6	0.41	-0.8
Calvin Brown	LA	6.3	7.1	4.9	1.4	0.41	-0.8
Kyle Henke	DAL	8.6	6.6	6.6	1.6	0.42	-0.7
Jay Froude	DAL	9.9	7.5	6.6	1.6	0.28	-0.7
Andrew Roney	TB	7.2	6.1	4.6	1.3	0.35	-0.7
Nathan Hirst	TOR	9.9	1.6	6.9	1.6	0.56	-0.7
André Arsenault	MTL	5.1	7.6	6.7	1.7	0.35	-0.7
Brad Houser	SEA	9.9	3.4	5.1	1.6	0.29	-0.7
Drew Swanson	CHI	3.6	9.1	8.4	1.8	0.31	-0.6
Chris Wilen	MAD	10.8	4.2	7.5	1.6	0.35	-0.6
Danny Landesman	LA	9.4	1.9	4.9	1.4	0.33	-0.6
Nick Mahan	SEA	6.9	4.7	5.1	1.6	0.38	-0.6
Allan Laviolette	CAR	11.4	1.5	8.3	1.6	0.46	-0.5
Antoine Davis	ATL	9	4.9	8.2	1.8	0.56	-0.5
Sawyer Thompson	OAK	8	3.6	5.8	1.6	0.42	-0.5
Rami Paust	MAD	9.8	2.8	7.5	1.6	0.46	-0.5
Conner Henderson	IND	9.4	4.1	6.8	1.6	0.33	-0.5
Garrett Braun	DC	10.8	3.5	8.3	2.1	0.38	-0.5
Ryan Chard	TB	4.2	6	4.6	1.3	0.44	-0.5
Jeremy Keusch	AUS	6.4	5	5.9	1.9	0.4	-0.5
Andrew Carroll	TOR	10.4	0	6.9	1.6	0.39	-0.4
Jake Thorne	OAK	8.5	4.1	5.8	1.6	0.28	-0.4
Joe Merrill	DC	11.3	3.7	8.3	2.1	0.29	-0.4
Henry Babcock	BOS	9.9	1.6	7.7	1.6	0.46	-0.4
Joshua Venegas	IND	2.5	6.5	6.8	1.6	0.33	-0.4
Lior Givol	SD	7.8	2.5	6.1	1.8	0.49	-0.4
Kevin Tien	OAK	4.6	5.5	5.8	1.6	0.31	-0.4
Charles Weinberg	MIN	8.7	0.8	7.3	1.9	0.38	-0.4
Brett Matzuka	MIN	7.3	4.7	7.3	1.9	0.35	-0.4
Mike Lee	OTT	10.4	3.7	6.6	1.6	0.28	-0.3
Abe Coffin	DAL	11.3	0	6.6	1.6	0.36	-0.3
Jonathan Nethercutt	SD	8.9	0	6.1	1.8	0.12	-0.3
Chris Mazur	DAL	5.8	2.2	6.6	1.6	0.17	-0.3

Focusing at the top, Harris, McDougall, Kerr, Poitte-Sokolsky, and Halkyard were not just key players; they each had the highest E100 scores on their respective teams and played a ton. They will be leaving crater-sized holes, and the excitement of any new season is to see how well the team is able to shovel them full. In all, 81 players were estimated to cause a sting by not suiting up for their 2021 team in 2022.

The Welcome List

Without the ability to evaluate “rookies” who had not previously played in the AUDL, 54 player movements were estimated to make a positive impact on the new 2022 team (Table 2).⁴ Given the premise of the exercise, most of those on this list are going to teams that did not make the playoff in 2021. Those going to expansion teams tend to be concentrated in the top half of the list, a function of setting a low initial replacement value for those teams. Players going to 2021 playoff teams tend to be concentrated at the bottom of the list. Again, there’s only so much one can do in joining a good team that is already pushing the ceiling of the sport’s efficiency range, except perhaps fill the hole of a departed, which is why New Team replacement values are adjusted to account for the loss of departed players and are typically smaller than the RVs in Table 1.

Table 2. Projected benefit of players to their new team, measured in average game goal differential for the team. Projections are based on each player playing 20 points per game in the same proportion of OL/DL as in their most recent AUDL season. New Team RV EO100 and RV EB100 represent the adjusted team efficiencies after accounting for departing players.

Name	New Team	AUDL in 2021?	EO100	EB100	New Team RV EO100	New Team RV EB100	Goal Differential Difference
Jonathan Nethercutt	COL	Yes	13.6	0	4.8	1.2	1.64
Jay Froude	COL	Yes	9.9	7.5	4.8	1.2	1.62
Jordan Kerr	SL	Yes	10.2	6.4	4.8	1.2	1.56
Joe Merrill	SL	Yes	11.3	3.7	4.8	1.2	1.49
Drew Swanson	SEA	Yes	3.6	9.1	5	1.4	1.39
Quinn Snyder	MIN	No	14.7	2	7.3	1.8	1.32
Raphy Hayes	POR	No	11	2	4.8	1.2	1.2
Ian Engler	PIT	Yes	11.8	1.6	5.8	1.4	1.16
Matt Rehder	SEA	No	9.3	5	5	1.4	1.16
Chris Mazur	SD	Yes	11.3	2.2	6	1.7	1.08
Noah Robinson	PIT	No	10.9	2.2	5.8	1.4	1.14
Aaron Weaver	LA	No	10.9	2.1	4.8	1.2	1.07
Garrett Martin	SL	Yes	9.5	4.3	4.8	1.2	1.05
Leandro Marx	POR	No	10	2	4.8	1.2	1.02
Sean McDougall	SD	Yes	8.3	7.3	6	1.7	1.02
Kyle Henke	AUS	Yes	8.6	6.6	5.8	1.8	1.01
Chris Walsh	DET	Yes	8.3	2.8	4	1.2	0.99
Danny Landesman	COL	Yes	9.4	1.9	4.8	1.2	0.96
Chris Larberg	POR	Yes	8	4.4	4.8	1.2	0.93
Jack Hatchett	POR	No	7	4	4.8	1.2	0.75
Dalton Smith	CHI	Yes	11.4	3.7	8.4	1.6	0.73
Dave Wiseman	COL	No	7.5	2.8	4.8	1.2	0.68
Brett Matzuka	IND	Yes	7.3	4.7	6.7	1.5	0.66
Jacob Miller	SL	Yes	5.8	4.1	4.8	1.2	0.66
Eli Friedman	POR	No	8.5	1.3	4.8	1.2	0.63
Tyler Bacon	SD	No	9.5	1.6	6	1.7	0.6
Ben Green	SL	Yes	2.4	5.4	4.8	1.2	0.6
Mike Pannone	PIT	No	10.7	2	5.8	1.4	0.6
Rami Paust	MIN	Yes	9.8	2.8	7.3	1.8	0.59
Tom Doi	POR	No	8.1	1.5	4.8	1.2	0.57
Tyler Monroe	DC	No	12.1	0.6	8.3	1.9	0.57
Jeremy Keusch	IND	Yes	6.4	5	6.7	1.5	0.54
Johnny Bansfield	DET	No	6.4	2	4	1.2	0.54
Connor DeLuna	DAL	Yes	8.9	2.8	6.4	1.4	0.52
Nathan Champoux	DET	No	3.3	4.5	4	1.2	0.48
Matthew Armour	AUS	Yes	8	2.4	5.8	1.8	0.38
Antoine Davis	NY	Yes	9	4.9	8.9	1.7	0.36
Mark Whitton	DET	No	5.1	2.4	4	1.2	0.34
Tyler DeGirolamo	PHI	No	6.6	3.5	6.3	1.3	0.33
Keegan North	IND	Yes	8	2.2	6.7	1.5	0.32
Abe Coffin	MIN	Yes	9.8	0	7.3	1.8	0.31
Mark Lin	NY	Yes	9.2	3.4	8.9	1.7	0.23
Charles Weinberg	NY	Yes	10.4	0.8	8.9	1.7	0.21
Jake Steslicki	DET	No	4.5	1.5	4	1.2	0.13
Ryan Poloz	TOR	Yes	7.8	0	6.4	1.5	0.13
Charlie McCutcheon	CAR	Yes	5.7	3.6	8.3	1.6	0.11
Christian Boxley	DC	No	10.1	0.5	8.3	1.9	0.11
Khalif El-Salaam	ATL	Yes	3.7	4.6	8.2	1.7	0.11
Delrico Johnson	DC	No	8.2	2.7	8.3	1.9	0.1
Scott Radlauer	POR	Yes	3.7	2.2	4.8	1.2	0.09
Scott Hearne	CHI	Yes	9.3	1.4	8.4	1.6	0.04
Hayden Stone	MTL	Yes	6.1	2.5	6.5	1.6	0.03

The Fortune Shifters

Consider an Oakland vs. Salt Lake game in this coming year. It’s projected that Salt Lake will improve its scoring margin by 3.1 goals just from luring Jordan Kerr away from Oakland (Table 3). This is rare leverage; only six players generate better than a two-goal swing, including Drew Swanson (Chicago to Seattle), who does so based on his prodigious block rate. Table 3 shows the top 10 in combined effect of leaving one team for another, although Cam Harris makes the list just on the impact of his not playing this year for Toronto.

Table 3. Players with the largest combined effect (measured in projected goal differential) of leaving one team for another.

Name	Former Team	NewTeam	Combined effect on old and new teams
Jordan Kerr	OAK	SL	3.09
Sean McDougall	LA	SD	2.64
Jay Froude	DAL	COL	2.34
Drew Swanson	CHI	SEA	2.02
Jonathan Nethercutt	SD	COL	1.97
Joe Merrill	DC	SL	1.92
Garrett Martin	SEA	SL	1.84
Cameron Harris	TOR		1.77
Kyle Henke	DAL	AUS	1.76
Danny Landesman	LA	COL	1.57

Win Predictions

What is the net effect of all the movement? To get an estimate, I summed up each team’s credit and debits in terms of offensive and defensive point differences.⁵ For each, I added them to their 2021 per-game average, split between home and away games. Using the new expected goals and goals allowed totals, along with the standard deviation of their 2021 averages, I simulated the 2022 schedule 25 times. The average of those runs result in the team’s expected win total for the season. Again, the team projections are based on the assumption that all players will play all games.

As an example, let’s look at whether it’s possible that Detroit did enough in the offseason to likely end its losing streak. In 2021, its average home score was 14.2 – 26.5 and its average away score was 16.7 – 26.8. The glimmer of hope is that there was very high variation (StDev=5.47) in what their opponents scored against them at home (remember 34-10?). This means if they can narrow the expected point differential a bit though player acquisition, enough stars may align for a win or two.

The main additions of Chris Walsh from Indy and Johnny Bansfield, Nathan Champoux, Jake Steslicki, and Mark Whitton from their past, coupled with no significant personnel losses, gives more hope. Walsh in particular had a very strong 2021 with an E100 rate of 11.7. This would make him a productive addition to any team, but going to the Mechanix is reason for a parade, given that their replacement value of 4.0 is half that of the Empire’s 8.9. Combined, these five additions to Detroit are expected to add 1.5 goals to the Mechanics score while reducing opponents scores by 1.56. This 3.06 swing is the largest in the league, except for the expansion Shred and Summit and reduces their average margin of loss from 11.1 goals to 8. That still seems like a lot, but recall that they came close last year without the new pieces, and the high variance/stars align factor in the model could result in predicted wins. Let’s hit go and see what happens.

Detroit Wins!

The streak ends. Detroit scored at least one win in 19 of the 25 simulations. (Pittsburgh was the most frequent, but not only, victim). As shown in Table 4, the simulations predict anywhere from 0-4 wins for the Mechanics, which is actually a better forecast than that for the Cannons, who will be competing with no big-name acquisitions and the loss of Andrew Roney and Bobby Ley in a division full of solid teams.

Table 4. Model output for the effect of player movement on projected team wins. Projected wins are the average of 25 simulations of the season’s schedule; “low range” and “high range” show the highest and lowest win totals of the simulations.

Team	Div	Projected Wins	Low Range	High Range	2021 Wins	Net Gain
CHI	Central	9	7	12	10	-1
MIN	Central	9	7	11	9	0
MAD	Central	6	3	9	8	-2
IND	Central	6	3	9	3	3
PIT	Central	5	2	8	1	4
DET	Central	1	0	4	0	1
NY	East	9	6	12	10	-1
DC	East	10	7	12	10	0
BOS	East	5	2	8	5	0
MTL	East	5	3	7	9	-4
OTT	East	5	3	6	4.5	0.5
TOR	East	4	1	7	4.5	-0.5
PHI	East	4	2	7	3	1
CAR	South	10	7	12	8	2
ATL	South	8	5	11	9	-1
AUS	South	6	4	9	6	0
DAL	South	5	3	8	8	-3
TB	South	1	0	2	2	-1
SD	West	10	8	12	10	0
SL	West	7	5	10	0	7
COL	West	7	5	10	0	7
POR	West	7	4	9	0	7
SEA	West	5	2	8	3	2
LA	West	3	0	7	5	-2
OAK	West	3	0	7	4	-1

At the other end of the spectrum, Carolina, DC, and San Diego are projected to have the best regular seasons at 10-2, and add to them Chicago and New York as teams that ran the table at least once in the season simulations. Every 2021 playoff team appears to be in good position to return to the playoffs, except one: Dallas. The combination of losing so many top players and joining a highly competitive division results in a projected three-win drop compared to 2021.

But Dallas doesn’t suffer the biggest fall. That would be Montreal, whose main sin was to rejoin the East and play Boston, New York, and DC for half their games. Treating their 6-2 Canada Cup record as a 9-3 full season record, the Royal are projected to win only five games in 2022. Madison and LA are projected to each drop two games more than they did in 2021.

If we exclude the expansion teams, the hot-stove league award for gaining the most projected wins goes to Pittsburgh, where the incoming combo of Ian Engler, Mike Pannone, and Noah Robinson⁶ (if they actually play) more than makes up for the departure of Thomas Edmonds. The Thunderbirds are projected to win four additional games this year. Counterintuitively, Indy is the next biggest gainer, even though they are battling for division wins with Pittsburgh.

Carolina’s four losses in 2021 (all by one goal) were more than would be expected from their overall point differential, and the cosmos evens things out this year with an expected additional two wins, despite the season-ending loss of Allan Laviolette to injury. Seattle is the only other team expected to improve by at least two wins, although that is heavily Rehder-dependent. A more sophisticated version of this exercise might value equivalent gains by a group of players more highly than by an individual, on the basis that the gains might be more stable across all games of the season.

In the battle of the expansion teams, the model proves to be a complete cop-out, projecting each team with a 7-5 record. On the other hand, it’s hard to look at the comparable lists of pick-ups (often at the expense of division rivals) and easily pick out the most impressive grouping. Given that each franchise has a well-developed local talent pool to build on, the projections seem reasonable, even if boring.

One of the premises of this exercise is that it considers a player’s efficiency to travel intact to another team. But a player’s efficiency is also a reflection of the team around him: handlers benefit from better cutters and vice-versa, the opportunities to be productive are greater if teammates are not turning it over excessively, and so on. One of the opportunities of “The Yardage Era, Season Two” now upon us is that player movement may be a particularly good way to help estimate the magnitude of the team effect. This is something I’ll probably come back to in the future, even at the cost of reminding us that I made some season predictions I may wish were long forgotten.

---

1. I’m grateful for Ruffner & Cohen’s two “Swing Pass” podcasts this week for the thorough review. Thanks also to Alex Rubin. ↩

2. In order to easily take advantage of the AUDL’s new official possession data, I have simplified the EDGE-O calculation by assigning a league-wide value of 0.51 goal equivalents for every turnover rather than using game-specific values; the difference in outcomes is minimal. I have also slightly shifted some of the weighting from the “scoring premium” to yardage. This is because the scoring premium is there to account for a theoretical “nose for the goal” ability, in which a player scores much more frequently (or infrequently) relative to their yardage than other players. A full year’s worth of data showed that that variation declined with increasing yardage, and that therefore we should reduce the weighting of the premium. This had the side benefit of simplifying the equation to EDGE-O (Goal Equivalents) = (0.007*total yards) + (0.2*scores) – (0.51*turns). We can call it the Bond Rating. EDGE-O100 is EDGE-O per 100 possessions. EDGE-B = 0.51*blocks. ↩

3. Although rostered, the expectation is that Sascha Poitte-Sokolsky will miss significant amount of the season. For most players, this was not considered, but given his outsized influence on the Royal, it seemed prudent to reflect this in his case. ↩

4. Given that Portland’s rookies included two World Games alternates and a U24 national team member, it seemed excessively distortive to not include any values for Raphy Hayes, Jack Hatchett, and Leandro Marx in the database, so all three were assigned proxy E100 values in the high, but not the highest, range. ↩

5. At the team level, because I can only account for incoming players who previously played in the AUDL, the cost of all departures from a team is divided by three in order to balance the overall league effect of comings and goings, and to account for the fact that players already on the team often adequately fill the hole of a departed player with more development and playing time. ↩

6. Several players had relatively small samples sizes (e.g. Nethercutt) or data from no more recent than 4 years ago, but Robinson was both; his huge offensive efficiency rating has been reduced by 20 percent to avoid overweighting. ↩