Huck It Deep: NexGen’s Playing Style Showed Reliance On Defense

November 16, 2012 by Sean Childers in Analysis with 7 comments

This is the third and final part in our defensive series on the statistics of NexGen. Photo courtesy of Ultiphotos’ Alex Fraser. Check out more photos of the NexGen tour.

Perhaps the strongest juxtaposition between offense and defense would be finding that neither NexGen’s offensive completion rate nor their average yardage per completion predicted their point margin, but that both the opposition’s completion percentage and average yardage per completion were powerful predictors.

As you already know, three-quarters of that proposition is supported by our data — but only three quarters. We crunched the numbers on the average yardage gained by NexGen’s opponents and found no statistically significant relationship; Graph 5 below shows the model. In other words, the amount of yards that NexGen’s opponents gained per completion did not predict how well NexGen performed.

It’s worth noting that our methodology and computer program inherently makes estimating the amount of yards generated per throw a much noisier proposition than the simple task of calculating a completion percentage. In particular, it’s hard to know how much of the greater variation in the NexGen yardage compared to the opposition yardage (Graph 2 compared to Graph 5) is due to different playing styles and how much is due to measurement error — though we attempt to dissect that a bit more below. The most intelligent takeaway point from these graphs may very well be that we just need a lot more data.

Hucks May Also Show NexGen’s Reliance On Their Defense

There is already some conventional wisdom in the ultimate community that teams which rely on their defense might be better off playing a huck-deep strategy on offense. Athletic players who can generate Ds tend to have skill sets that would also support offensive playmaking on deep hucks. And if your team isn’t confident that it can string a lot of short passes together, then you might be better off taking deep shots.

Elaborating on this hunch, we might expect two equal teams employing similar strategies to complete about equal number of passes in the course of a game. Of course, better teams complete more passes in general. But so do teams that rely on a lot of short passes. So if we have two relatively equal teams employing different strategies, we’d expect the team going for short passes to complete more passes while the team going for the deep ball might complete fewer (but scores on a higher percentage of the passes they do complete).

The NexGen data here is very interesting: In the 12 games we have data for, NexGen completed 43 fewer passes than their opponents. This is especially surprising because, over the course of those games, NexGen was the better team; our baseline expectation would have been that they should complete more passes. The gap between the number of passes attempted is even larger.¹

The allstars completed 23% more deep throws (defined as completions for more than 30 yards) than their opponents. On average, NexGen also generated more yards per completion than their opponents. Those three facts combined — that NexGen completed and attempted fewer passes than their opponents, that they completed more hucks, and that they averaged more yards per completion than their opponents — suggests that there could have been a substantial playing style difference between NexGen and their opponents. Furthermore, NexGen’s different playing style seems consistent with a theoretical approach that emphasized a lot of confidence in their ability to play defense rather than a surgical or perfectly functional offensive game.

It is more difficult to find evidence that the other teams necessarily played a more successful small-ball strategy than NexGen. NexGen’s completion percentage was on average 1% higher than their opponents — we might expect it to be lower if the opposition was playing with more methodological build up play. But we would also expect NexGen, as the superior team, to have a higher completion percentage. It’s hard to know whether the 1% difference undercuts the theory that NexGen played a more risky deep game or if they were just the superior team in general.

Table 1: Differences in Playing Styles Between NexGen and Opposition²

Limitations To The Theory

I’ll conclude with just a few of the more obvious limitations to the idea that it was NexGen’s dominant defense, as much as their offense, that drove their success. First, separating offensive performance from defensive performance in ultimate represents a bit of a fool’s errand: they’re two sides of the same coin, and anything that looks like good defense from NexGen could have just been bad offense from the opposition. Recall the example of the Machine game: it was NexGen’s worst offensive performance but their largest blowout win. It may be most apt to say that game was just sloppy, rather than pointing to it as a part of a larger trend showing the importance of defense.

Second, there are a lot of methodological limitations in our beta-deployment of the statistical application. We only have a sample size of about 12 games and, even in those games, we don’t think our data perfectly represents what happened in the game. Developing the application and recording the games was a learning and buggy process for everyone involved, one that will undoubtedly (and excitingly) get better in the future.

Third, we haven’t controlled for opposition team strength in any of our analysis, an essential and foundational piece of the sports analytics puzzle.³ We would need to gather a lot more data about a lot more teams before solving that issue.

Finally, its entirely unknown whether the relationships derived here are NexGen-specific or apply to ultimate teams as a whole. In all likelihood, different ultimate teams will rely on different team-wide metrics in order to obtain wins. But, at least with this starting point, it appears that defense might be an underappreciated and underdeveloped part of the story — at least when one is considering teams filled with young, athletic players like NexGen.

Regression Output

Graph 1 / Percentage Points Scored by NexGen x NexGen Completion Percentage:

Call: lm(formula = Win.Share…100 ~ Completion.Perc…100)

Coefficients: Estimate || Std. Error || t value || Pr(>|t|)
(Intercept) 11.4869 || 105.1450 || 0.109 || 0.915
Completion.Perc…100 0.4729 || 1.1377 || 0.416 || 0.686
Residual standard error: 10.06 on 10 degrees of freedom
Multiple R-squared: 0.01699, Adjusted R-squared: -0.08131
F-statistic: 0.1728 on 1 and 10 DF,    p-value: 0.6864

Graph 2 / Percent Points Scored by NexGen x NexGen Yards Per Completion:

Call: lm(formula = Percent.Points.Home.Team.Won ~ Home.Average.Yard.Per.Completion)

Coefficients: Estimate || Std. Error || t value || Pr(>|t|)
(Intercept) 0.558712 || 0.168448 || 3.317 || 0.00779 **
Home.Average.Yard.Per.Completion -0.000877 || 0.021051 || -0.042 || 0.96759
Residual standard error: 0.1015 on 10 degrees of freedom
Multiple R-squared: 0.0001735, Adjusted R-squared: -0.09981
F-statistic: 0.001736 on 1 and 10 DF,   p-value: 0.9676

Graph 3 / Percent Points Scored by NexGen x Number of Turnovers Generated:

Call: lm(formula = Percent.Points.Won…100 ~ Home.Blocks.Intercepts.and.Throwaways.Generated)

Coefficients: Estimate || Std. Error || t value || Pr(>|t|)
(Intercept) 41.5087 || 6.5909 || 6.298 || 8.93e-05 ***
Blocks.Intercepts.and.Throwaways.Generated 1.0127 || 0.4548 || 2.227 || 0.0501 .
Residual standard error: 8.3 on 10 degrees of freedom
Multiple R-squared: 0.3315, Adjusted R-squared: 0.2646
F-statistic: 4.958 on 1 and 10 DF, p-value: 0.05013

Author note: This regression equation technically falls just above the .05 threshold for statistical significance that scholars in some of the social sciences consider a minimum. Basically, given the limited number of data points that we have, we think it’s wiser to adopt a flexible (rather than extremely rigid) interpretation. We present the numbers here for the readers to make the final determination, but also note that no other pscores were remotely close or on the border (no other test in this article had a score beneath the more liberal .1 threshold but also above the .05 threshold).

Graph 4 / Percent Points Scored by NexGen x Number of NexGen Blocks + Intercepts:

Call: lm(formula = Percent.Points.Won…100 ~ Home.Blocks.and.Intercepts)

Coefficients: Estimate || Std. Error || t value || Pr(>|t|)
(Intercept) 43.5787 || 4.6849 || 9.302 || 3.07e-06 ***
Blocks.and.Intercepts 1.4970 || 0.5342 || 2.802 || 0.0187 *
—
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 7.597 on 10 degrees of freedom
Multiple R-squared: 0.4399, Adjusted R-squared: 0.3839
F-statistic: 7.853 on 1 and 10 DF,   p-value: 0.01872

Graph 5: Percent Point Scored by NexGen x Opposition Yards per Completion

Call: lm(formula = Percent.Points.Won…100 ~ Away.Average.Yards.Per.Completion)

Coefficients: Estimate || Std. Error || t value || Pr(>|t|)
(Intercept) 73.468 || 18.589 || 3.952 || 0.00272 **
Average.Yards.Per.Completion -2.510 || 2.522 || -0.995 || 0.34317
—
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 9.683 on 10 degrees of freedom
Multiple R-squared: 0.0901, Adjusted R-squared: -0.0008937
F-statistic: 0.9902 on 1 and 10 DF, p-value: 0.3432

1. One would also expect a team starting on defense more often (which was NexGen) to complete/attempt fewer passes based on the mere fact that they are pulling the disc more often than their opponents.  While this is an important fact to keep in mind when thinking about the completions and attempts numbers, it also supports the story that the team relied on their defense. ↩

2. NexGen’s average is the average completion percentage across games coded, rather than an aggregate completion percentage.  This was the methodology used in all of the averages in the table. ↩

3. Both the college basketball rankings by KenPom and the NBA power rankings by John Hollinger incorporate strength of schedule when producing statistical rankings of teams. ↩