November 7, 2013 by Charlie Eisenhood in Livewire, News with 5 comments
After selecting our All-Club 1st and 2nd teams over the past week, we have had a great deal of feedback. Everyone has an idea of who deserves to be recognized, and we heard the gamut of reactions from positive to sharply negative.
The one constant has been a focus on the statistics. That’s due in part, I think, to the emphasis we placed on them in our writeups of the players, which gave an appearance of the stats determining almost solely the outcome of the lists. That really couldn’t be further from the truth.
Keith Raynor and I, who made the selection for the Women’s and Men’s Divisions, respectively, absolutely used the statistics we had available from the US Open and Club Championships to help us make decisions about players. But the fundamental core of our choices were made based on our understanding of the performance of the players during the course of the regular season and playoffs.
Some players selected did not have their games tracked by our statistics team. Others didn’t shine in the statistical framework. But every player was a real difference maker for their team in one way or another.
On the Men’s side, I got a lot of pushback for not selecting certain players like Ironside’s George Stubbs, Sockeye’s Matt Rehder, and Chain Lightning’s Dylan Tunnell. Each of those players has serious cachet in the ultimate community, and each is undoubtedly an outstanding player. Stubbs and Tunnell represented the US on the World Games team this summer.
But, frankly, neither had a stellar club season. Stubbs was injured for much of the year, and was clearly hampered by that. He was frequently turnover prone (both to the eye and to the stats). Tunnell was perhaps the best player on the Chain O line this year, but both he and his teammates struggled during the regular season.
Both players had strong performances at Nationals. Stubbs, as many at home saw, was tremendous in the semifinal against Sockeye. Perhaps I should have weighted that ‘big game’ more than the bulk of the year. But that’s not how we put together our lists. If he’s healthy, I would almost certainly select Stubbs to play on my top line. We simply weren’t selecting based on talent — we were selecting holistically based on the whole season.
Yes, the statistics did sometimes sway my decision making. Rehder, who played very well in two of the Sockeye games I saw at Nationals, turned the disc over too often and didn’t have a great game in the Finals. The stats didn’t give him a boost, either. On the other hand, another Sockeye player, Danny Karlinsky, was a rock for the team. Not flashy, not an obvious pick, but a turnover-free performance at Nationals from a heavily involved handler on a finals-bound team speaks for itself.
Keith Raynor, who chose the Women’s lines, has taken even more heat. Selecting Opi Payne, Scandal’s star, on the 2nd line churned some discussion, but his first line, which did not include a player from San Francisco Fury, really ruffled feathers.
Keith has been making a good case on Twitter for his selections, and having watched many women’s games myself while calling games for our livestreams, it is obvious to me why players like Lien Hoffmann and Kami Groom, both young stars on Nemesis, made the team. They were superstars for Chicago from start to finish this season — not only racking up stats but making the big plays and getting the big minutes.
Some make a very fair case that Keith punished the depth of a talented team like Fury by selecting only one player (Lakshmi Narayan) from their obviously talented lineup. That’s fair, but, again, these lists were based on performance, not talent. Whether or not Hoffmann is a “better” player in a pure talent world than, say, Fury’s Claire Desmond did not really come into play. An all-talent list that ignores performance during the year simply isn’t what “all-pro” teams are about.
While we are the first to admit that our statistics (which are unquestionably in their infancy) are not even close to perfect, they are perhaps the most advanced analytics that have ever been devised for ultimate. We think that’s important, and we want to highlight them. Some metrics — like Expected Contribution — give us an “all-around” idea of how important a player is to their team’s chances of winning the point.
The fact is that advanced metrics should sometimes be surprising! They are useless otherwise. If all the stats do is confirm that the players you think are great are, in fact, great, there’s not a lot of value to glean from them.
So when we see subpar statistical performances from a player like George Stubbs, perhaps instead of decrying the stats as wrong, we should consider that those players simply didn’t have as strong a year as they might have been expected to on talent alone.
Of course, numbers can’t capture everything. Particularly, our stats do not adequately track break throwing ability, good movement away from the disc (i.e. clearing space), or intangibles like “clutchness” (if that exists) or leadership.
But we watched these players a lot this season and saw those other things in action. And our selections were based on our knowledge of the players in combination with the cold numbers. That made for a more fun process, and one that, I believe, more accurately reflects who were the top performers of the club season.
