NHL
We have a trio of “new” NHL stats releasing at The Athletic today — just in time for the playoffs: Offensive Rating, Defensive Rating and Net Rating. One measures offense, one measures defense and one just adds them together for a total number.
“New” is in quotation marks because there isn’t really anything all that new about any of them. We’re not reinventing the wheel here; just repackaging it.
I created an NHL version of Game Score in 2016, nearly seven years ago, and quickly turned it into a model that could measure a player’s value in wins called GSVA or Game Score Value Added. The idea went that if Game Score measures how good a player’s game is and a good player plays a lot of good games then Game Score should be a good measure of how good that player is. Sound good? Good.
Advertisement
In 2019 after a defensive Cup Final between the Blues and Bruins, I made necessary changes to GSVA to better account for skater defense and to move from Corsi to expected goals.
Those changes in 2019 improved the model immensely, but they arguably didn’t go far enough when it comes to valuing skater defense. And that’s why we’re here today with Net Rating which is essentially a rebranding of GSVA — one that simply separates the model into two distinct parts: offense and defense. Any metric revolving around offense goes into Offensive Rating and anything defensive goes into Defensive Rating.
Offensive Rating: A weighted combination of goals, primary assists, secondary assists, individual expected goals, faceoffs, penalties drawn, expected goals for impact at five-on-five, goals for impact at five-on-five, power-play goal impact and usage.
Defensive Rating: A weighted combination of blocked shots, faceoffs, penalties taken, expected goals against impact at five-on-five, goals against impact at five-on-five, penalty kill impact and usage.
Everything above is weighted almost the same as before. Individual metrics carry the same weight and the ratio between individual and on-ice impact also stays the same. But the ratio of expected to actual goals has changed as actual goals have become more important and predictive as goals per game has increased.
The separation of offensive and defensive metrics in GSVA is a long overdue one and that increase in goals per game since 2017-18 is part of the reason why it’s so vital. When goals become less scarce the ability to diminish them becomes more important.
Separating the two factions allowed me to compare how Offensive Rating predicts a team’s ability to score goals above league average and compare how Defensive Rating predicts a team’s ability to suppress goals above league average. Predictably, it is much easier to project a team’s offense compared to a team’s defense (which is why it’s generally weighted higher), but slightly less so than GSVA suggested — even after changes were made to better account for defense. As it turns out, GSVA was in fact underselling how important a skater’s defense was to a team’s underlying ability. Not by a lot, but by enough. Defensive Rating fixes that.
Advertisement
By extension of that, it also oversold teams comparatively speaking by overrating teams that play at a higher pace. In general, one standings win in the NHL is equal to six goals, but that number is not the same for every team. A good defensive team doesn’t need as high of a goal differential to win while a poor defensive team will need more goals to keep up.
Here’s an example. Team A wins 4-3 on average which would give them a plus-82 goal differential over a full season. Using Pythagorean win probability they would be expected to win 65.7 percent of games. Team B wins 2.7-2 on average which would give them a plus-57 goal differential. Using Pythagorean win probability they would also be expected to win 65.7 percent of games. GSVA, because it didn’t separate out offense and defense, would expect Team A to be far superior, but Net Rating would expect the two to be equal. When Islanders fans have asked in years past why The Model hates their team, this is probably why. And they had a good point.
The real-world differences won’t be nearly as drastic as the example above because the differences for teams aren’t that large. But it will help around the margins. GSVA thinks the Penguins are better than the Kings and their expected goal differential is indeed higher (mostly due to Pittsburgh’s strong priors and poor puck luck this season). Net Rating instead leans toward the Kings, who are a much stronger defensive team.
The other thing this fixes is win probabilities at the extremes. Using a model based on GSVA was too certain about the strongest teams beating the weakest teams, something that didn’t work out so well this season with a lot of unexpected underdog wins. (A nice reminder that having skin in the game is important for making improvements.) Certainty is good in recent seasons with such a massive league-wide drop in parity, but it has to be the right amount of it. Too little and too much are both wrong.
There’s one other important change to be made and it’s the reason for the name change, rather than going by something like oGSVA and dGSVA: units of measurement.
When first creating Game Score, one of the most important factors was using a familiar scale to make it easier to understand. I chose to put it on the same scale as points per game for that reason.
Advertisement
GSVA is essentially a WAR metric meaning it measures player value in wins above replacement. A lot of hockey fans have come to be familiar with the terminology over the years and what is and isn’t a good WAR — but the idea of “replacement level” can still be a bit cumbersome at times. To players who may pay attention to these stats, being at something called “replacement level” or even below it can be taken as a slight which we should perhaps be more sensitive to. Using “wins” as the unit can also be a bit abstract.
So that’s the other change that makes this “new” — a different baseline and unit that are hopefully more intuitive to more casual fans.
Offensive and Defensive Rating are both centered around an average skater at their position where positive is always good. The idea of “average” is something that’s easier to wrap your head around and it creates an immediately intuitive baseline for the middle. An average player by GSVA isn’t quite so easy to figure out without simply knowing. Shifting the baseline to start from the middle out helps. An average player by Net Rating is right at zero.
It’s also worth noting that a player’s rating can be different depending on whether the “average skater” is based on per-game stats or per-60 stats. The latter would better reward efficiency, while the former might better reflect a player’s value in his role. I’m not sure any current model can properly scale a player’s per-60 stats up and down the lineup — sometimes the efficiency doesn’t translate — and for that reason, Net Rating is based on per-game stats. It’s a topic worth revisiting once I do a bit more work on usage effects, but for now, this works as a failsafe against players that get crushed by big minutes and players that crush soft minutes.
The unit of measurement being used for Offensive and Defensive Rating is goals, since what the model is and always has tried to predict is goal differential anyway. Not wins. That’s a small difference, but a big one in the grand scheme when thinking about the age-old question “Why does your model hate my team?” A lot of the time the reason is that team’s record doesn’t match their goal differential which is oftentimes a sign of good or bad luck. Using wins as a unit of measurement creates an expectation of wins being the target prediction. Technically it is … but that’s by way of predicting a team’s goal differential first and foremost.
And at its core, that’s what Net Rating is: If you take an average team with average skaters and swap one of them out for Connor McDavid, what would the model project their goal differential to be? The answer according to Net Rating is plus-28; plus-27 on offense and plus-1 on defense. With replacement level at roughly minus-7 goals that would put McDavid’s win value at 5.8 wins, a shade below how valuable GSVA figured he was.
Here’s what a player card for that would look like. The circles denote McDavid’s league-wide percentile with each slice of color representing 10 percent. The thickness of each circle denotes how important each metric is toward contributing to McDavid’s Offensive and Defensive Rating. That should better illustrate why the model rates a player the way it does.
That’s all well and good, but introducing this all just before the playoffs start will likely leave a lot of people in the dark. Forgive us for that, but a little bit of work now will be worth it in the long run with the ability to better separate a player’s contributions to the two sides of the game.
Most people who have seen GSVA used a lot over the years have a general idea of the degrees of goodness when it comes to player wins. Zero wins is replacement level, one win is average, two wins is top line, three wins is elite, four wins is superstar, five wins and above is MVP level. Something like that. It’s not the same for what makes a good or bad Offensive or Defensive Rating. Obviously zero is average, above average is good and below average is bad — but the degrees won’t be as immediately clear off the bat.
So here’s a crash course with what constitutes a good or bad Offensive, Defensive and Net Rating for each position using examples from around the league. This should help assess what it means if someone is plus-5 or minus-1 or anywhere else.
At the end of the day, because goalies exist, offensive value still trumps all. But when it comes to understanding why a model hates your favorite 80-point scorer or loves a random 20-point defender, the other side of the ledger will explain why.
GSVA and Net Rating aren’t drastically different in how they view players. There are some minor differences here and there as strong defensive players receive a bit more credit for their work without the puck — a more accurate accounting of that based on predictive value. But it’s not changing the fact that a good player by GSVA will still be a good player by Net Rating. They just might rank 20th or 40th instead of 30th. Or anywhere in between.
Advertisement
The changes made today are meant more to create an easier sense of understanding regarding player value when it comes to this model and where that value comes from: with or without the puck. That, and using a more intuitive baseline and unit of measurement that should be more immediate when it comes to assessing a player’s worth with one simple mission statement: What’s an average team’s goal differential with Player X on it?
Offensive Rating, Defensive Rating and Net Rating are not a finished product just yet. There are more improvements to be made with how offense and defense interact with each other when it comes to quality of teammates and competition. GSVA and Net Rating both account for that to an extent, but there’s more work to be done there. This change creates a better framework to explore that over the summer — how usage affects player efficiency and how that efficiency translates up and down the lineup.
Even if there’s still more work to be done, Net Rating is still an improvement over GSVA today just by way of its better accounting for defensive value and its arguably more intuitive baseline and units. Separating things into offense and defense is obviously also a crucial element.
Neither stat is perfect. That’s impossible and not the goal anyway. Hockey is an unpredictable sport at its core and this model (and others) are simply trying to make sense of that the best it can.
“All models are wrong, some are useful.” The goal is to create a model that’s more useful than the last. Offensive Rating, Defensive Rating and Net Rating should accomplish that.
(Photo: Nick Turchiaro / USA Today)
Dom Luszczyszyn is a national NHL writer for The Athletic who writes primarily about hockey analytics and new ways of looking at the game. Previously, he’s worked at The Hockey News, The Nation Network and Hockey Graphs. Follow Dom on Twitter @domluszczyszyn
