The Puck Stops Here

Yesterday, I posted the top 20 on/off ice adjusted +/- ratings. This is an attempt to look at the sabermetrics and hockey problem of rating an individual player's defensive ability. I discuss the method here and credit Gabriel Desjardins of behind the net for its development.

I cannot argue that this method directly determines the best defensive player in the NHL. It will select players who played well in their roles, where their team played better with them on the ice than with them off the ice. These players will be offensive and defensive stars (most are solid two-way players).

In the comments Mathman criticizes it, claiming it tends to select good players on bad teams. I do not buy this criticism completely. In order to normalize +/- ratings to compare between teams, some assumptions must be made. The assumption made here is that the NHL is a parity-filled league. No team is really bad and no team is really good. The difference between the performance of a player while he is on the ice and and his team's performance while he is off the ice on his team would be roughly constant if a player was used in the same role on a different team. Given the weak link between a player's +/- and his ability (as he is often on the ice for goals he is not directly involved in), I think this is a reasonable assumption. I think it is no more problematic than using +/- as a starting point for this study. Obviously, other normalizations are possible and will give different results, but I do not think these differences are huge (at least in most cases).

The fact that 15 of the 20 players on the top 20 list come from playoff teams is strong evidence that it is not tiled toward bad teams. In fact they are under-represented if this list was a random list of teams. It shows that good teams tend to have good players - which agrees with common sense.

The specific example Mathman points out is Shawn Horcoff of the Edmonton Oilers, who makes the list in a large part because his team was awful when he was not on the ice. Horcoff is the player (who played 50 or more games last year) who has the worst off ice +/- rating. The fact he does not come from a team which can be realistically called the worst in the NHL (Edmonton finished 19th out of 30 teams last year) helps to show the model of a parity-filled NHL is a good one. It is probably not surprising that the player with the worst off ice +/- has a good adjusted rating in a parity filled league. The worst off ice +/- ratings are not all from one or two bad teams. They are spread out throughout several teams in the league. The player who's team played the worst without him on the ice is likely a valuable player to that team and likely to be one of the better players in the league. Of course, there are many questions that can be asked upon looking at the list. How would Shawn Horcoff have done on a better team? How would Milan Michalek have done if he didn't play with Joe Thornton? How would David Perron have done if he was played in a larger role in St Louis? These types of questions cannot be answered easily. They show one of the fundamental flaws in hockey sabermeterics. It is likely impossible to remove a player's contribution from his environment. This is a limiting factor in any kind of sabermetric argument.

Along with the top 20 players we can also look at the 20 worst players. These are players who struggled last season. They did not do well in their roles. In most cases, they neither scored well, nor played defence well. These are players who will likely see reduced roles in the upcoming season (if they don't wind up out of NHL jobs).

Top 20 On/Off Ice Adjusted +/- Ratings 2007/08

Rank	Player	Team	On Ice +/-	Off Ice +/-	Adjusted +/-
1	Craig Adams	Chi	-1.96	+0.74	-2.70
2	Dallas Drake	Det	-1.30	+1.16	-2.46
3	Niclas Wallin	Car	-1.32	+1.09	-2.41
4	Marcel Goc	SJ	-1.66	+0.71	-2.37
5	Radek Bonk	Nas	-1.61	+0.63	-2.24
6	Eric Godard	Cal	-1.56	+0.59	-2.15
7	Adam Burish	Chi	-1.48	+0.45	-1.93
8	Mark Smith	Cal	-1.19	+0.71	-1.90
9	Ryan Hollweg	NYR	-1.26	+0.59	-1.85
10	Colton Orr	NYR	-1.36	+0.48	-1.84
11	Trevor Letowski	Car	-1.18	+0.57	-1.75
12	Alexander Semin	Was	-0.96	+0.79	-1.75
13	Vitaly Vishnevski	NJ	-1.03	+0.67	-1.70
14	Jerred Smithson	Nas	-1.20	+0.48	-1.68
15	Kirk Maltby	Det	-0.82	+0.79	-1.61
16	Kris Draper	Det	-0.30	+1.27	-1.57
17	Jarret Stoll	Edm	-1.58	-0.04	-1.54
18	Maxim Afinogenov	Buf	-1.04	+0.50	-1.54
19	Mike Fisher	Ott	-0.80	+0.73	-1.53
20	Jed Ortmeyer	Nas	-1.07	+0.40	-1.47

A quick look at this list shows that Mathman's complaint about the top 20 list is better suited for the bottom 20 list. This list is primarily filled with players on playoff teams. Those six who did not make playoffs were on some of the better teams that missed the playoffs (two Chicago Blackhawks, two Carolina Hurricanes, one Edmonton Oiler and one Buffalo Sabre). Every player on this list played for a team that had a positive +/- rating when he was off ice except for Jarret Stoll (who was barely a minus at -0.04). This somewhat confirms the talent structure in the NHL. There are a few best players. Most come from the best teams in the NHL. That is why those teams are the best. At the bottom of the NHL talent pool are many players who are roughly interchangeable with one another. Those who are used in tough checking roles on good teams who are not particularly good players will be on the bottom of this list. That is almost the entire list. There are a couple players who are nominally scorers, who did not score particularly well who also make the list. This list identifies some of the players who most struggled in their roles in 2007/08 and thus may see reduced roles in the future.

Craig Adams ( who I had listed as the worst regular in the NHL at one point this season) does the worst on this list. He is followed by the retiring Dallas Drake and a rare defenceman in Niclas Wallin. Marcel Goc and Radek Bonk are next. Bonk is the first player who is nominally a scorer (in that he is not used in a checking role). Eric Godard, Adam Burish and Mark Smith are next. They are followed by New York Rangers Ryan Hollweg and Colton Orr ( who I picked as the worst regular player in the NHL last season). Next up is Trevor Letowski and then Alexander Semin. Semin is probably the best offensive talent on this list, but he fell to a disappointing 42 points last year and has never been considered a defensive talent, even when he is scoring. Vitaly Vishnevski is the second defenceman on the list (joining Niclas Wallin). He is followed by Detroit's Kirk Maltby and Kris Draper. These players were once valuable checkers (Draper won the 2004 Selke Trophy), but time is passing them by. Jarret Stoll is next, followed by Maxim Afinogenov (who is nominally a scorer, but his 28 points were a failure). Mike Fisher and Jed Ortmeyer round out the list.

This list is a group of players who did not have good 2007/08 seasons. Most of them played on pretty good teams, but were used unsuccessfully in checking roles against top competition. These are players who probably should have been used in reduced roles, given their effectiveness last season, but they may not be the worst players in the NHL. This +/- normalization method makes it hard to fall this far if you play on a bad team. There are other ways to look at the problem to try to find such players. I would imagine that the worst players are roughly evenly spread throughout the NHL. Each team's weakest couple players are roughly interchangeable. It is the top players that differentiate good from weaker teams. This list is too heavily slanted toward players on good teams, but I think the worst players in the NHL are likely found uniformly on both good and bad teams. There is nobody on this list that I would suggest was a good defensive player in 2007/08.

Adjusting +/- for on and off ice ratings is a valuable tool to understand the NHL. Like any number it is imperfect and those imperfections must be kept in mind. That said, the top players in their adjusted +/- rating were some of the best players in the role and the worst +/- rating belong to players who struggled in their roles. Their identification is valuable to planning the future of your team.