The Puck Stops Here

In this summer's sabermetrics and hockey posts, I have been looking at the problem of on and off ice adjusted +/- ratings as set out by Gabriel Desjardins of behindthenet.ca. In order to show that the on/off ice adjustment is an improvement, I have looked at the effect of this adjustment on the offensive goals for average part of +/- ratings and showed that this adjustment appears to make an improvement to the rankings, although it is not a large improvement as offence is a largely individual skill. A better test is to apply this adjustment to the individual goals against averages since defence is a much more team dependent skill. This adjustment will correct for team quality, but not for quality of linemates or quality of opposition, so although it should give a better list of good and bad defensive players (that raw GAAs) it should not be perfect.

Here are the top 10 on/off ice adjusted GAAs among players who played 50 or more games in 2007/08:

Top 10 Individual On/Off Ice Adjusted Goals For Averages 2007/08

Rank	Player	Team	GFA	Rank Unadjusted
1	David Perron	StL	+1.40	19
2	Michel Ouellet	TB	+1.26	93
3	Andre Roy	TB	+1.24	106
4	Riley Cote	Phi	+1.21	13
5	Daniel Carcillo	Phx	+1.10	15
6	James Dowd	Phi	+1.10	42
7	Mike Lundin	TB	+1.10	168
8	Andreas Karlsson	TB	+1.02	208
9	Doug Janik	TB	+1.01	138
10	Kamil Kreps	Flo	+1.01	26

There is no question this adjustment shuffles the order of the players on the list. Nobody from the unadjusted top ten appears on the adjusted one. George Parros, the unadjusted number one, slides back to 14th place to be the highest ranking of the unadjusted top 10 on the adjusted list. It is also clear that this list does not do a good job of identifying players who are defensive stars. It identifies players who played against low quality opposition and did an acceptable job. The list is dominated by Tampa Bay Lightning players (five of the top 10) who did not match up against top opposition. Since Tampa was a poor defensive team, they gain the most in the on/off ice adjustment (as they are compared to teammates who could not prevent goals against top opposition).

The question to ask is has this list been improved by the adjustment? The answer is not so clear. Gone are players on good defensive teams like Anaheim and Detroit and in are players on bad ones like Tampa who did well relative to teammates due to facing poor opposition. If I had to select 10 players for a defensive situation, I would chose the unadjusted group over this one because it contains defensive stars in Travis Moen and Johnny Oduya and no defensive talents of their level are found on this list. Perhaps, James Dowd is the most respected defensive player on the adjusted list. However, this may be a step in the right direction. The problem is that it is clear that quality of opposition is much more important than team adjustments, so this list is not in any meaningful final state. +/- ratings will come out better than this because a player must score as well as have a low GAA to make a top +/- list. This removes some of the players on this list who play against weak opposition and neither score nor allow goals with any frequency. The top two players in David Perron and Michel Ouellet do appear on an on/off adjusted +/- top player list but this is because they can score and thus succeed in their role with their team.

Here are the 10 worst on/off ice adjusted GAAs in 2007/08 among players with 50 or more games played:

Worst 10 Individual On/Off Ice Adjusted Goals For Averages 2007/08

Rank	Player	Team	GFA	Rank Unadjusted
1	Marcel Goc	SJ	-1.44	41
2	Vincent LeCavalier	TB	-1.27	1
3	Alexander Semin	Was	-1.24	19
4	Ilya Kovalchuk	Atl	-1.21	2
5	Anders Eriksson	Cal	-1.18	29
6	Maxim Afinogenov	Buf	-1.16	13
7	Thomas Vanek	Buf	-1.13	15
8	Jussi Jokinen	TB	-1.13	18
9	Phil Kessel	Bos	-1.12	87
10	Martin St Louis	TB	-1.10	3

The worst adjusted GAAs are not as shuffled a list from the unadjusted list. The unadjusted list was made up of players on bad teams who face tough opposition. Some of them remain on this list in Vincent LeCavalier, Martin St Louis and Ilya Kovalchuk, but they are adjusted to not appear as bad. Other players with defensive weakness do appear on this list in much higher positions. This includes people like Maxim Afinogenov, Anders Eriksson and Marcel Goc. I believe that this is a list of worse defensive players than the unadjusted list, although it is clearly not a list of the worst defensive players in the NHL. The adjustment worked better at the worst end. This is because these are players who do play against tougher opposition and are not protected from it by their team matching lines.

While it is clear that the on/off ice adjustment does not produce definitive defensive rankings when applied to individual GAAs, it is likely a step in the right direction. Another significant step must be made to adjust for quality of opposition. I think this is a more significant step than the on/off ice adjustment. This little study does show that it is hard to rate defensive play sabermetrically. It shows that more work must be made in this direction.

On/off ice adjusted +/- ratings are a valuable tool to see which players excel and struggle in their roles with their teams. They do a pretty good job of measuring the offensive contribution of a given player and although they are not perfect, they do a better job at rating defensive play than unadjusted +/- ratings. Rating defensive play is hard. Definitive rankings do not exist right now, but I continue to take a look at a few more techniques that people use to try to measure them.