The Puck Stops Here

In order to look at the biases of the sabermetric problem of adjusting +/- ratings using the on/off ice method, I thought I would apply it to a simpler problem. That of goals for average (GFA) (namely how many goals are scored per 60 minutes a player plays in 5 on 5 situations). As we have seen, the goals for average leaders are among the best offensive players in hockey and the worst GFA players are rarely significant NHL players. By subtracting the off ice GFA from a player's on ice one we should get a better look at the offence a player brings independent of his team (if there is value to on/off ice adjusted +/- ratings). The potential problem with this line of logic comes from the assumption that both offence and defence benefit from the on/off ice adjustment even though offence is far less team dependant than defence. It is possible that there are few team effects to remove from individual offensive numbers when compared to defensive numbers and thus defensive numbers are the only place where the adjustment makes sense. However, as +/- rating are made up of both offensive and defensive numbers, looking only at the offensive numbers is a simplified system to see that the adjustment provides meaningful results. Let's see what the results look like.

Here are the top ten on/off ice adjusted GFAs among players who played 50 or more games in 2007/08:

Here are the ten best goals for averages (among players who played 50 games last season):

Top 10 Individual On/Off Ice Adjusted Goals For Averages 2007/08

Rank	Player	Team	GFA	Rank Unadjusted
1	Dany Heatley	Ott	+2.47	1
2	Sidney Crosby	Pit	+2.29	5
3	Jason Spezza	Ott	+2.22	2
4	Ryan Getzlaf	Ana	+2.20	19
5	Pavel Datsyuk	Det	+2.14	4
6	Paul Stastny	Col	+2.11	3
7	Alexander Ovechkin	Was	+2.04	8
8	Corey Perry	Ana	+1.98	32
9	Jarome Iginla	Cal	+1.87	13
10	Evgeni Malkin	Pit	+1.84	10

We are subtracting the goal scoring rate when a player is off ice from the player is on the ice. This is intended to see the effect the player has independent of his team. Does it? There are not many significant changes on the list by subtracting off ice goals for. The biggest change is the addition of two Anaheim players in Ryan Getzlaf and Corey Perry. Their climb is significant. I think this is because Anaheim has a low scoring dedicated checking line and Getzlaf and Perry benefit from their comparison to players who rarely score. The unusual Anaheim checking line makes simple sabermetric calculations more difficult.

If we ignore Anaheim players, does this list look improved as a result of the on/ice adjustment? Gone from the previous top 10 are Henrik Zetterberg (who is now 11th), Jason Arnott (who is 12th) and Derek Roy (who is 18th). In are Jarome Iginla and the Anaheim pair of Getzlaf and Perry. Neglecting the Anaheim pair and thus adding Zetterberg and Arnott to the list, only one player makes this list that did not before (Iginla) and only one player is dropped (Derek Roy). That is a small change, but one that makes sense to any reasonable hockey fan. As for changes within the top 10, the biggest rise is Sidney Crosby who moves from 5th up to 2nd. Again, this move is one that seems reasonable. Its clear Crosby is a very important offensive player and an offense ranking system that sees that is a good one. The biggest drop who remains in the top 10 (neglecting Getzlaf and Perry) is Jason Arnott. That seems reasonable. He is not expected to be as big a part of his team's offence by most fans as his top 10 rank would show. The biggest drop among top players is Derek Roy. Roy is another player who ranks surprisingly high in GFA and (like Arnott) does it playing on a balanced team that does not overly rely on one offensive line. As a result, the off ice GFA is strong enough to drop him on the list (this is despite the fact both Roy and Arnott do not play on as strong teams as many of the players ranked ahead of them). I would argue that from looking at the top 10 on/off ice adjusted GFAs, this list is closer to a list of the best offensive players in the game than the unadjusted GFA list, as long as we take into account the complication of Anaheim's checking line. There is no huge change between either list, so this is not exactly a strong convincing argument, but it is suggestive of being on the right track.

Now let's look at the 10 worst on/off ice adjusted GFAs. Here they are:

Worst 10 Individual On/Off Ice Adjusted Goals For Averages 2007/08

Rank	Player	Team	GFA	Rank Unadjusted
1	Ryan Hollweg	NYR	-2.14	1
2	Craig Adams	Chi	-2.02	14
3	Jarred Smithson	Nas	-2.02	10
4	Dallas Drake	Det	-1.95	7
5	Mark Smith	Cal	-1.92	8
6	Colton Orr	NYR	-1.91	2
7	Jed Ortmeyer	Nas	-1.87	24
8	Eric Godard	Cal	-1.81	11
9	Jeremy Reich	Bos	-1.69	3
10	Sami Kapanen	Phi	-1.65	13

The bottom list is more shuffled than the top is, which is expected because there is more differentiation between the best players in the NHL than the worst (in general the worst players in the league are roughly interchangeable with one another). The biggest surprise is that nobody from the Anaheim checking line makes this list, especially given Travis Moen having the 4th worst GFA unadjusted (he is 11th so he barely misses this list). Players on teams that are higher scoring than Anaheim tend to rise on this list (to worse rankings) and players on lower scoring teams end to have rankings improve. That is the point of adjusted rankings. Team effects are being taken into account. Four players make this list that do not make the unadjusted list. They are Craig Adams, Jed Ortmeyer, Eric Godard and Sami Kapanen. All play on relatively high scoring teams. Other players on higher scoring teams such as Detroit (Dallas Drake) and Calgary (Mark Smith) find themselves moving up the futility list. Players on lower scoring teams tend to improve their rankings (getting less bad). Examples of this are Colton Orr, rising from 2nd to 6th and Rodney Pelly who was 5th and is now 14th. I would argue that if we are looking for offensive talent (or lack of it) these adjusted lists are better than the unadjusted lists.

In order to test the effect of the on/off ice adjustment to +/- that I have looked at in several posts this summer, I decided to look at a simpler system. Goals For Averages (GFA) is this simpler system. Defensive contributions to +/- are removed. Since defence is harder to measure than offence this makes a simpler system. For the most part, lists with the on/off ice adjustment are closer to what I might expect as a ranking of top (or bottom) offensive players in the league. There is one problem introduced by this method. The problem occurs in Anaheim. Anaheim is a low scoring team. They have a low scoring checking line made up of Travis Moen, Rob Niedermayer and Sami Pahlsson who get significant ice time. This appears to give a large boost to Anaheim's scorers due to a low off ice GFA. This pushes Ryan Getzlaf and Corey Perry further up an adjusted list than they otherwise should be. Clearly, this adjustment is not perfect but it does appear to improve these offensive numbers by a moderate amount.