Wednesday, July 23, 2008
Another Way To Adjust +/- Ratings
Over the past few days, I have been looking at the sabermetrics and hockey problem of adjusting +/- ratings for on and off ice performance, the framework of which was developed by Gabriel Desjardins of behind the net. I have posted the top 20 and worst 20 players last season in these stats, in hopes of eventually addressing player's individual defensive values.
There are many ways to look at this problem and today, I plan to lay out the framework of another method. One of the flaws of the on/off ice adjustment is that it is a rate measurement and not a counting measurement. This allows for players who have limited roles against either high or low level of opposition to be rated in the top or bottom of the league ahead (or behind) those players who play significantly more minutes in much more varied circumstances. Nobody cares who scored the most points per minute, unless they also scored the most total points. Similarly there should be less interest in who had the best +/- per minute unless they played enough minutes to lead the league. In part, this problem prevents many defencemen from being in the top or bottom of the on/off ice adjusted +/- ratings since they usually play more minutes and in more varied circumstances than forwards. In fact, only Johnny Oduya and Nicklas Lidstrom made the top 20 ratings and only Niclas Wallin and Vitaly Vishnevski made the bottom 20. Defencemen are under-represented. One way to fix this is to treat +/- ratings as a counting stat and not a rate stat.
We still have the problem of how to normalize +/- ratings between teams. One solution is offered in the book The Hockey Compendium by Jeff Klein and Carl-Eric Reif. Jeff Klein can be found today writing on the New York Times hockey blog SlapShot.
The method is to find a team's +/- rating (we will use Detroit as an example). Last year they scored 252 goals. 81 were power play goals, so they scored 171 goals that will count toward +/-. They allowed 179 goals. 57 were allowed while shorthanded. This gives 122 goals that count toward +/-. The Detroit Red Wings have a team +/- rating of +49. Now there are five players on the ice at most times, so this rating is split between these five players. Dividing it by five we get a team +/- adjustment of +9.8 for the Detroit Red Wings last year. That is the baseline that all Red Wings players are compared to. On Detroit, Pavel Datsyuk had a +41 +/- rating, while Kris Draper had a -2. After adjustment, this gives Datsyuk a +31.2 rating (which is very good) and Draper has a -11.8 rating (which is not so great).
In the future, I will look at who are the top rated players and the worst rated players by this method in the future and compare them to the on/off ice adjusted players to see exactly where the differences are and how meaningful they are.
There are many ways to look at this problem and today, I plan to lay out the framework of another method. One of the flaws of the on/off ice adjustment is that it is a rate measurement and not a counting measurement. This allows for players who have limited roles against either high or low level of opposition to be rated in the top or bottom of the league ahead (or behind) those players who play significantly more minutes in much more varied circumstances. Nobody cares who scored the most points per minute, unless they also scored the most total points. Similarly there should be less interest in who had the best +/- per minute unless they played enough minutes to lead the league. In part, this problem prevents many defencemen from being in the top or bottom of the on/off ice adjusted +/- ratings since they usually play more minutes and in more varied circumstances than forwards. In fact, only Johnny Oduya and Nicklas Lidstrom made the top 20 ratings and only Niclas Wallin and Vitaly Vishnevski made the bottom 20. Defencemen are under-represented. One way to fix this is to treat +/- ratings as a counting stat and not a rate stat.
We still have the problem of how to normalize +/- ratings between teams. One solution is offered in the book The Hockey Compendium by Jeff Klein and Carl-Eric Reif. Jeff Klein can be found today writing on the New York Times hockey blog SlapShot.
The method is to find a team's +/- rating (we will use Detroit as an example). Last year they scored 252 goals. 81 were power play goals, so they scored 171 goals that will count toward +/-. They allowed 179 goals. 57 were allowed while shorthanded. This gives 122 goals that count toward +/-. The Detroit Red Wings have a team +/- rating of +49. Now there are five players on the ice at most times, so this rating is split between these five players. Dividing it by five we get a team +/- adjustment of +9.8 for the Detroit Red Wings last year. That is the baseline that all Red Wings players are compared to. On Detroit, Pavel Datsyuk had a +41 +/- rating, while Kris Draper had a -2. After adjustment, this gives Datsyuk a +31.2 rating (which is very good) and Draper has a -11.8 rating (which is not so great).
In the future, I will look at who are the top rated players and the worst rated players by this method in the future and compare them to the on/off ice adjusted players to see exactly where the differences are and how meaningful they are.
Comments:
<< Home
The best possible system would apply linear regression to play-by-play data, a la the NBA's adjusted +/-:
http://www.countthebasket.com/blog/2008/06/01/calculating-adjusted-plus-minus/
It automatically accounts for backups, opponents, and just about everything else you can think of. Unfortunately, I don't think the dataset necessary for the calculations is freely available on the internet. Behindthenet obviously has a similar dataset, though; I wish they would take the plunge and run the true "adjusted +/-", the linear regression kind.
http://www.countthebasket.com/blog/2008/06/01/calculating-adjusted-plus-minus/
It automatically accounts for backups, opponents, and just about everything else you can think of. Unfortunately, I don't think the dataset necessary for the calculations is freely available on the internet. Behindthenet obviously has a similar dataset, though; I wish they would take the plunge and run the true "adjusted +/-", the linear regression kind.
The concern with a calculation like that is the uncertainties that go with any of the numbers. Although it might be mathematically satisfying to have an answer, unless it provides any further insight than simpler methods it serves little purpose.
I would argue that the methods I am outlining are for all reasonable intents as useful as the result of such a calculation. You cannot reliably draw any conclusions as to who is better from two players who have approximately the same rating (regardless of method). Their difference is lost in the error of the method.
My interest lies in knowing what is needed to get reliable results (with some estimate of reliability) with as simple a model as possible. I don't believe that a more complex linear regression model is any better given the level of statistical information gathered in a hockey game and the often weak correlation between individual players and events on the ice.
Post a Comment
I would argue that the methods I am outlining are for all reasonable intents as useful as the result of such a calculation. You cannot reliably draw any conclusions as to who is better from two players who have approximately the same rating (regardless of method). Their difference is lost in the error of the method.
My interest lies in knowing what is needed to get reliable results (with some estimate of reliability) with as simple a model as possible. I don't believe that a more complex linear regression model is any better given the level of statistical information gathered in a hockey game and the often weak correlation between individual players and events on the ice.
<< Home
