Saturday, February 18, 2006

The Incorrect (But Common) Way To Determine That A Coach Is A Good One

It is a very hard sabermetrics and hockey problem to determine which NHL coaches are the best and how valuable they are to their team. How much value does the average coach add to (or take away from) a given team? In a perfect sabermetric world, we would be able to analyze hockey on the level of Bill James's win shares system in baseball (where he builds up the number of wins a player produces from his baseball stats - and a total of all the players on a team's win shares correspond very well to the team total wins). This type of system is not available in hockey, but if it were I define the best coach as the one who would produce the most win shares.

I think it is clear that, with few exceptions, coaches are less important to a team than a superstar player is. It is also clear that having a superstar player does not guarantee that a team will be a serious contender. There are examples of teams with very good goalies (ie Roberto Luongo) that have not been very good teams. There are examples of teams with very good defenders (ie Scott Niedermayer in Anaheim) that are not very good teams. There are examples of teams with very good forwards (ie Alexander Ovechkin or Ilya Kovalchuk) that are not very good teams. There should also be examples of very good coaches on teams that are not very good. A coach is usually far less valuable then any of these superstar players and if superstar players miss the playoffs, then superstar coaches should miss playoffs too (and probably with greater frequency then superstar players)

When evaluating coaches, this fact is often ignored. Because of the lack of any meaningful statistical data to evaluate coaches, people often attribute the wins- losses record to the coach. When evaluating coach of the year candidates, generally the coaches who happen to coach teams that do better than their prediction are chosen. With this method, there is nothing that is more successful at making good coach than a bunch of young players who come of age and become NHL stars at the same time (except possibly comeback years from superstar players). Clearly none of these things are directly consequences of the coach. In the coach of the year comments, Tom Renney of the New York Rangers, Peter Laviolette of the Carolina Hurricanes and Lindy Ruff of the Buffalo Sabres are picked as coach of the year candidates. I think that they are quite likely the three guys who will wind up as Jack Adams trophy nominees this year, but I do not believe that they have been the three best coaches in the NHL this season. All of these coaches are chosen by the method of picking out the coach who happens to be on a team that did way better than their team was expected to do.

How much of the New York Rangers improvement was Tom Renney's coaching and how much was the resurgence of Jaromir Jagr and rookie goaltender Henrik Lundqvist? How much of the resurgence of Jaromir Jagr and success of rookie goaltender Henrik Lundqvist are due to the way Tom Renney has coached them? Can this really be decoupled in any meaningful manner?

Similarly, how much of the Carolina Hurricane success is due to the emergence of Eric Staal as an NHL superstar and the resurgence of Rod Brind'Amour (as well as the addition of Cory Stillman and other good players)? How much of the emergence of Eric Staal is due to Peter Laviolette? How much of the great season Rod Brind'Amour is having is due to Peter Laviolette's coaching?

Similarly in Buffalo, there is a huge group of young talented players who have taken a big leap this season. This group includes Ryan Miller, Maxim Afinogenov, Ales Kotalik, Tim Connolly and Thomas Vanek. They have a hot young improving core. They also added very steady talented defenceman Teppo Numminen. Could their success be attributed to him and his steadying influence on defence? How much of the success of the developing young core on the Sabres is due to Lindy Ruff's coaching? How much of Teppo Numminen's comeback year in Buffalo is due to Lindy Ruff's coaching?

All of these are open sabermetrics questions that we cannot clearly answer, but all of them show these teams improvements are due to many factors, only one of which is coaching.

It is very hard to show statistically that a coach is a very good coach. It is very hard to show that a team's improvement is due to coaching and not other factors. It is very possible that a team that improves will improve for reasons that have nothing to do with coaching. This is especially true in cases where a coach has been with a team for several years (ie Lindy Ruff in Buffalo). There is likely no change in the team's performance due to coaching because it is the same coaching. It is easiest to show good coaching in cases where there is a coaching change, because there is a clear difference in coaching from year to year.

The best methods to try to identify good coaching come from looking at "system" type statistics. A well coached team will often show up statistically. One attempt to do this that I showed earlier this season (that I give as an example) is a reduction in goals against this season despite the NHL's goal scoring increasing. This will not find all the teams that are well coached. It will best find teams that had changes in coaching from one season to the next. In November, when I showed these numbers the four teams that had reduced their goals against were the New York Rangers, Phoenix Coyotes, Detroit Red Wings and Ottawa Senators. All have good new coaches playing good systems. These coaches are Tom Renney (Rangers), Wayne Gretzky (Coyotes), Mike Babcock (Red Wings) and Bryan Murray (Ottawa). To update this study, at the Olympic break, goals against this down this season in only one city. Only the New York Rangers have managed to maintain a lower goals against average so far this year (compared to 2003/04). Ottawa, Detroit and Phoenix have all fallen off of the pace. There are likely several reasons for this. Good coaching brought in a good defensive scheme that was lacking in previous years and the emergence of goaltender Henrik Lundqvist are probably the two biggest.

No methods like this will necessarily identify the best coaches in the NHL. Most of these studies are based on the fallacy that good coaching will make a team improve. Good established coaches who have been in a city for many years should not lead to an improvement of the team (as their coaching is no better than it was last year). In those cases the improvement is mostly due to the players on the team. Picking coaches of the year as coaches of the most improved team will make poor choices. It is this method that is generally used in the NHL. This method often prevents the best coaches from receiving the recognition they deserve. Scotty Bowman is thought by many people to be the best coach in the NHL history (if not he is likely the best coach in the time when the Jack Adams trophy was given out for coach of the year). Bowman coached for 30 NHL seasons. Only twice was he picked as coach of the year. That is a crime. Its clear he was the best coach in the NHL many more times then this, although he was not coach of the most improved team.

How does one determine the best coach in hockey? The only successful method is to watch games and see how teams are coached. There are no clear rules. There are no perfect statistical tricks. Finding the best coach is more art then it is science.

When I watch hockey, the coach I see who has the biggest effect on the way his team plays and the success that it has is Jacques Lemaire of the Minnesota Wild. Lemaire is the only coach in the history of the franchise, so its improvement will not be due to coaching change. In 2003, when Minnesota was the most improved team in the NHL and made the playoffs for the first time in their history, Lemaire won coach of the year. I argue he has been the best coach in the NHL for the last several years. Of course he is not coach of the most improved team in the NHL each of those years (and that is how the coach of the year is chosen). There are other well established coaches such as Pat Quinn and Ken Hitchcock that I think are very good coaches, though not as good as Lemaire.

There is no NHL team that has their success clearly linked to the coaching system then Minnesota. Minnesota plays in the best division in the NHL. The other teams have superstars like Jarome Iginla, Joe Sakic, Markus Naslund and Chris Pronger. Minnesota's best player is Marian Gaborik, a man who probably will be as good as they are one day, but he is yet to have a truly dominant NHL season. Minnesota has only one player who has scored over 45 points this year (Brian Rolston). They only have two more with more than 35 points (Gaborik and Pierre-Marc Bouchard). They do not have a potent offence. Minnesota has a fast hard working team defence, but they lack any star defenders. Probably Filip Kuba is their best defender, most NHL teams have more than one player of his calibre on their defence. This defensive system sets up a situation where goaltenders can thrive. Manny Fernandez and Dwayne Roloson are putting up excellent statistics, though it is largely due to the way their team plays in front of them. Minnesota does not have a top dominant team, yet in the toughest division in hockey they are fighting for a playoff berth. If you found an "average" coach to replace Lemaire, they would likely be a clear also ran.

I am pretty certain that Jacques Lemaire will not win coach of the year, because his team is not the most improved team in the NHL. I am pretty sure Lemaire will not win coach of the year because his team will most likely narrowly miss the playoffs. I am pretty sure that the coach of the year will be one of the coaches on one of the most improved teams (Renney, Ruff or Laviolette). I think there is no coach who has been more valuable to his team's success this year then Jacques Lemaire has in Minnesota. I think that is what should win coach of the year. However, history has shown that it is not the way the Jack Adams trophy is decided.

Comments:
I don't think Lemaire should be considered if his team doesn't make the playoffs... the ultimate goal for every coach in the regular season is to make the playoffs and if they can't do it, then they don't deserve to win the award.
 
It is very possible to be the best coach one season and also be on a team that does not make the playoffs.

The same way it is quite likely that Alexander Ovechkin will be the best rookie this year and miss the playoffs.
 
I have to agree with anonymous.
Comparing rookie of the year vs coach of the year is apple and oranges.
When you have as it stands now, 9 teams in your conference ahead of you, you don’t blame the rookies for that, you blame the coach.
You only have one head coach on the team. His performance is measured on how well his team does. Tenth in the conference doesn’t cut it for that award.
Ovechkin’s success is measured by his stats. Since there are about 20 other players on the team you can’t hang the team’s standings on the rookie.

Yeah I know I am biased, but if the Flyers make out of the first round of the playoffs my vote would be for Hitchcock. Just for having the Flyers where they are currently in the standings with the insane amount of injuries they have had to deal with so far this year.
 
Alexander Ovechkin is more valuable to his team then any coach is to his team. That said, Washington will not make the playoffs with Ovechkin this year.

It is very possible that the best coach in the NHL could be in a similar situation. In fact, I am claiming Jacques Lemaire is in a similar situation in Minnesota. A coach - even the best coach in the NHL - is not enough to put that team in the playoffs. That doesn't change the fact that he is the best coach in the NHL.

That said, he won't win the coach of the year. Other people who were not as good coaches this year will win.

The problem is that people don't know how to pick out a good coach. They pick the coach ofr the most improved team for coach of the year and that is often a very different thing from the best coach in the NHL that season.
 
I editted the initial post a little bit to try to better state my initial point (mostly addition to the opening paragraph), as these comments suggest that maybe it was missed by some of its readers.
 
If a coach does not get his team to the playoffs, then that coach does not deserve to be coach of the year. Coaches that get results, are much more deserving to get coach of the year, since it is an award based on what you have done during one season and not what you have done during your coaching career or coaching tenure with a team.

The rookie of the year is more of an individual award than coach of the year, and therefore can be won by an individual on a poor team.

I would also have to disagree with how you value Ovechkin, because Ovechkin isn't getting the Capitals into playoffs... coaching will. If they brought in a coach such as Jacques Lemaire, and the players played how he wanted them to, then the Capitals would be competing for a playoff spot, and the coach WOULD be more valuable to his team than one player because that coach would get wins, which no one player can do on his own.
 
If a coach produces more wins than any other coach (this is the results coaches produce), then the coach should be coach of the year. It should not matter if the coach's team produces enough wins to go with that for the team to make the playoffs or not (no coach by himself can produce this result).

You make two further claims that I disagree with:

1) The rookie of the year is more of an individual award than coach of the year, and therefore can be won by an individual on a poor team.

The coach of the year is as much an individual award as the rookie of the year (or MVP or best defenceman or ... ) or any other award that is given to an individual. Thats why it is given to one coach. The awards given to individuals (as opposed to teams) are individual awards. You are merely using this false premise to justify a double standard.

2) I would also have to disagree with how you value Ovechkin, because Ovechkin isn't getting the Capitals into playoffs... coaching will. If they brought in a coach such as Jacques Lemaire, and the players played how he wanted them to, then the Capitals would be competing for a playoff spot, and the coach WOULD be more valuable to his team than one player because that coach would get wins, which no one player can do on his own.

Ovechkin is one of the best players in the NHL. That said hockey is a team game. You cannot win with one superstar but no team. You also cannot win with one superstar coach but no team.

When you claim that Jacques Lemaire coaching Washington would be enough to put them into the playoffs (I'm no certain if this is true - but for the sake of argument I can accept it), you basically concede that Lemaire is a very valuable coach. Now were Washington to add a coach like Peter Laviolette would you imagine he would put them into the playoffs? If not what does that tell you about the value that you perceive for these coaches? And you want to give coach of the year to the one that is less valuable?
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?