Wednesday, August 15, 2007

The Successes of the Goals Saved Method

In my look at sabermetrics and hockey this summer I have looked at the goals saved method of assessing goalies. This method attempts to come up with a single number to score goaltenders - which represents the number of extra goals the goalie saved over the average goalie that season playing for the average team. This method is developed by the hockey outsider (Peter Albert).

This method is my no means perfect. Since shot quality data only goes back to the 2002/03 season, it does not include it. It also does not include anything a goalie contributes to his team besides making saves (i.e. defending dumpins or covering loose pucks in the crease that are not shots on goal) and it does not attempt to correct for the quality of schedule a goalie faces (which might be an issue in the unbalanced schedule of today's NHL). Nevertheless it gives a good look at who the best goalie in the NHL is in a given season. Saves percentage data only exists as far back as the 1951/52 season, so there are no results before that point.

Any goaltender who has a Vezina Trophy calibre season (using the modern day voted Vezina - until 1981 the Vezina was won by the goalie(s) on the team with the best goals against average) should be at (or at least near) the top of the league in goals saved. Usually this is true, but sometimes it is not. This method is very good at isolating those goalies. In the years before the Vezina Trophy was under its modern definition, I use the first team all star goalie in its place.

Here are the goalies who led the NHL in goals saved each year the data exists:

Goals Saved Leaders By Season
Goals Saved Leader Season Vezina Winner
Terry Sawchuk1951/52Terry Sawchuk
Terry Sawchuk1952/53Terry Sawchuk
Harry Lumley1953/54Harry Lumley
Harry Lumley1954/55Harry Lumley
Jacques Plante1955/56Jacques Plante
Glenn Hall1956/57Glenn Hall
Jacques Plante1957/58Glenn Hall
Jacques Plante1958/59Jacques Plante
Johnny Bower1959/60Glenn Hall
Glenn Hall1960/61Johnny Bower
Jacques Plante1961/62Jacques Plante
Glenn Hall1962/63Glenn Hall
Glenn Hall1963/64Glenn Hall
Glenn Hall1964/65Roger Crozier
Johnny Bower1965/66Glenn Hall
Eddie Giacomin1966/67Eddie Giacomin
Gump Worsley1967/68Gump Worsley
Jacques Plante1968/69Glenn Hall
Tony Esposito1969/70Tony Esposito
Jacques Plante1970/71Eddie Giacomin
Tony Esposito1971/72Tony Esposito
Ken Dryden1972/73Ken Dryden
Bernie Parent1973/74Bernie Parent
Rogie Vachon1974/75Bernie Parent
Ken Dryden1975/76Ken Dryden
Ken Dryden1976/77Ken Dryden
Ken Dryden1977/78Ken Dryden
Ken Dryden1978/79Ken Dryden
Denis Herron1979/80Tony Esposito
Richard Sevigny1980/81Mike Liut
Grant Fuhr1981/82Billy Smith
Pete Peeters1982/83Pete Peeters
Roland Melanson1983/84Tom Barrasso
Pelle Lindbergh1984/85Pelle Lindbergh
Bob Froese1985/86John Vanbiesbrouck
Ron Hextall1986/87Ron Hextall
Patrick Roy1987/88Grant Fuhr
Patrick Roy1988/89Patrick Roy
Patrick Roy1989/90Patrick Roy
Ed Belfour1990/91Ed Belfour
Patrick Roy1991/92Patrick Roy
Curtis Joseph1992/93Ed Belfour
Dominik Hasek1993/94Dominik Hasek
Dominik Hasek1994/95Dominik Hasek
Dominik Hasek1995/96Jim Carey
Dominik Hasek1996/97Dominik Hasek
Dominik Hasek1997/98Dominik Hasek
Dominik Hasek1998/99Dominik Hasek
Olaf Kolzig1999/2000Olaf Kolzig
Sean Burke2000/2001Dominik Hasek
Jose Theodore2001/02Jose Theodore
Marty Turco2002/03Martin Brodeur
Roberto Luongo2003/04Martin Brodeur
Miikka Kiprusoff2005/06Miikka Kiprusoff
Martin Brodeur2006/07Martin Brodeur

The goalie selected as the best goalie in the NHL agrees with the choice made by the goals saved method slightly over 67% of the time (37 out of 55 times). Nevertheless, there are times when the goals saved method clearly was wrong. The most obvious examples are when a team with a strong defence had their goaltender lead the league for several years in a row despite changing goalies annually.

This happened twice. The Montreal Canadiens goal led the league by goals saved from the 1975/76 season every year until 1981/82. For the first four years, Ken Dryden was their goalie, the next year it was Denis Herron and in the final year it was Richard Sevigny. More likely this is a testament to the Canadiens having the best defence in the league (Larry Robinson, Serge Savard, Guy Lapointe were all members of those teams) then to them having the best goalie in the league despite changing goalies. This also happened to the Philadelphia Flyers from 1984/85 to 1986/87- Over those three seasons, their goalies were Pelle Lindbergh, Bob Froese and Ron Hextall respectively. This was a team that had a defence built around Mark Howe, Brad McCrimmon and Brad Marsh and it was the best of its time. In both cases, likely this is a function of a top defence allowing few high quality shots more than it is goaltending.

Interestingly, the team that is known for having a top defence for many years that cannot pull off a long string of their goalies leading the NHL in goals saved is the recent New Jersey Devils. Martin Brodeur deserved his first career Vezina by the goals saved method this season, despite it being the third he actually won in his career. Brodeur is a good goalie to have deserved even one Vezina, but this is evidence that he may be a product of top defences giving him good statistics when he has only once led the league in goals saved and given the low shot quality his team allows, one would expect him to have been the goals saved leader on more occasions than just one.

The goals saved method is not a perfect way to analyze goaltending, especially because it does not take into account shot quality, but it does a good job of picking the top goalie in a given season and is correct more then 2/3 of the time in predicting the Vezina winner.

I appreciate you taking the time to discuss my Goals Saved method and you make some good points.

- I agree that my inability to include "adjusted save percentage" is a limitation. This tends to overrate goalies on good defensive teams (and underrated goalies on bad defensive teams). Unfortunately, I'm pretty sure that the data needed to calculate "adjusted save percentage" has only been around for a few years.

- It's interesting that Goals Saved matches up with the Vezina winner (or first-team all-star prior to 1982) about 2/3 of the time. Keep in mind that I'm not _trying_ to predict the winner, and I do believe that the actual Vezina winner has been undeserving in many cases.

I'm actually trying to see if there's a way to _predict_ the winner of the Vezina (I'm hoping for a 90-95% success rate) and I'll let you know if I can come up with a formula that works. (It might shed some light on what the awards voters are looking for when picking the Vezina winner).

- I know I'm tough on Brodeur sometimes, but he still does extremely well according to GS. He was runner-up to Hasek in 1997 and was often in the top five. His "career value" (cumulative GS) is quite high and should keep on rising. You also correctly point out that this method doesn't take Brodeur's puckhandling into account, so GS underrates his performance.
Sometimes the discrepancies are extremely small (between the GS leader and the Vezina winner).

- In 1961, Hall has 36.6 GS and 1st team all-star Bower had 36.3 GS.

- In 1966, Bower had 34.2 GS and 1st team all-star Hall had 33.7 GS.

- In 1975, Vachon had 63.8 GS and 1st team all-star Parent had 62.1 GS.

- In 1982, Fuhr had 40.0 GS and 1st team all-star Smith had 39.9 GS.

- In 2001, Burke had 31.7 GS and Vezina winner Hask had 31.5 GS.

So, five more of the discrepancies are extremely small (around 1%... a difference so small that no method could reasonably be expected to differentiate between two goalies). This brings the match-up rate to about 76%. (Though I again stress I'm not _trying_ to predict the Vezina here).

There are a few huge discrepancies, though:

- In 1958, 1st team all-star Glen Hall actually has a _negative_ score! I don't know why he won the Vezina. Looking at traditional stats, he's 3rd in wins (in a six-team league) and 4th in GAA, on a team that missed the playoffs by a wide margin. Chicago had a really bad team, but was Hall that good? (The GS leader was Jacques Plante, who made the second all-star team).

- In 1988, Fuhr won the Vezina and had a weak GS (2.3). But, he had an amazing playoff reputation at this point and reached the elusive 40-win mark. (Many modern fans don't realize how rarely goalies reached 40 wins in the 1980s and before). (GS leader Patrick Roy was the second-team all-star).

- The other two big discrepancies involve Brodeur in 2003 and 2004. I still think the award voters made bad decisions both years, especially in 2004, but I've already written lots about that elsewhere.
There will always be situations that the Vezina selection is a poor one (Jim Carey, early Brodeur wins) and situations where the goals saved method gives a poor selection (Richard Sevigny for example). It is interesting to see why both of those happened. It can show the mindset of the voters and it cvan show failings of this method (Sevigny led in goals saved most likely because the Montreal defence was so strong that any semi-competent Montreal started would have likely led at that point).

Now you claim to want ot find a method to predict Vezina winners (which obviously this isn't but its success at it shows it is a useful tool), but I think that is an exercize in futiliy. People vote the way they vote for reasons that are often irrational. Predicting irrational behavior with a rational model is a non-starter (and creating an irrational model is pointless). The best you can hope for is a method to decide who should win the Vezina (which often this is - although it has failings) and see if the voters get it right or not - sometimes they wont.
The strangest thing is seeing Sean (Shawn?) Burke on that list.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?