The Puck Stops Here

When studying sabermetrics and hockey as it relates to goaltending the biggest problem is assessing shot quality. This data is not easily available to a fan my looking at a boxscore, however it is recorded with reasonable reliability and available through the nhl.com play-by-play reports.

In hockeyanalytics.com, Alan Ryder takes a look a shot quality to see what conclusions can be drawn. His published analysis is on the 2002/03 season and shows proof of principle of the method, but its unclear exactly how all of its conclusions would hold up in other seasons of NHL play.

If we look at the play-by-play for a random game (chosen as the February 4th, 2007 game where Washington defeated the New York Islanders 2-1) we can see multiple shots recorded. For example at 48 seconds in the first period, the first shot is recorded. It is an even strength shot by Shaone Morrisonn which is recorded as a 58 foot snap shot which was stopped by Islander goalie Rick DiPietro and held for a faceoff. All the shots in the game are recorded in this manner.

The NHL records six different kinds of shots. They are wrap-around, tip-in, backhand, slapshots, wristshots and snapshots. They also record the distance of each shot. This is done by the scorer of each game. He will click on a location on a screen that is the shape of an NHL rink and enter the shooter and type of shot. This type of manual data entry will have some errors in it. Sometimes (for example) wrap around shots are recorded at huge distances from the goal (in excess of 60 feet). This is clearly a coding error. I am sure there is sometimes error in recording the shot type (most commonly confusing snap and wrist shots) and the distance of the shot is sometimes coded incorrectly. Since the data is entered by individual scorers, it is quite possible that there is a bias in some scorers to (for example) enter shots as closer or further then they actually are or to list more shots as (for example) wrist shots then is correct. The data is imperfect, but nevertheless can be used for analysis. Of the longterm of an entire season, probably the errors average out to become unimportant.

From this data, it is possible to produce a model that will give the expected number of goals from a given group of shots. Alan Ryder separates several special case shots from the data to treat separately. First there are empty net goal shots. These are 100% dangerous as they will score every time. Second there are penalty shots. He lists them as 25% dangerous since thats roughly the penalty shot success rate in 2002/03. He defines long shots as any shot over 60 feet. They have a very low chance to go in. In the 2002/03 season they only score 0.6% of the time. Rebounds, which were identified as any goal or shot within 2 seconds of a previous shot (assuming the distance of the rebound shot to be less than 25 feet). These scored 41.1% of the time on power plays and 34.8% of the time at other times. He also looked at scramble shots, which were defined as any shot of less than 6 feet that was neither a rebound nor an empty net goal. On average 21.2% of these shots scored. All other shots were treated as normal shots. On the power play, 12.2% of normal shots scored, at even strength 7.9% of them scored.

By taking into account shot type and location, it is possible to produce a spreadsheet model that gives the expected number of goals for a given group of shots against. For example, if we have a 40 foot wristshot, we can look through the data and find the chances of a forty foot wristshot scoring. This can be normalized for the number of shots faced to give the shot quality against. Teams with good defences will allow low quality shots. Teams with poor defences with allow high quality shots. When we look at saves percentagers of goalies, it will become possible to see that two goalies with equivalent saves percentages are not equivalent goalies if they face differing shot quality.

It is also possible to study long term trends in the NHL. Does the obstruction crackdown of the last couple years lead to more traffic in front of the goal (and thus more tip-in goals for example)? To be meaningful, any spreadsheet model of expected goals from a group of shots must include enough shots to be statistically valid, but must not go back so far in history that changing circumstances are averaged into the data (as an obvious example shot quality from the 1950's when goalies went maskelss will be very different from that of today).

This attempt to assess shot quality is very valuable to the sabermetric study of goaltending. It allows for far better understanding of how well a given goalie is playing.