Sunday, August 05, 2007

Assessing Shot Quality

When studying sabermetrics and hockey as it relates to goaltending the biggest problem is assessing shot quality. This data is not easily available to a fan my looking at a boxscore, however it is recorded with reasonable reliability and available through the nhl.com play-by-play reports.

In hockeyanalytics.com, Alan Ryder takes a look a shot quality to see what conclusions can be drawn. His published analysis is on the 2002/03 season and shows proof of principle of the method, but its unclear exactly how all of its conclusions would hold up in other seasons of NHL play.

If we look at the play-by-play for a random game (chosen as the February 4th, 2007 game where Washington defeated the New York Islanders 2-1) we can see multiple shots recorded. For example at 48 seconds in the first period, the first shot is recorded. It is an even strength shot by Shaone Morrisonn which is recorded as a 58 foot snap shot which was stopped by Islander goalie Rick DiPietro and held for a faceoff. All the shots in the game are recorded in this manner.

The NHL records six different kinds of shots. They are wrap-around, tip-in, backhand, slapshots, wristshots and snapshots. They also record the distance of each shot. This is done by the scorer of each game. He will click on a location on a screen that is the shape of an NHL rink and enter the shooter and type of shot. This type of manual data entry will have some errors in it. Sometimes (for example) wrap around shots are recorded at huge distances from the goal (in excess of 60 feet). This is clearly a coding error. I am sure there is sometimes error in recording the shot type (most commonly confusing snap and wrist shots) and the distance of the shot is sometimes coded incorrectly. Since the data is entered by individual scorers, it is quite possible that there is a bias in some scorers to (for example) enter shots as closer or further then they actually are or to list more shots as (for example) wrist shots then is correct. The data is imperfect, but nevertheless can be used for analysis. Of the longterm of an entire season, probably the errors average out to become unimportant.

From this data, it is possible to produce a model that will give the expected number of goals from a given group of shots. Alan Ryder separates several special case shots from the data to treat separately. First there are empty net goal shots. These are 100% dangerous as they will score every time. Second there are penalty shots. He lists them as 25% dangerous since thats roughly the penalty shot success rate in 2002/03. He defines long shots as any shot over 60 feet. They have a very low chance to go in. In the 2002/03 season they only score 0.6% of the time. Rebounds, which were identified as any goal or shot within 2 seconds of a previous shot (assuming the distance of the rebound shot to be less than 25 feet). These scored 41.1% of the time on power plays and 34.8% of the time at other times. He also looked at scramble shots, which were defined as any shot of less than 6 feet that was neither a rebound nor an empty net goal. On average 21.2% of these shots scored. All other shots were treated as normal shots. On the power play, 12.2% of normal shots scored, at even strength 7.9% of them scored.

By taking into account shot type and location, it is possible to produce a spreadsheet model that gives the expected number of goals for a given group of shots against. For example, if we have a 40 foot wristshot, we can look through the data and find the chances of a forty foot wristshot scoring. This can be normalized for the number of shots faced to give the shot quality against. Teams with good defences will allow low quality shots. Teams with poor defences with allow high quality shots. When we look at saves percentagers of goalies, it will become possible to see that two goalies with equivalent saves percentages are not equivalent goalies if they face differing shot quality.

It is also possible to study long term trends in the NHL. Does the obstruction crackdown of the last couple years lead to more traffic in front of the goal (and thus more tip-in goals for example)? To be meaningful, any spreadsheet model of expected goals from a group of shots must include enough shots to be statistically valid, but must not go back so far in history that changing circumstances are averaged into the data (as an obvious example shot quality from the 1950's when goalies went maskelss will be very different from that of today).

This attempt to assess shot quality is very valuable to the sabermetric study of goaltending. It allows for far better understanding of how well a given goalie is playing.

Comments:
Nice summary - since a typical NHL season sees over 70,000 shots these days, I'm comfortable in having enough data to work with, but the issue becomes one of accuracy. Check out Ryder's piece "Product Recall Notice" over at Hockey Analytics, where he breaks down the trends in various arenas. For example, games at Madison Square Garden have their shots scored with much higher SQ than in other venues. In Tampa, however, the SQ is much lower.

My goal right now is to put a player's offensive production out there in terms of shots created and Expected Goals created, in order to take the level of opposing goaltending out of the equation. The trick is to figure out who's on the ice for each and every shot, so I'm combining time-on-ice data with the shot detail to drill down to that level...
 
I have looked at the NHL play by play data and the RTSS stats a lot and I have found them to be of very poor quality. I wouldn't quite say they are useless but they need to be used with that knowledge and understand than any conclusions made using that data. I have also found the shift data is significantly lacking too as a number of shifts are missing. Overall, the quality of NHL statistics is severely lacking.
 
David

Off topic. I cannot get onto your site right now. Is there a problem with it? Is www.hockeyanalysis.com down?
 
It looks like it is working now. Must have been a temporary problem.
 
Hi there every one, here every one is sharing these kinds of knowledge, so it's good to read this website, and I used to visit this webpage all the time.

Here is my web site ... internet marketing for dummies
 
Good day! I could have sworn I've been to this site before but after looking at some of the posts I realized it's
new to me. Regardless, I'm certainly pleased I came across it and I'll
be bookmarking it and checking back often!

my webpage - wiki.livepark.pro
 
Its not my first time to visit this website,
i am browsing this site dailly and obtain good data from here everyday.


Also visit my page; guaranteed seo
 
This is very interesting, You are a very skilled
blogger. I've joined your rss feed and look forward to seeking more of your excellent post. Also, I have shared your web site in my social networks!

Feel free to surf to my website ... ecig
 
Great info. Lucky me I found your blog by chance (stumbleupon).
I've bookmarked it for later!

Review my webpage akribos xxiv
 
What's up, just wanted to say, I loved this blog post. It was helpful. Keep on posting!

my web-site armitron watches for women
 
Having read this I believed it was extremely enlightening.
I appreciate you taking the time and energy to put this short article together.
I once again find myself spending a significant amount of time both reading and posting comments.
But so what, it was still worthwhile!

Also visit my blog ... stuhrling original watches for men
 
Excellent post. I will be going through many of these issues as
well..

Also visit my blog post :: relic watches
 
Thanks for some other informative site. Where else may I get
that kind of info written in such an ideal way? I have a project that
I'm just now operating on, and I have been on the look out for such information.

My homepage: powersupplywiki.com
 
I think this is one of the most vital information for me.

And i am glad reading your article. But wanna remark on few general things, The site style is ideal,
the articles is really great : D. Good job, cheers

Review my webpage http://moostic.ch/blog/index.php?userid=41134
 
Yesterday, while I was at work, my cousin stole my
apple ipad and tested to see if it can survive a forty
foot drop, just so she can be a youtube sensation. My iPad is now broken and she has 83 views.
I know this is completely off topic but I had to share it with someone!


my web blog ... quit smoking fag
 
whoah this weblog is fantastic i like reading your posts. Keep
up the great work! You understand, lots of persons are searching round for this info, you could aid them greatly.


Here is my blog Online HOme Careers
 
Excellent Blog! Thanks for sharing an informative post. ciri-ciri crystal x asli
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?