The Big Questions

I might as well just call this "quantifying hockey". What are we trying to learn?

  • What Is "Value"? Ultimately it's about things that either cause or prevent goals being scored.
  • What stochastic models can teach us about the game proper.
  • How do we communicate this in a clear way to practitioners?

The Big Questions (Statistical)

  • How do we value a "win"? In our context, we convert from "goals" (rule of thumb: 6 goals equals 2 standings points) – so how do we value a goal?
  • Do we want to measure "true ability", or correct for statistical anomalies in "realized performance"?
  • Why should we compare performance to "replacement" and not "average"?
  • A touchstone: OpenWAR for Baseball (Jensen, Matthews, Baumer)

The Basics

The Toolbox

The three most basic assumptions I'm going to make here are about structure:

  • The rate of the next occurrence of an event is semi-Markov, but Poisson/Cox processes get us most of the way there in a pinch.
  • Each event may also have degrees of success or failure, that we can model separately with binomial models.
  • We can swap out pieces later on to improve the realism, accuracy and precision of the model.

Prior Work

Way back to 2004-05 (Lockout II): Let's manually get some tracking data. Thomas 2006, JQAS

Prior Work

How to make it (semi)-Markovian: added in composite states reflective of real game outcomes.

Prior Work

Thomas 2007, JQAS: A distribution that generally works for goal-scoring times is almost Poisson distributed, but we didn't have enough game-level data to go beyond simple corrections, to uncover individual impacts.

Prior Work

Well, now we have enough data, more or less! Thomas et al 2013, AOAS

Premise: Both teams (home and away) have a goal scoring process; both compete with each other.

Simplest two-team model:

\[ \log \lambda_H = \mu_H + \omega_C + \delta_D \] \[ \log \lambda_A = \mu_A + \omega_D + \delta_C \]

\(\omega\) reflects the offensive skill of each team (higher is better); \(\delta\) reflects defensive skill (higher is worse).

The Full Model

Hand-coded fully Bayesian MCMC solution for this model:

\[ \log \lambda_H = \mu_H + \sum_k (\omega_k I(k\ on\ ice\ H) + \delta_k I(k\ on\ ice\ A))\] \[ \log \lambda_A = \mu_A + \sum_k (\omega_k I(k\ on\ ice\ A) + \delta_k I(k\ on\ ice\ H))\]

Elastic-net style Laplace-Gaussian shrinkage for parameters grouped on position (centers, wingers, defenders, goaltenders).

Pros: Great statistical properties. Interesting findings on each position's relative contributions. Can do semi-Markov structure if we need it.

Cons: Takes way too long to run in full. No one in industry hockey will get the details of what MCMC is. Needs years of data to overcome collinearity with goaltenders.

Integrating Data Sources

So we forge forward. First, let's take the existing data sets from the previous analysis and augment them. Primary: nhl.com

Integrating Data Sources

  • Secondary additions (x,y) data: espn.com, sportsnet.ca

All Together Now

Integrated data:

##     season gcode refdate event period seconds etype        a1        a2
## 7 20152016 20001    5027     7      1      51  SHOT deshada86 weiseda88
## 9 20152016 20001    5027     9      1      65  SHOT deshada86 weiseda88
##          a3        a4        a5        a6        h1        h2        h3
## 7 fleisto84 petryje87 emelial86 xxxxxxxNA kadrina90 boyesbr82 vanrija89
## 9 fleisto84 petryje87 emelial86 xxxxxxxNA kadrina90 boyesbr82 vanrija89
##          h4        h5        h6 ev.team ev.player.1 ev.player.2
## 7 harrisc93 gardija90 xxxxxxxNA     TOR   boyesbr82   xxxxxxxNA
## 9 harrisc93 gardija90 xxxxxxxNA     TOR   kadrina90   xxxxxxxNA
##   ev.player.3 distance  type homezone xcoord ycoord awayteam hometeam
## 7   xxxxxxxNA       35 Wrist      Off    -55      6      MTL      TOR
## 9   xxxxxxxNA       11  Snap      Off    -79      0      MTL      TOR
##   home.score away.score event.length    away.G    home.G home.skaters
## 7          0          0          5.5 priceca87 bernijo88            6
## 9          0          0          7.0 priceca87 bernijo88            6
##   away.skaters adjusted.distance shot.feature import.ies loc.section
## 7            6         30.976837       rushn8          0           0
## 9            6          9.958361                       0           0
##   new.loc.section newxc newyc score.diff.cat subdistance
## 7               6    59    -5              3           7
## 9              12    80     0              3           3

All Together Now

Events are recorded as the time from the previous event. We have event information on many contributing predictors:

  • Teams on the ice (home and away)
  • Players on the ice (separated into skaters and goaltenders)
  • Where on the ice the event occurred (by three zones)
  • x,y coordinates of shots (of all types)
  • The score and time during the game

Data (x,y) coordinates are manually recorded; stadium location bias fixed with a variant of a method from Schuckers 2011

All Together Now

Each of the processes lead to goals:

  • Faceoffs: Head-to-head one on one contest (Binomial)
  • Penalties: Events that lead to man advantages (Poisson)
  • Shot Rates: Put shots on net (Poisson)
  • Shot Conversion: How does a shot become a goal? (Binomial)

(Listed in relative order of importance, as it happens)

General approach: We can still use differential shrinkage, but:

  • Use glmnet() for speed and cv.glmnet() for optimal shrinkage in each group.
  • Speed advantages for now are extreme
  • Uncertainty calculations can come later

But First: What Do We Mean By "Replacement"?

A concept in development: A player that could be immediately acquired from the open market (standard definition).

Two purposes: 1) Comparative – how would a player in a similar position do? 2) Statistical – what should we shrink towards if not "average"?

Our proposed definition is the "Poor Man's Replacement": replace the indicator for a player who has a limited number of "trials" with a generic player identifier for all such players.

Step 1: Faceoffs

Initial inspiration Model is Bradley-Terry like. For the outcome as a faceoff win for the home team:

\[ Y_i \sim Bin(1, p_i) \]

\[ logit^{-1}(p_i) = \mu_i + \alpha_{home[i]} - \alpha_{away[i]} \]

What modifications?

  1. PMR 1: pool all players who are centers with fewer than 50 faceoffs taken in a season
  2. PMR 2: pool all players who are not centers with fewer than 50 faceoffs taken in a season
  3. Shrinkage: use glmnet() and cross-validation to regularize estimates (very little needed)

Step 1: Faceoffs

Step 1: Faceoffs

In both cases, we have detected a distinct, negative effect for a replacement level player.

Estimated effect against replacement:

\[ alpha_k - alpha_{repl} \]

Estimated performance: difference between realized wins and modelled wins with replacement player.

\[ F_{net}(k) = \sum_i (Y_i*I(home[i]=k) + (1-Y_i)*I(away[i]=k)) - \sum_i E(Y_i|replacement_k) \]

Conversion: 0.013 faceoff wins/goal.

Step 1: Faceoffs

Step 1: Faceoffs

Top 5 performances of last 10 years:

##               Name   season FO.GAR
## 1  Rod Brind'Amour 20052006   5.15
## 2 Patrice Bergeron 20132014   3.55
## 3  Rod Brind'Amour 20062007   3.45
## 4      Chris Drury 20052006   3.32
## 5 Antoine Vermette 20132014   3.27

Bottom 5 performances of last 10 years:

##              Name   season FO.GAR
## 1      Eric Staal 20092010  -0.74
## 2    Mike Ribeiro 20142015  -0.76
## 3 Markus Granlund 20142015  -0.78
## 4 Andrew Cogliano 20082009  -0.93
## 5     Kevin Hayes 20142015  -1.05

Step 2: Penalties

Full Poisson model for each individual, taken or drawn:

\[ Y_{k} \sim Po(\lambda_k T_k) \]

Execution: your favorite GLM method with shrinkage built in.

Poor man's replacement: one for each of taking/drawing, one for offense/defense (four total)

Step 2: Penalties

Step 2: Penalties

Conversion: 0.17 goals/net penalty (based on success rate of power plays)

Step 2: Penalties

Step 2: Penalties

Top 5 performances of last 10 years:

##               Name   season diff.GAR
## 1     Dustin Brown 20082009 9.110639
## 2     Dustin Brown 20092010 9.013138
## 3    Brad Richards 20052006 8.643763
## 4 Patrice Bergeron 20052006 7.759772
## 5     Dustin Brown 20072008 7.648206

Bottom 5 performances of last 10 years:

##           Name   season  diff.GAR
## 1    Ben Eager 20062007 -4.266480
## 2 Brendan Witt 20052006 -4.363690
## 3   Chris Neil 20052006 -4.719525
## 4 Jarkko Ruutu 20052006 -5.250302
## 5   Sean Avery 20052006 -6.876339

Step 3: Shot Rates

Reminder: the 2013 model for Poisson and goals started like this:

\[ \log \lambda_H = \mu_H + \alpha_C + \beta_D \] \[ \log \lambda_A = \mu_A + \alpha_D + \beta_C \]

\(\mu_H\) and \(\mu_A\) initially includes:

  • score of the game
  • in which zone the event happened
  • if the home team won the faceoff

Step 3: Shot Rates

Original paper had general criticism: goals are relatively infrequent when measuring ability. Splitting into shots on goal and goals from shots lets us decouple the goaltender from the shooting process.

Except there are building biases when it comes to recording shots on goal by building.

Shot Rate Pre-Processing

Assume for now that the only biases are undercount and overcount, without a "homer" bias to a particular team. Then we can preprocess our offset for every interval of the form:

glm (TotalEventsFor ~ factor(ScoreDifference) +
                      factor(FaceoffZoneWin) +
                      factor(TeamFor) + factor(TeamAgainst) +
                      offset = log(TOI), family=poisson)

Eliminate the team-for and team-against effects and fit the subsequent model with players with the remaining offset.

Shot Rate Splits

Decision time: known that different shots have different probabilities of success.

  • Could fit one Poisson model for all shot attempts, one super-binomial model for scoring on shots
  • Could fit multiple models for shot attempts based on discretization, one model each for scoring on shots

"Danger Zones"

Earlier adoptions of "shot quality" arguments weren't taken as seriously by many in the online community or by broadcasters. Our group's innovation:

  • Start with corrected shot locations as in the previous plot and bin by location.
  • If a shot is classified as a "rebound" or "rush", upgrade it to the next bin.
  • Discount shots that are blocked by opposing players due to (x,y) being the location of the block.

We now have low, medium and high danger shot attempts; we assign a Poisson process to each of them and fit accordingly. Also fitting 3 models for 5v5, 5v4 and 4v4 play respectively.