The score is tied. Time is ticking down. The kicker is lined up for a Super Bowl-winning field goal. Does the opposing coach call a timeout to ice him or let him kick?
Conventional wisdom says yes, call the timeout, give the kicker an extra second to overthink the kick. But what do the analytics say?
Ryan Elmore, associate professor in the Department of Business Information and Analytics at Daniels, is an expert in statistics in sports and has published research on this exact scenario. His research, Beyond the Box Score: Does Icing the Field Goal Kicker Work in the NFL?, explores the factors that go into icing the kicker and whether it truly impacts the result.
With the Super Bowl just days away, the Daniels Newsroom spoke to Elmore about his research and what he’d advise coaches to do if they face this decision.
What originally motivated this research?
This research grew out of my previous work on NBA timeouts, where I studied whether calling a timeout when the opposing team is on a run actually affects the game. After that paper was published, I was talking to Ron Yurko, who runs the Carnegie Mellon Sports Analytics Camp, and I told him I thought I had a problem that would be really good for students in that camp. I wanted to continue looking at problems where you’re evaluating a coaching decision—not just team performance or player performance. Icing the kicker fit that mold perfectly. It’s completely dependent on the coach calling a timeout, and there’s a very clear outcome that happens right after that decision. We can measure that outcome and ask, “Is this affecting the result?” I just liked the idea of evaluating a coaching decision that directly.
What were your primary findings? Was it different than what you expected?
We (myself, Ron, Adriana Gonzalez Sanchez and Sierra Martinez) found that icing the kicker has a negative but negligible effect on field goal success. The effect is not statistically meaningful, however. The bottom line: icing the kicker really didn’t have much of an impact on the outcome. It’s mainly a function of distance and the conditions present in the game—turf versus grass, dome versus outdoor, and so on. The direction of the effect aligned with what most people believe (I think), but the effect size is pretty negligible. It’s essentially folklore.
How much should NFL teams/coaches lean on analytics vs. their own gut?
I would say: Don’t hold onto a timeout just to ice the kicker. Icing isn’t really going to have much of an effect. You should call timeouts when you need them throughout the game. However, at the end of the game, if you still have a timeout, there’s no reason not to use it since it doesn’t carry over into the next game. Go ahead and use it even though our research suggests it’s not going to affect the outcome. What’s it going to hurt, really?
The more important lesson is that analytics can reveal better timeout strategies. In the paper, we highlight a 2022 Wild Card game between the Chargers and the Jaguars. At the end of that game, Brandon Staley, the Chargers’ coach, called a TO right before the Jaguars attempted a 36-yard field goal. Looking at Staley’s decision to save his timeout during the 2022 NFL Wild Card playoff game, our estimated effect of his decision to call the timeout corresponds to a decrease in probability of making the kick of less than one percent. But Staley could have called a timeout earlier to force the Jaguars into running another play, which could have resulted in a longer kick. After Riley Patterson successfully made the field goal, Mike Lopez, director of data and analytics at the NFL, tweeted that the kick “snuck inside the right upright by about 5 inches. If the kick was from 39 yards, it would’ve hit the right upright.” That’s the kind of strategic thinking analytics can support—focusing on what actually matters, which in this case is kick distance.
When you watch sports, are you scrutinizing decisions through an analytical lens?
Honestly, I don’t watch a lot of sports in general aside from my 11-year-old’s games—and that’s as a parent, not an analyst! And if I’m totally honest, I’m not a huge football fan. I’m more of an NBA or professional soccer fan when I do watch. I occasionally go to hockey games with my family and I’m always telling them that the losing team should pull their goalie earlier than they typically are pulled (analytics suggest this). I was super happy to pick my son up from a hockey game the other day and his first comment was “They didn’t pull the goalie until 1 minute and 30 seconds left!” I think I’m naturally drawn to situations where there’s some element of gamesmanship, moments where I can evaluate whether a coaching decision actually affects the outcome.
Are you working on any other sports-related research?
Always! I just had a paper published in the Annals of Applied Statistics on shot taking and making in the NBA. I’m revisiting that idea with two DU students (Kaela Guerra and Nathan Stanley) and fellow Daniels professor Ben Williams because I thought of a much easier method to accomplish the same goal. I’m also working on a golf paper comparing LIV vs. PGA tour players, and another paper with professor Andrew Urbaczewski on behavioral economic implications in gambling.
If you are interested in sports-related research, get in touch! We are building out the DU Sports Analytics Lab and I’m always interested in collaborating with faculty or students.