Statistics on Offensive Predictability

Editor's Note: Bumped to the front because this is really, really well done. --Joe

Based on data available from cfbstats.com, I took a statistical look at how predictable VT’s offensive play calling has been thus far this season (note that I believe the last game is missing). The site provides basic information on every play in every game, but does not give information on specifics like formation, whether a pass was completed or not, etc. So this analysis (based on logistic regression if you’re a geek) simply looks at the most basic aspect of predictability – whether or not we rush or pass.

When we talk about predictability, the most important aspect is whether, based on easy-to-digest information, you can reasonably predict well what will happen. Predicting a team is 50% likely to rush is not particularly helpful because you have to plan for both. Predicting a team is 80% likely to rush (or 20%, which means 80% likely to pass) is very helpful.

This analysis was done by modeling the likelihood of VT rushing based on down, distance to go, position on field, score difference, and quarter. I consider these to be the major factor an opposing coach would use to generally guess what the offense will do (other than the obvious other factor of what have they done all game).

First, here are the odds of VT rushing on 1st and 10 based on the score difference (positive differences mean VT leads…negative differences mean they trail):

This general pattern is fairly obvious, as most teams with a lead want to kill clock by rushing and when trailing want to save clock by passing. But there is also an element of what you do best and how important that time is – in other words, if you’re good at rushing then early in the game you should probably continue to rush and not worry about the clock even if you trail, and start worrying about the clock later in the game.

So I grouped the graph by what quarter of the game it is and fit a regression to each quarter:

It appears that even in the first quarter VT is already trying to save or kill clock based on the score. This provides for some very predictable and less-than-optimal play-calling.

Next I compared the original graph to what all teams do to see how we match the norm:

VT appears to be much more likely to change play-calling as a reaction to the score difference rather than call what is most effective. The average team calls for a rush about 10% more often with a 20-point lead than with a 20-point deficit, likely because you don’t start killing clock with the rush until late in the 4th. VT, on the other hand, calls for the rush almost 60% more with a 20-point lead than a 20-point deficit. Remember, the further you get from the middle (50%) the more predictable you are becoming.

Next I looked at third-down plays. First some more descriptive information:

  • On 3rd and 5 or greater, we have run 3 times out of 45 plays. These occurred when we had leads of 21, 27, and 37 points. While these may be obvious passing downs, you have to at least occasionally surprise the defense.
  • Conversely, when we lose yardage on 1st down (2nd and >10), we have rushed 7 out of 15 times. This combined with 3rd downs makes me think we’re trying to catch defenses off guard but are too risk averse about it.
  • On 3rd and 1 or 2, we have rushed all 13 times. On 2nd and 1 or 2, we have rushed 8 out of 12 times.

Without accounting for the effect of score difference, here are the probabilities of rushing based on field position and yards to go (The red points are where we have data):

So almost regardless of field position, with 3rd and 7 or longer there is less than a 20% chance of a running play. I don’t think teams are stacking the box on us in those situations. But maybe all teams become this predictable in those situations? Well, here's the same graph for the average team:

Or maybe not. Turns out that there are very few areas where the pass becomes as likely as it does for VT, or similarly where it becomes as unlikely...for most teams, even 3rd and 1 from the opponent's 1 still has less than an 80% chance of being a run.
For fun, here is the same graph for VT's 2nd downs:

And once again, here is the same graph for the average team:

Also, the WR screens aren’t working.

Comments

Fuck yeah stats

Running serious statistics on football is one of my favorite things ever and this is really well done. The odds of rushing as dependent on score difference are really quite incredible and I imagine that the "3rd-and-Logan" plays are equally incredible. Hopefully Miami hasn't picked up on it so we can get the ACCCG for a serious #goacc moment.

"And loud, listen for yourself..." - Ron Franklin

The odds of rushing on 1st and 10 while leading in the first quarter gave me a serious sad. Although, I always felt like we do way too much of it, so I feel kind of good for noticing a trend.

I'll say it again, really well done, Sir.

And, I'm not going to lie, I had to read it twice, stats for engineers was so long ago.

I got a serious sad too. I always listened to predictability criticims with a grain of salt, but these are eye popping.

The lower left corners of those last 4 graphs really highlight how we live by our tendencies. If this was Vegas or AC I'd be rich armed with this information.

I'd be curious of how those graphs looked for the top teams. It is possible that similar tendencies of superior teams, that can win the mano-a-mano battles, are masked in the use of the "average" team in this analysis. If you got better talent, you win those battles despite being predictable. But just because Bama does things this way doesn't mean we should.

On the other hand, if the best teams are not this predictable, then we have some serious issues. If Bama isn't doing it we probably shouldn't either.

why oh why

do we have so much dark blue.

Other top teams

That should be the next post actually...how do we compare with the top teams in the country rather than everyone?

Great work!

Btw - I think that last graph should say 2nd down.

That is correct, but changing it sounds like a lot of work. Plus that last one incorrectly saying "3rd" instead of "2nd" is the only thing keeping opposing DC's from reading this and know what our offense is going to do (/end sarcasm).

I think this has the beginnings

of legitimate thesis.

I think your avatar has the beginnings of an even better thesis.

Moneyball beats beamer ball

Really this is Virginia TECH. I wonder if the team breaks down stats like this. Baseball does it, why not football.

I certainly haven't gotten a phone call, and I doubt Ken Massey - a VT alum who programs one of the BCS computer ranking systems - has either. VT has an excellent Stat department and I'm sure could get work done for free.

I like the analysis, but I have a dispute.

You are comparing the VT offense to "the rest". Not every offense is the same. You've got teams that are going to run every time (GT, Military academies) and you've got teams that are going to pass every time (Texas Tech). This makes the stats way more "average" in terms of showing the trends and comparing them to the VT offense. What I would like to see, and what may be more telling, is side by side comparison's with "successful" offenses that can be considered "balanced". I'd like to see the comparison of our offense to teams like Oregon, Clemson, heck, even West Virginia. Because I think it would be a it different than what you have here. If you're willing to do that, it'd be fantastic!

Logan 3:16

kind of what I was getting at above too...

I thought top teams would effectively eliminate unbalanced offense attacks, LSU not withstanding.

That should be next, although keep in mind that programs like GT and Texas Tech would move the intercept of that line but not change the slope. When GT is down by 10 in the first quarter, they don't start throwing right away because it's not what they're good at. You stick with what you're good at until late in the game.

Also, if you're going to be very one-dimensional like those, you better be really, really good at it. We're not so good at either as to get away with predictability.

I think we're in violent agreement, good stuff man, look forward to further analysis if you get the time.

Very cool and interesting stuff here

I will play devils advocate here. Against Clemson, two of those third and short runs were Logan dropping back and then running up the middle when Joey Phillips was covered in the right flat. Ok, now that I have that off my chest #FIRENEWSOME

Viva El Guapo

Great job man, really well done. I'm not a stats geek so some of the analysis behind the stats went over my head (plan on reading it later at home a few more times), but just the fact that you did this shows what kind of fans we have: intelligent and passionate ones. Go Hokies!

Kendall Fuller - future Thorpe winner
(STOP CRYING ABOUT DOWNVOTES)

what happend to

game planning against your own tendencies? Can we ask O'Cain that?

Take the shortest route to the ball and arrive in bad humor.

Do other teams...

...have this kind of in-depth, well-prepared analysis? And not just this post (kudos to joelestra) but every week after a game -as well as pregame musings?! And oh-my-f'eeng gawd the animated gifs always rock.

THIS
SITE
KILLS IT
!!!!......!

"...sticks and stones may break my bones but I'm gonna kick you repeatedly in the balls Gardoki!"