QB Development Criticism: A Data Driven Analysis

Since the beginning of this season, I've noticed that comments trashing ability of the VT staff to develop QBs have become increasingly common and generally agreed upon on TKP, particularly the opinion that Hendon Hooker and Quincy Patterson are way behind schedule in their development as 4* recruits.

I had a hunch that the fan expectations for our QBs were largely born from recruiting accolades and did not align with the historical realities of QB performance at the collegiate level. As such, I decided to compile data on the Top 30 QB recruits per 247 composite ranking and Elite 11 Finals attendees for each recruiting class going back to 2010 (247 started rating recruits in 2010 and the older data sourced from other sites is prone to inaccuracies). I chose to use the Top 30 QBs + Elite 11 recruits as these constraints capture players with similar expectations to HH and QP based on recruiting accolades, as well as provide a significant collection of data points (324 total, though the performance of many recruits from 2016 onward is still TBD).

Next I attempted to develop an objective ranking system based on collegiate performance. This system was refined as I started looking at the career stats (taken from Sports Reference) for each of the players. After some tweaks, I decided upon the following system and including the use of 0.5 points to capture performance between two given ratings:

  1. P5 - very limited/no experience or poor performance; G5 - below-average or worse performance
  2. P5 - backup/limited experience, average performance or spot starter, below-average performance; G5 - full-time starter, average performance
  3. P5 - full-time starter, average performance (no all-conference teams); G5 - full-time starter, above-average performance
  4. P5 - full-time starter, multi-season above-average performance or single season all-conference; G5 - full-time starter, multi-season all conference or Player of the Year
  5. P5 - full-time starter, multi-season all-conference or single season POY or national award winner

After recording a rating for each player, I looked at historical performance of the entire sample set, 5* recruits, 4* recruits, 3* recruits, Elite 11 recruits, recruits comparable to Quincy Patterson (+/- 0.03 recruiting rating), and recruits comparable to Hendon Hooker (+/- 0.03 of his recruiting rating). Additionally, I created broader classification categories including Start % (% of recruits in a given category that made at least one P5 start), Bust % (% of recruits with a rating < 3), and Stud % (% of recruits with a rating > 3.5).

So what does this data tell us?

First off, being a 5* recruit means a hell of a lot more than anything else. Nearly 1/4 of 5* QBs are studs and 2/3 make a P5 start... but 1 in 3 are still busts. Elite 11 QBs have a similar chance of being a stud but less than 1/2 make a P5 start and almost twice as many are busts compared to 5*s. The difference between 4* recruits and 3* recruits in the top 30 are much less dramatic, with 4* having a slightly better chance to make a P5 start and lower chance to bust while the stud rate is essentially equal.

As far as QP and HH go, recruits with comparable ratings are nearly twice as likely to be a bust as they are to make even 1 P5 start and are 3.5 times more likely to be a bust than a stud. In fact, the probability of both QP and HH being a bust is 47%, while the probability of both being studs is 4%. Honestly I don't know enough to judge Fuente and staff's ability to develop QBs, but even if QP and HH never succeed at VT it is far from damning evidence. In fact, they exceeded the odds with Josh Jackson, who was a 75%+ chance to bust, developing into a serviceable multi-year starter.

Anyway, I hope this gives some context to the current situation at QB and shows that while it's exciting to get caught up in recruiting hype and hope for the next coming of MV7, the reality of recruiting hype translating into on-field performance at the P5 level is far less certain. Ultimately, I agree that the staff needs to find or develop a QB that can run the offensive more effectively- whether that guy is a highly touted 4*, a JUCO transfer, a walk-on, or whatever. Prior to this season, I had suspected that QP's development was going to be the deciding factor in Fuente's tenure but after running the numbers I can see why Fuente continues to bring in QBs to compete and would want to play the numbers game to increase the chances of finding a stud.

**BONUS TRIVIA QUESTION**
Which QB has the most P5 starts since 2010?

Finally, I would be more than happy to share my master spreadsheet to anyone interested. I'm not a statistician by any means and would love to see someone in the community add to it or look at the data in a different way to provide additional insights. Initially I thought this would be a quick project and it took far more time than I expected, as figuring out seemingly simple things like how many P5 starts some players made turned out to be pretty time consuming (damn you, Jeremy Johnson).

DISCLAIMER: Forum topics may not have been written or edited by The Key Play staff.

Comments

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Plan for the worst and hope for the best, not the other way around.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Plan for the worst and hope for the best, not the other way around.

Please join The Key Players Club to read or post comments.

"I liked you guys a lot better when everybody told you you were terrible." -Justin Fuente

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Plan for the worst and hope for the best, not the other way around.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Plan for the worst and hope for the best, not the other way around.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

"A person is smart. People are dumb, panicky dangerous animals and you know it." - K

Please join The Key Players Club to read or post comments.

Hokies, Local Soccer, AFC Ajax, Ravens

Please join The Key Players Club to read or post comments.

Take the shortest route to the ball and arrive in bad humor.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

If it ain't orange, it better be maroon...and if it ain't maroon, it better be soon!

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

"A person is smart. People are dumb, panicky dangerous animals and you know it." - K

Please join The Key Players Club to read or post comments.

"A person is smart. People are dumb, panicky dangerous animals and you know it." - K

Please join The Key Players Club to read or post comments.

"A person is smart. People are dumb, panicky dangerous animals and you know it." - K

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Joffrey, Cersei, Ilyn Payne, the Hound, Jeff Jagodzinski, Paul Johnson, Pat Narduzzi.

Please join The Key Players Club to read or post comments.

Plan for the worst and hope for the best, not the other way around.