I've been a sports official since 1988 – almost as long as I've been writing about sports. I've called basketball, baseball, softball, and football games at various levels over the years, and was an official with NASCAR throughout my time there. I've been inspired by the detailed film reviews done here on TKP.
The ACC, and most other conferences, review performance of each crew and provides weekly feedback. Reviewing game film and feedback from the prior week is part of the routine for crews the night prior to a game. They also review film of the teams they will be officiating, much like an opposing coach. Therefore, one should realistically expect to see a continuous improvement in the performance of crews. Performance should improve game over game, as well as year-over-year.
If you're not familiar with the general responsibilities and roles of the officials, take a look at this article by Jon Johnston. Although there have been changes in mechanics, Johnston's article is a good primer. A recurring concept is the "key" - a matchup or area of responsibility of a specific official. Keys change each play, and are determined by formation, down/distance, and many other elements.
Officiating: Looking at Game FIlm
So what types of things can you see when you break down officiating film? Let's take a look at an important 3rd down pass play in the ECU game. Granted, this is an AAC officiating crew supported by an ACC replay staff. Third and seven is generally a passing down, and the keys for the officials are pretty clear. The Side Judge and Field Judge (out of the photo downfield) have the two widest eligible receivers, Phillips (Bottom) and Hodges (top). The Back Judge, also out of the picture, keys on the next interior receiver to the strong side of the formation, in this case Malleck on the field side. This frees up the Line Judge (out of the photo to the bottom) to key on Sam Rogers coming out of the backfield. That leaves the Linesman to take the interior receiver (Ford) on the boundary side.
The key to this sequence is the Linesman, visible at the top of the image. The Referee (top right) is responsible for Motley. The Umpire (center left) is responsible for interior line calls. On the snap, Isaiah Ford pushes upfield on a quick out beyond the first down marker. There is contact with the inside defender; the pass is nearly picked as the outside help jumps the route. Whether Motley led the throw too much, or whether the contact inhibited Ford's route is a judgment call by the official keying on Ford. Therein lies the problem. The Linesman is very late turning his head to follow Ford and is shielded off the play by Bucky Hodges and the ECU defender.
As Motley's arm starts moving forward, the Linesman is still looking across the line of scrimmage. By the time the Linesman turns, his head following the ball, Ford is already into his route and 9+ yards downfield and the contact has occurred.
A quick look at the end-zone replay angle shows the defender's right arm holding the back of Ford's jersey. After the near-pick, Ford is clearly looking around for a flag for holding or defensive pass interference. The same camera angle clearly shows the Field Judge focused on Bucky's matchup, as he should be.
Compounding the issue are three general philosophies of officiating. First, call what you see. The Linesman cannot make a call here, because he is not following his key. In essence, Ford and his defender are not being observed by any official until the Linesman turns to pick them back up. Second, stay with your keys. This means that the Side Judge and Back Judge are committed to their matchups, and would be scrutinized for making a call on someone else's key. Third, and most importantly, when in doubt – there was no foul.
This was an important point, even though early in the game. The change of possession gave ECU the ball on their own 45, and they subsequently scored a game-tying touchdown. This play epitomizes how much attention to detail is required by everyone on the field.
So What About VT's Penalties in 2015?
Here's an image showing some of the data, but for anyone interested, here's the actual Google Docs spreadsheet.
I've compiled a LOT of data from the game reports and film. I'm slowly working through game film to identify individuals penalized, official that threw the flag, and doing QI on the game reports. I'm sure you're all shocked to know stats geeks are human after all.
There are five sets of columns:
- Officiating Crew
- Flag Count
- VT Penalties, Offensive, Defensive and ST
- Opponent Penalties, Offensive, Defensive and ST
- Flags with a Direct Impact on the Game
The flag count breaks down accepted, declined, and offsetting.
Each team includes a breakdown of pre-snap, interior, downfield, etc.
Note that I included all flags thrown, not just those accepted.
One interesting metric was flags per offensive / defensive play.
If my fellow TKP members would like to see more breakdowns (literally and figuratively), let me know. Thank you to HokieTapes for access to the broadcast game footage.
Comments
love this. our own ref review. would love to see a "grade" for officiating for every game as you see it. I think laymen fans like myself often see refs as "out to get" the Hokies with bad calls, or at the very least possessing of Ron Cherry level incompetence. (you're not him are you?)
mostly it's just comforting to see the game from the blind mouse point of view. it's a fast game with lots of moving parts. interesting to know how this part works, or should work.
I literally laughed out loud. Nope, not Ron Cherry!
One of the interesting (initial) correlations is in the VT Defense - Opponent Offense, and in VT Offense - Opponent Defense. VT's defense and the opposing offense were flagged on ~14% of downs played, while the VT Offense and opposing defense were flagged at around 7% of snaps played. My curiousity wonders if this is a side effect of our D's aggressiveness, or is it the Ref's perception of our Defense?
Lots of metrics to potentially look at, not to mention the game film to see what wasn't called.
Thanks for the feedback.
This is excellent! Thanks for all of the work!
And I think this topic is perfectly timed for the end of the season. Allows a lot of bellyaching without being drowned out by the regular season :)
Oh, man. That is outstanding and, a very good piece to include regularly, if possible.
I have no idea what any of it means, but I can't resist this:
Ron Cherry,
Hmmm, that's the pot...oh...nevermind.
Great stuff! The game has gotten so much more interesting since I started officiating a couple seasons ago. Glad to see some time being devoted to the guys in stripes on here.
I agree; I'd love to be able to write a lot of positive but unfortunately I think it's more systemic. The ACC has a history of only turning over very few officials per year (per an interview by the ACC Football Officiating guru).
I'll refrain from making too many generalizations until I have more data to back it up, but I just don't see how the QI by the league office is impacting the on-field product.
Tell me about it. I'd love to get a shot at officiating big-time college football, but the overall quality of officials coupled with the low turnover rate isn't exactly encouraging.
I have worked with several of the ACC officials...believe me, they need to have MUCH more turnover. Many of those guys are pathetic....poor test results and terrible mechanics
That sentiment mirrors what I'm seeing on video; not following keys, poor signal mechanics, and the like. Just like the OOB play at the end of the Duke game, poor mechanics winding the clock. Yes, our coaches should've seen the clock turning. But the officials' mechanics were... dreadful. So the question then becomes, what's the ACC's motivation in maintaining status quo?
The old ACC days - where Lenny Wirtz gave every team NOT from North Carolina the business (Sampson-Worthy, anyone?), well that was actually what motivated me to become an official in the first place. I'm still irritated that Fred Barrakat (former ACC Director of Officials) didn't bother to answer the nicely written letter from a 12-year-old.
The "good old boy" network seems to remain thriving for some reason....ACC official selection is definitely NOT results based.
This is like an African Safari...scouting the Zebras! Love this type of stuff. Many thanks!
You ain't lion!
Nobody do that Kudu like you do.
I had to give you a leg because that was so terri-bull.
Oh, the hue-manatee.
Come on. Stop kitten around.
Can someone post a tldr?
Different official look at different things. There are apparently a lot of numbers associated with officiating and penalties.
Officiating is inconsistent, consistently. And the QI by the league doesn't seem to be effective.
This is fantastic. If you do decide to do a series of posts out of this I would love to see how different conferences handle the evaluation of their officials, if it leads to different rates of turnover, and if their is any correlation between turnover and improved grades (i.e. consequences for poor performance leading to better performance). Their is so much material to be mined from this subject.
Do you officials feel like the targeting call is becoming footballs version of the block/charge call in basketball, but with much more severe consequences? It's usually a very fast play and hard to tell who initiated the contact. What needs to change to make this call easier to get right?
I took some time to think about your comment before answering. It's an interesting analogy. First, football has the benefit of instant reply. That should be the equalizer but so far, it only seems to have added to the inconsistency of the call. Second, in football it's a lot about angle, maybe even intent. In basketball, it's much simpler to determine position and contact. I've Ref'd more basketball than any other sport, and block/charge is really not as difficult as it gets made out to be at times. A big part of officiating in the paint is staying fluid and positioning yourself to make the calls, and keeping your head on a swivel.
The last part of your question is tough. Making the call easier isn't necessarily the problem - getting officials to apply a consistent definition of the rule is probably the root cause. And two years in, they don't seem to be any closer to sorting it out even with league oversight and official education. That's one of the questions I want to look at as I continue compiling data. Once I have pulled data on all the ACC calls for 2015, maybe we can string together video clips and see exactly how this played out.
I officiated intramural basketball at tech for three years and found the block/charge was difficult on non breakaway type of plays when there are two or three people in the paint. An SEC ref came in and told us to look at the way a defender fell, if he fell pretty much square on his back it was a charge if he fell on either hip it was a block. I tried to think of a similar pattern for football and it seems logical that if a player makes contact with his facemask then it would be clean no matter what part of the body was contacted, any other part of the helmet should be reviewed. I also would like to see this called on offensive players who lower their head to ram through a tackle and make contact with the defender's head or neck.
Add me to the list of those who would love to read any thoughtful commentary on officiating. No other subject draws as much thoughtless, uninformed commentary as the hard work and occasional mistakes of the officials.
Like it and thanks.
This is... AWESOME. Few sites have anything approaching the football analysis of French, and I've never heard of any analyzing the officiating. (Analyzing, as opposed to bitching about.) And it's cool to learn a little bit more about how the job actually works... which has always been a blind spot for me (and, I think, many fans).
This is a great write-up. It saddens me that the ACC league office is apparently turning a blind eye to quality improvement for officiating.
Reading through your review raised a question that may be more for French or Mason than you. You pointed out that the Linesman's view of his "key" was obscured by Hodges and his defender. So my question is this: Do coaches spend any time designing route trees around Official positioning and keys? It would seem prudent for coaches to be familiar with which refs are looking at which match-ups and where those refs are positioned. I wonder if it's worth adjusting routes to ensure the appropriate ref has a clear view of each match-up.
This is an intriguing and insightful comment. Look, if you have to play the officials too, why not acknowledge it and do just that?
Interesting write up, would love to see more. Your comments about the low turnover and lack of , consistency leads some credence to my jokes on another post about who is worse, FIFA or the NCAA. Definitely looking forward to future posts!
Haha, FIFA is far worse in terms of corruption, but the NCAA takes the cake in officiating. FIFA is just ignorant, in my opinion, in regards to officiating. Soccer is still stuck in this mindset that an entire match can be judged by three guys, only one of which is on the field. That's their excuse, NCAA doesn't have one.
Fascinating stuff! I, too, would like to see this as a regular feature. All around excellent content.
Excellent work here! Thank you for taking time to break this down and share with us.
Maybe this is a good place to ask...but is Ron Cherry actually incompetent?
I know he's kind of goofy in his delivery of calls, but after watching the last couple of seasons, I've come to realize that he's about the only ACC ref I like seeing in a game. I don't recall many instances where it felt like he was "giving us the business", moreso than any other ref. Even that call against Dadi for hitting him, it felt like to me that he didn't want to necessarily through the flag on that, because he understood the emotion, but he was forced to as part of his job -- which is probably why he didn't eject him right then and there.
Also thinking back, he does try to be as clear and specific in his calls as he can. I just don't think the right words always come to him. He does tend to repeat himself sometimes, but it all appears to be in the interest in making sure everything gets communicated in the call. And yes, his movements are often exaggerated, but that's also because he's one man on a field that is being watched by thousands of people in the stands. Again...clarity. For instance, go back and look at the one point safety in the 2013 Fiesta Bowl (Oregon vs. K-State). Is there any confusion after Ron announces the call?
I honestly don't think Ron is incompetent. I may have joked about it before, and I've always loved the "business" gag.
But really, I've never seen him do anything egregiously incompetent. Maybe someone on his crew, but never him specifically. I think because of Ron's connection to that one call in an NC State game, he has become a punchline of sorts.
That's pretty much where I'm at.
He does add a level of entertainment to the game. I was flipping over to the Foster Farms Bowl after ours, and during the Redskins game, hoping for a penalty to see Ron call it. One of those times was the targeting review with ejection, complete with hand motion.
Not incompetent? Ron Cherry's crew did the UNC game this season, no?
This is a GREAT question. He definitely has a personality, but having personality as an official is a double-edged sword. Your job is to officiate the contest fairly, and let the players/coaches determine the winner. In that regard, I have to say initial data shows his crews are on the lower end of calling penalties that
hashave a direct impact on the game. The exception was the UNC game, and that was skewed by the DPI's on our comeback drive. I'm in the process of pulling other ACC games for the crews, just to see what we can learn.One of my officiating mentors put it best. You want to be recognized as "that Ref" but you don't want anyone to remember your name. At NASCAR, we constantly had to remind officials that they are there to support the show, not be a part of it. People pay to see the drivers / athletes, not the officials.
EDIT: Sorry for the rainy day grammatical error!
Hear, hear. This is why I find basketball so hard to watch.
Another one of my common criticisms of the ACC hoops bunch. Being a showboat (1) is poor officiating mechanics, and (2) is unprofessional. When a story on officials is the lead story (Karl Hess) it just detracts from the game. I had the PRIVILEGE of working games with a guy named Chase Vaden (son of NBA VP, Dir of Officials Don Vaden) in the Lynchburg area, and it was an amazing experience just by osmosis. Be friendly but not familiar, smile, laugh, know your rule book inside out, watch your own film to get better every game, and use really good positioning and mechanics. That's all it takes to be a superior official.
So, not Joey Crawford.
That's just too darn funny right there....
None of what I'm saying should be construed that there isn't a time and a place for some humor in the job (giving him the business... ) But that those moments are usually spontaneous and an exception, and are more appropriate when you're doing the rest of your job with excellence.
I also think that as fans, we tend to hear and see the same names and faces year after year after year. Even if the ref is just doing his job in a reasonably competent fashion, we're eventually going to know these refs. They cannot remain nameless and just "the ref" when we've seen them dozens of times calling our games. Picking on Ron Cherry is fun, and Lenny Wirtz was fair game for the big razz back in the day, but familiarity does breed contempt, especially when you are on the losing end of their call.
I might have done something close to this when I T'd up Chris Tucker in an intramural game when he was a GA. I also love that he looks over right after that second step to see if Kobe is watching
I prefer this version.
You win.
I didn't know Michael Bay did gifs too
Are you going to try at look at non-calls that have direct impact on the game?
Yes, but I'm looking primarily for inconsistency in calls. For example, holding in the UNC game. There were three holding calls against the VT offense, and none against the UNC offense even though it appeared our guys were being held on virtually every snap. In fact, my next piece is likely going to be specifically on holding for that very reason.
Sounds like you should be refereeing our games.
Can you look at timing of calls too, as they relate to in game "swing"? I have noticed, while we are on the Ron Cherry subject, that he will always try and even out the calls, usually to our detriment ina game time sort of way. In other words, some calls at certain times have more weight than others, and i think Ron Cherry is good at hiding some bias by covering his calls with other, less meaningful calls at other times, but can still claim: "But I called the same number of penalties on them too!" While that may be true, the impact is vastly different.
Yes. One of the things I've identified is what I am calling a a "Direct Impact" flag. Generally, these are flags that:
- Extend a drive that would otherwise have ended, resulting in points.
- Killed an offensive drive
- Negated a significant play, typically a scoring play, turnover, or sack
- Decrease FG Distance with a FG made, or increased FG distance resulting in a miss
- Back an offensive team away the goal line, with a FGA or other result when a TD was the most likely outcome.
I'm still refining these as I watch film, so it may change over time. So far, about 28% of flags thrown have this kind of impact, and all games but one have been within one standard deviation of that 28%. The outlier is the ECU game.
Another good example of inconsistency... 13 of 19 defensive offsides occurred in 5 games. Six games had only a single infraction, and none in the LOLUVA game. A lot of voices have said this pointed to "undisciplined" line play, and certainly some of these were obvious execution errors. But the video shows that in a couple of these, Lineman A lines up in Game A, no flag. , Same position, general situation, Game B, flag. And the more I look, the more examples I find across the board (i.e., not just VT games).
I know that inconsistent officiating on PI, holding, etc. must drive Bud nuts in the film room. I think that's what's driving me here; I love officiating and am usually the first to defend the stripes because I have a few of them hanging in the closet. But our guys - and the other ACC member schools - deserve consistent officiating, and if the inconsistency is this obvious to me from the cheap seats, why isn't it obvious in Greensboro?
because someone has an agenda?
#conspiracyTheoryThings
You're seeing it because you are actually looking. I'm willing to bet that the ACC collects data and grades the officials just so they can say that they do it when asked, but not as a genuine attempt to actually improve their officiating issues. The Good Ol' Boy network is rampant at all levels of the ACC. It is a culture that will continue to hold the conference back in relation to it's peers.
A few well written articles with video backing them up posted on a highly trafficked website that can be shared all over the web seems like a damn fine way to ask that exact question.
He is absolutely is incompetent. From his incomprehension of the rules, to horrendous mechanics, to inability to communicate....he is the epitome of lazy, and that is the most vile characteristic an official can possess. Most of all...he seems to place all priority on the "importance" of being a head referee rather than a passion for accuracy, precision, and organization. And it is not just him...but he stands out to me as the worst (shortest midget if you will)
Great stuff! I look forward to reading more of your work.
Wow I just learned that I apparently don't know anything about how officials watch the game. This is great stuff!! Please write more. Can you start with an overview written in crayon for the majority of us who only know they throw yellow hankies when they don't like you?? /s
Thanks!
This is a fantastic idea. It seems like a lot of us "non-refs" are finding this most basic information insightful. I am!
I know it's slightly different and this probably isn't the place, but Darren McFadden (RB) of the Cowboys looks like he has sent two Redskin DBs to the locker room with head/neck/concussion injuries. One of the DBs was Kyshoen Jarrett.
How many defensive pass interference calls were against the Hokies this year? Torrian's whole wait-until-the-last-possible-second-don't-locate-the-ball-play-to-jar-it-loose style of coverage generates a whole mess of penalties when not executed to perfection.
Statistically, I don't think I can isolate the flag-throwing to the secondary's play style even though in watching the games real-time, it had me venting "TURN AROUND!!!!" ... Virginia Tech Defenders were flagged for 17 holding / defensive pass interference calls, out of 50 total defensive penalties. Opposing defenses were flagged 13 times for holding / DPI, out of 30 defensive flags. In looking at film, there was a questionable / inconsistent application of the rule in several games on both sides of the ball. The ECU game, for starters.
The numbers so far seem to indicate that there might be a correlation between the overall defensive style of VT and increased penalties against the VT defense and opponent offense, but I will have to wait until I have numbers entered for all the 2015 ACC games to really say anything conclusively. The data shows that both teams were penalized less heavily when VT's Offense / Opponents Defense were on the field (7%) and more heavily when VT's Defense / Opponent Offense were on the field (nearly 14%).