You need more of the probability distribution for the plays to make that decision. Most trick plays involve a handoff deep in the backfield which can result in large losses, which can kill drives. One reason why football teams run the ball even though the yards per play is lower than passing is that the lower variability of yardage gain helps to sustain drives resulting in points.<p>On 4th down you need to figure in the opportunity cost of the punt (~35 net yards). There was a paper ~12 years ago that covered this in detail using a dynamic programming approach to calculate the value of a first down at each yard line for a team, to calculate the value of say punting. It concluded NFL teams punt too much. Nowadays you see a lot more 4th down attempts than 10 years ago. I duplicated this approach for Division 3 college games and came up with an even more aggressive result.<p>But good read, it gave me an idea for some analysis myself.
This is a fun post. I love stuff like this. I don't think, however, that it's a particularly good example of "good analysis" - this analysis is clearly done either by (a) someone who isn't fully aware of all of the various strategies and nuances of NFL football, or (b) someone who is aware of such things but was in a rush to get this post written. There's just too much missing information to draw any conclusions such as those drawn by OP.<p>A trick's play's success rate is based on many factors. Saying that the down number is the primary factor is like rating a woman's attractiveness by how smart she is. Such a rating would, of course, give you false positives and negatives because it represents only part of the whole. A trick play's success rate depends on down number, position on the field, the clock, defensive tendencies, score, weather, field conditions, player injuries, and more. For example, a flea flicker isn't going to be attempted as often in really bad weather, or when you are up by four touchdowns, or when the main player who is the ultimate receiver of the ball (the "flea", perhaps?) is out of the game due to injury.<p>So I like this sort of analysis but it's completely useless due to its incompleteness. But I still am glad OP shared because I like seeing all the thoughts, functions, and so forth.
The 1977 Broncos finished the regular season with one loss to the Dallas Cowboys. It seemed a bit fragile to me, though, given the three or four games won by trick plays. They me the Cowboys in the Super Bowl, and were soundly beaten...