Masters of the Universe Are Doing it Wrong

https://fivethirtyeight.com/features/th ... 538twitter

Opening paragraph

No position in professional sports is more important or more misunderstood than the quarterback. NFL scouts, coaches and general managers — the world’s foremost experts on football player evaluation — have been notoriously terrible at separating good QB prospects from the bad through the years. No franchise or GM has shown the ability to beat the draft over time, and economists Cade Massey and Richard Thaler have convincingly shown that the league’s lack of consistent draft success is likely due to overconfidence rather than an efficient market. Throw in the fact that young QBs are sometimes placed in schemes that fail to take advantage of their skills,1 that red flags regarding character go unidentified or ignored2 and that prospects often lack stable coaching environments, and there is no shortage of explanations for the recurring evaluation failures.

Click to expand...

And the drum that people who are the apparent unimpeachable masters of the universe of football knowledge aren't actually that great at what they do keeps beating.

[tweet]https://twitter.com/Moo12152/status/1101137192829751296[/tweet]

Supporting Tweet by author of article

[tweet]https://twitter.com/SeahawksMachine/status/1101164527289335808[/tweet]

Additional work done by Field Gull's John P Gilbert.

So now we have a notion that RW is elite, right

First; the efficient market hypothesis only works as a descriptive tool, meaning it can only reasonably tell you want happened in the past, not what will happen in the future.

Second; any model done retrospectively can be adjusted to fit whatever narrative you want.

What they've done is create a model using the efficient market hypothesis to fit their narrative. If NFL teams applied this model it would lead to entrainment which in effect would lead to many and more players slipping through the cracks, this is in effect just as fragile of a system as it is trying to replace.

A simplistic example of this fragility being exposed is Russell Wilson; if the NFL's previous model (albeit less scientific) were that you should only draft QBs over 6', Wilson would represent a shock to the system that would lead to more short QBs drafted higher. The same would be true of this model; the moment 1 or 2 players who don't fit the 538 model start producing results that model will also start to breakdown under scrutiny, and in 10 years 538 will write about how NFL teams are still drafting QBs wrong based on these new variables.

It's interesting, but it doesn't really tell us anything. Instead of trying to come up with a model to eliminate uncertainty; NFL teams would be better off embracing the uncertainty. Striving to challenge whatever narrative is dominant in that given cycle. In other words, limit your risk to the fragility in the system, don't try to be better at forecasting which players fit a specific model of what has worked, in the past.

You're suggesting that coaches and GMs find up and down in trends by purely qualitative analysis that solely rests in their unimpeachable domain?

Second, yes curve fitting exists. Let me ask you, a data scientist, if you ever make predictions with a model and then evaluate how close the results match your model. I know you're a data scientist but I think you're almost presumptive on the definiteness of things I reference. I'm not saying, nor is the author saying that this is a definitive model. He's not building a production model car in a CAD program and then assuring us that the car doesn't need any sort of testing before the company ramps up full production.

Following point two, you seem to presume that the author or myself thinks there CAN be a definitive holistic model that spans decades and eras. I don't want to call it a strawman but it feels close to that because you're knocking down a claim that was never made.

Fourth, as a data scientist, have you EVER found anything of value in sports analytics? If so, explain when and where and why. I've never seen you have a positive opinion towards anything in that regard to the point that I think you're a crank about the whole enterprise at all and the only insight you can ever capably lend is "it's all hokum"

Following on point four, the failure of previous models to accurately capture expected future value and then translate that into prospective draft position seems like it's put you into a position of not even being curious. Does being a data scientist eventually rob you of curiosity or the pursuit of satiating that curiosity to the extent that you don't think other people should as well? If there had been more robust forum apparatus back when Sabre metrics were being developed you would have ghoulishly stalked the forums hissing at those who wanted to develop a tool to inference value?

Sixth, no matter what tools are developed GMs and coaches are always going to bet on their hunches thus delivering your wish of uncertainty. John Elway could be told repeatedly using any manner of anything that he kinda stinks at QB evaluation and he would never let go of the fact that he was a QB somehow gives him more prescient insight into what it takes to be a QB, despite the role difference in being an GM selecting QBs and a QB playing the position being pretty wide. You will always get someone who thinks they can beat alpha year over year over year by an almost esoteric and intrinsic understanding of how things work in the game. In spite of actual results and THAT creates the uncertainty.

For Pete's sake, people do irrational and circumspect things all the time based on how it indulges themselves. And to reference PC, he seems to pull down a whole lotta satisfaction in winning the game HIS way over simply winning at all to the extent that it seems to negatively impact his tactical prowess. I of course am speculating about just how much ego validation he pulls down but I've run thin on alternative explanations and it's congruent with what we often see on the field and statements he's made and books he's written. He has a method, no doubt, but he seems to love the method more than the results.

Please tell me this is all a noble crusade to protect data science from being viewed as alchemy.

You can have all the physical measurable tools in the world but you can't measure competitive drive, influence to team mates and the speed of processing a situation and acting on it.


Take Elway, great physical specimen as a QB, like Newton now his size and strength allowed him to take hits that would knock out other QB's. But he loved his arm more then the coaches and would force a lot of stuff. He wasn't a brilliant player in my book just a physical freak for his day.

Now look at a Stabler, Unitas, Montana, Wilson, Greise, Tarkenton type, every one of them had huge questions about a physical aspect, all were the type that led and raised the level of play of those around them, had that IT factor in more ways then just physicality.


My fellow Pre Wilson members here can pretty much attest to watching Wilson in his game in KC that first pre season and if not by the end of the first quarter then by the end of the game certainly knew that he had IT and only Armageddon would take him out of the starter position.


Brady could be stacked into that group as well, he had nothing other then a drive Belichek knew about from someone to convince him to get drafted.


Not things you measure, oh crap almost forgot about Dave Kreig also.


Why when I see people want to paste a wall of stats I cringe, mostly because they don't understand the game well enough to see what is going on beyond a stat sheet or feel that because they have a number associated with something they have to be right because they have advanced stats.

chris98251":240xowf6 said:

You can have all the physical measurable tools in the world but you can't measure competitive drive, influence to team mates and the speed of processing a situation and acting on it.


Take Elway, great physical specimen as a QB, like Newton now his size and strength allowed him to take hits that would knock out other QB's. But he loved his arm more then the coaches and would force a lot of stuff. He wasn't a brilliant player in my book just a physical freak for his day.

Now look at a Stabler, Unitas, Montana, Wilson, Greise, Tarkenton type, every one of them had huge questions about a physical aspect, all were the type that led and raised the level of play of those around them, had that IT factor in more ways then just physicality.


My fellow Pre Wilson members here can pretty much attest to watching Wilson in his game in KC that first pre season and if not by the end of the first quarter then by the end of the game certainly knew that he had IT and only Armageddon would take him out of the starter position.


Brady could be stacked into that group as well, he had nothing other then a drive Belichek knew about from someone to convince him to get drafted.


Not things you measure, oh crap almost forgot about Dave Kreig also.


Why when I see people want to paste a wall of stats I cringe, mostly because they don't understand the game well enough to see what is going on beyond a stat sheet or feel that because they have a number associated with something they have to be right because they have advanced stats.

Click to expand...

It's been my experience that this "IT" (intangibles) translates to most professions and life in general.

Military special operators (Army SF, Navy SEALs, Marine Raiders, Air Force PJs/CCs) are not consistently the biggest, best fighting, best shooting, meanest dudes in the military (although some of them are). Those making decisions during assessment and selection are looking for the ones who have a certain level of baddassery but also have things really squared up in their heads. They're looking for good team members who can also act exceptionally as an individual that won't ever quit and are smart, stay cool under pressure, and know how to use their brains on the fly.

This applies to regular civilian professions too. You can have brilliant people with PhDs or high GPAs in your organization, but if they're lazy, not good team members, can't self direct, and can't see the end result and how to get there they aren't really that good.

I'm more interested in how someone (proverbially) takes a punch to the face and reacts than what they look like on paper. You need some level of ability, just like with a QB, but if you can only be good when things are going right and can't pick it up when they go wrong you don't have "it".

bmorepunk":3eelinym said:

chris98251":3eelinym said:

You can have all the physical measurable tools in the world but you can't measure competitive drive, influence to team mates and the speed of processing a situation and acting on it.


Take Elway, great physical specimen as a QB, like Newton now his size and strength allowed him to take hits that would knock out other QB's. But he loved his arm more then the coaches and would force a lot of stuff. He wasn't a brilliant player in my book just a physical freak for his day.

Now look at a Stabler, Unitas, Montana, Wilson, Greise, Tarkenton type, every one of them had huge questions about a physical aspect, all were the type that led and raised the level of play of those around them, had that IT factor in more ways then just physicality.


My fellow Pre Wilson members here can pretty much attest to watching Wilson in his game in KC that first pre season and if not by the end of the first quarter then by the end of the game certainly knew that he had IT and only Armageddon would take him out of the starter position.


Brady could be stacked into that group as well, he had nothing other then a drive Belichek knew about from someone to convince him to get drafted.


Not things you measure, oh crap almost forgot about Dave Kreig also.


Why when I see people want to paste a wall of stats I cringe, mostly because they don't understand the game well enough to see what is going on beyond a stat sheet or feel that because they have a number associated with something they have to be right because they have advanced stats.

Click to expand...

It's been my experience that this "IT" (intangibles) translates to most professions and life in general.

Military special operators (Army SF, Navy SEALs, Marine Raiders, Air Force PJs/CCs) are not consistently the biggest, best fighting, best shooting, meanest dudes in the military (although some of them are). Those making decisions during assessment and selection are looking for the ones who have a certain level of baddassery but also have things really squared up in their heads. They're looking for good team members who can also act exceptionally as an individual that won't ever quit and are smart, stay cool under pressure, and know how to use their brains on the fly.

This applies to regular civilian professions too. You can have brilliant people with PhDs or high GPAs in your organization, but if they're lazy, not good team members, can't self direct, and can't see the end result and how to get there they aren't really that good.

I'm more interested in how someone (proverbially) takes a punch to the face and reacts than what they look like on paper. You need some level of ability, just like with a QB, but if you can only be good when things are going right and can't pick it up when they go wrong you don't have "it".

Click to expand...

To both your and Chris' points, there are a host of personality attributes (resiliency, ability to learn and apply technique, managing slumps, leadership and fellowship roles, delegation) that matter in performance and there's no way you're going to 'stat' around that reality and I shutter to think about all the hackneyed Wonderlic (itself a travesty) adjacent personality tests that try to measure those attributes.

FWIW, Pete seems to understand the value of players who have perseverance and determination and probably compensates for weaker area's of PC's coaching abilities.

We should keep in mind that tools don't have to be exclusionary to one another and really should be complimentary to one another. No one tool is going to provide a silver bullet of insight but we can get a less fuzzy composite picture by finding useful tools in concert.

bmorepunk":34z2ccjx said:

chris98251":34z2ccjx said:

You can have all the physical measurable tools in the world but you can't measure competitive drive, influence to team mates and the speed of processing a situation and acting on it.


Take Elway, great physical specimen as a QB, like Newton now his size and strength allowed him to take hits that would knock out other QB's. But he loved his arm more then the coaches and would force a lot of stuff. He wasn't a brilliant player in my book just a physical freak for his day.

Now look at a Stabler, Unitas, Montana, Wilson, Greise, Tarkenton type, every one of them had huge questions about a physical aspect, all were the type that led and raised the level of play of those around them, had that IT factor in more ways then just physicality.


My fellow Pre Wilson members here can pretty much attest to watching Wilson in his game in KC that first pre season and if not by the end of the first quarter then by the end of the game certainly knew that he had IT and only Armageddon would take him out of the starter position.


Brady could be stacked into that group as well, he had nothing other then a drive Belichek knew about from someone to convince him to get drafted.


Not things you measure, oh crap almost forgot about Dave Kreig also.


Why when I see people want to paste a wall of stats I cringe, mostly because they don't understand the game well enough to see what is going on beyond a stat sheet or feel that because they have a number associated with something they have to be right because they have advanced stats.

Click to expand...

It's been my experience that this "IT" (intangibles) translates to most professions and life in general.

Military special operators (Army SF, Navy SEALs, Marine Raiders, Air Force PJs/CCs) are not consistently the biggest, best fighting, best shooting, meanest dudes in the military (although some of them are). Those making decisions during assessment and selection are looking for the ones who have a certain level of baddassery but also have things really squared up in their heads. They're looking for good team members who can also act exceptionally as an individual that won't ever quit and are smart, stay cool under pressure, and know how to use their brains on the fly.

This applies to regular civilian professions too. You can have brilliant people with PhDs or high GPAs in your organization, but if they're lazy, not good team members, can't self direct, and can't see the end result and how to get there they aren't really that good.

I'm more interested in how someone (proverbially) takes a punch to the face and reacts than what they look like on paper. You need some level of ability, just like with a QB, but if you can only be good when things are going right and can't pick it up when they go wrong you don't have "it".

Click to expand...

I want the guy that see's that punch coming and makes you pay for throwing it by beating to the punch or throwing the uppercut and sending you reeling into the corner, or the guy that takes it but has the loaded counter punch that knock you out. Thats an IT factor.