this post was submitted on 17 Nov 2024
11 points (61.2% liked)

Ask Lemmy

27240 readers
2601 users here now

A Fediverse community for open-ended, thought provoking questions


Rules: (interactive)


1) Be nice and; have funDoxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them


2) All posts must end with a '?'This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?


3) No spamPlease do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.


4) NSFW is okay, within reasonJust remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either [email protected] or [email protected]. NSFW comments should be restricted to posts tagged [NSFW].


5) This is not a support community.
It is not a place for 'how do I?', type questions. If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email [email protected]. For other questions check our partnered communities list, or use the search function.


6) No US Politics.
Please don't post about current US Politics. If you need to do this, try [email protected] or [email protected]


Reminder: The terms of service apply here too.

Partnered Communities:

Tech Support

No Stupid Questions

You Should Know

Reddit

Jokes

Ask Ouija


Logo design credit goes to: tubbadu


founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 2 points 1 month ago* (last edited 1 month ago)

I think speculation and guesswork is perfectly fine. It's part of a path towards an answer. However, that speculation and guesswork needs to have its uncertainty clearly indicated.

I'll give an example using football.As "analytics" have emerged, everyone has their own model to give a guideline on decisions. This is done using things like "win probability" of all the possible choices and outcomes. You can do out the math, using a model, to say something like "going for it gives you a 35% chance to win, and kicking the field goal gives you a 33% chance".

And that sounds great. But, all the numbers that go into that math are incredibly noisy, with very small sample sizes. A great kicker has a better chance of making a field goal than a bad kicker, and they can account for that, to a point. But they can't really account for that, plus the specific weather conditions, plus the kicker is a little sore today, ...

And the chances of a stop, and of scoring if you're successful, etc, are even worse, because it's specific to how your offense matches up to that defense, plus the context of the game, the context in the game/moment, etc.

It's perfectly fine, and reasonable, to use a model as the best indicator you have and make a decision aided by that model. But the correct way to present statistical models is provide some guidance on how uncertain it is, in addition to the raw number. If you phrase that "35% +/- 10% if you go for it, 33% +/- 10% if you kick", you realize that there's a significant range where a better model might tell you to make the opposite decision, and it's a lot closer to a toss up.

But despite the inherent uncertainty due to the limited sample sizes used to create the models, you see "analytics experts" all over the place calling coaches morons for decisions that are pretty ambiguous because their specific model gives one decision a small edge and it didn't work out. If they had explicitly evaluated and acknowledged the uncertainty of their model given the factors it can't account for, they would have a much clearer picture of what the decision actually was.

Make guesses. Speculate. But make it clear (to others, and yourself) what you're doing so the guesses aren't given more weight than they deserve.