- Nov 11, 2019
- 10,619
- 394
- 0
Is the simplest explanation always the best one, as Occam's razor states?
What is Occam's razor? : Read more
What is Occam's razor? : Read more
A math teacher once told our class that Occam's razor was used to shave Plato's beard.It's often been said that in Biology Occam's Razor is best suited to cut one's own throat.
But it often comes down to the purpose of the explanation or model. Sure, given enough data, more complex models will often produce better predictions, but without those data, or data that reflects fundamentally different distributions, more complex models tend to overfit and make lousy predictions on cases worthy of prediction -- as Niels Bohr said, 'Prediction is difficult, especially of the future.'
There are several often used entropy based measures to help find the right balance between complexity and simplicity -- Akaike information criterion is a popular one.
Explanation or models meant for teaching or human understanding of course have limitations in complexity. Everyone knows that simpler models that delineate cause and effect are heuristics, but often immensely valuable ones.