In a very surprising paper Steven Piantadosi shows that a simple function of one parameter (θ) can fit any collection of ordered pairs {Xi,Yi} to arbitrary precision. In other words, the same simple function can fit any scatter plot exactly, just by choosing the right θ. The intuition comes from chaos theory. We know from chaos theory that simple functions can produce seemingly random, chaotic behavior and that tiny changes in initial conditions can quickly result in entirely different outcomes (the butterfly effect). What Piantadosi shows is that the space traversed in these functions by changing θ is so thick that you can reverse the procedure to find a function that fits any scatter plot.

the paper also tells us that Occam’s Razor is wrong. Overfitting is possible with just one parameter and so models with fewer parameters are not necessarily preferable even if they fit the data as well or better than models with more parameters.