Home Business Why deep networks generalize despite going against statistical intuition

Why deep networks generalize despite going against statistical intuition

0
Why deep networks generalize despite going against statistical intuition

Introductory statistics courses teach us that, when fitting a model to some data, we should have more data than free parameters to avoid the danger of overfitting—fitting noisy data too closely, and thereby failing to fit new data. It is surprising, then, that in modern deep learning the practice is to have orders of magnitude more parameters than data. Despite this, deep networks show good predictive performance, and in fact do better the more parameters they have. Why would that be?

Read More