Are there any theoretically-guided practices left in machine learning nowadays? [D]

There was a period in the development of machine learning where application seemed to be informed by theory. Some of the best known theories include: If you train a model with too much data, then you get overfitting and your test performance will be suffer. Big models do not generalize because theoretically you will never have enough data. Never train on the test set, because it will result in high-bias. Never even look at the test set, because you as a modeller will instantly be biased and use the wrong model. Good results can only come from "compatible" models and optimization routines. You