3. Statistical criteria of bias–variance tradeoff and analysis of 60 exchange–correlation functionals
The dispute between counting parameters and analytical fits is not a new
scene in statistics and machine learning, where the problem is generally
known as the bias/variance dilemma \cite{geman_neural_1992}. Especially in supervised
learning, where a model is learned from (fitted to) some training data,
this dilemma translates to the necessity to strike a balance between
underfitting the data (bias error), resulting in methods that don’t
incorporate all the relations between the data, and overfitting them
(variance), resulting in methods that are poorly transferrable. Several
criteria for model selection are available in this context, and they all
generally include two components, one that accounts for the performance
of the model on the training data, and another that accounts for the
transferability of the model to unseen data. For a good introduction,
see \cite{cherkassky_learning_2007}. The goal of the next section is to
borrow some of the methodologies that have been developed in the context
of supervised learning and apply them to the analysis of DFT
approximations.