What do we call the process of improving a machine learning model's performance during training?

The process of improving a machine learning model's performance during training refers to model tuning. This term encompasses several techniques that directly adjust or improve the model to achieve better results on given data. Model tuning includes a variety of practices, such as adjusting the model architecture, optimizing the loss function, and selecting the appropriate evaluation metrics to measure performance.

In practice, model tuning can involve iterative processes where adjustments are made based on the model's performance on validation datasets. These adjustments help to fine-tune the model, leading to improved accuracy, reduced loss, or better generalization to unseen data.

While data augmentation, feature selection, and hyperparameter optimization are also relevant concepts in the context of machine learning, they serve different purposes. Data augmentation refers to techniques used to increase the diversity of training data by adding modified versions of existing data points. Feature selection involves choosing the most relevant variables to include in the model, thereby improving efficiency and potentially enhancing performance. Hyperparameter optimization, on the other hand, is a part of model tuning focused specifically on adjusting the hyperparameters of a model—these are the parameters that are not learned within the training process and require manual setting.

Model tuning, thus, encapsulates a broader range of activities aimed at optimizing performance during the training