Self-supervised learning: Training a model using labels that are naturally part of the input data, rather than requiring separate external labels.

Self-supervised learning is often used to pre-train models. By fine-tuning a pre-trained model, we can very quickly get state-of-the-art results with very little data.

Supervised learning (data that requires human annotators) are expensive, labor-intensive, and time-consuming. Transfer learning lowers the barrier for Machine Learning, making it accessible for everyone.

Let’s take a look at some examples of self-supervised learning!

Image

Context Encoders: a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings.

…

Ensemble learning is the process by which multiple models, such as classifiers or experts, are strategically generated and combined to solve a particular computational intelligence problem.

Ensembling works because it averages the errors of many models. The kinds of errors that each model make would be different from the others, so the average of their predictions should be better than either one’s individual predictions.

Ensembling works better when the models are diverse, so many ensemble methods promote diverse models to combine.

For example, in Chapter 9 of Fastai’s course, we learned to create a random forest model to predict tabular…