Techniques

Self-Supervised Learning

Definition

A training paradigm where a model generates its own supervision signal from unlabelled data—for example, by predicting masked words in a sentence. Self-supervised pre-training on internet-scale text or images is what enables foundation models to acquire broad world knowledge.

Related Terms

Foundation Model

A large AI model trained on broad data at scale that can be adapted to a wide range of downstream tasks through fine-tuning or prompting. GPT-4, Claude, and Gemini are examples; under the EU AI Act, providers of GPAI models face specific transparency obligations.

Unsupervised Learning

A machine learning approach that discovers structure in unlabelled data. Common techniques include clustering (grouping similar items) and dimensionality reduction. It is often used for anomaly detection and exploratory data analysis.

Training (AI)

The process of exposing a model to large volumes of labelled or unlabelled data and adjusting its internal parameters to minimise prediction errors. Training is computationally intensive, often performed on GPU clusters, and constitutes the major cost in foundation model development.

Knowing the Terms Is Step One. Applying Them Is Step Two.

Book a 30-minute call to discuss how these AI concepts translate to your specific industry and business challenges.