Architecture

Logit

Definition

The raw, unnormalised score produced by a model's final layer before the softmax function is applied. Logits are useful for calibration, temperature scaling, and understanding a model's confidence distribution across possible output classes.

Related Terms

Softmax

A mathematical function that converts a vector of raw logits into a probability distribution summing to 1. Softmax is applied at the output of classifiers and language models to produce token probabilities used in sampling and decoding.

Temperature (LLM)

A sampling parameter that controls the randomness of an LLM's output. A temperature of 0 produces deterministic, focused responses; higher values introduce creative variability. Selecting the right temperature is part of operationalising LLMs in production workflows.

Cross-Entropy Loss

The most common training objective for classification and language modelling tasks, measuring the difference between predicted probability distributions and true labels. Minimising cross-entropy loss during training drives the model to assign high probability to correct outputs.

Knowing the Terms Is Step One. Applying Them Is Step Two.

Book a 30-minute call to discuss how these AI concepts translate to your specific industry and business challenges.