Techniques

Top-p Sampling (Nucleus Sampling)

Définition

A decoding strategy that restricts the model's next-token choices to the smallest set of tokens whose cumulative probability exceeds a threshold p. Used alongside temperature, top-p sampling balances output diversity and coherence in production LLM deployments.

Termes Connexes

Temperature (LLM)

A sampling parameter that controls the randomness of an LLM's output. A temperature of 0 produces deterministic, focused responses; higher values introduce creative variability. Selecting the right temperature is part of operationalising LLMs in production workflows.

Large Language Model (LLM)

AI models trained on vast amounts of text data that can understand and generate human-like text. Examples include GPT-4, Claude, and Llama. LLMs power modern chatbots, content generation, and code assistance tools.

Inference

The process of running a trained model on new data to produce predictions or generated outputs. Inference cost and latency are the dominant operational concerns in production AI, particularly for large generative models that can cost cents per request at scale.

Besoin d'Aide pour Comprendre l'IA?

Réservez une consultation pour discuter de l'application des concepts IA à vos défis.