Architecture

GPT Architecture

Definition

A family of autoregressive, decoder-only transformer architectures pre-trained to predict the next token. GPT models are the basis for ChatGPT and most commercial LLM APIs. Their autoregressive design makes them excellent text generators but less efficient for classification tasks than encoder models.

Related Terms

Transformer

The dominant neural network architecture for language, vision, and multimodal AI, introduced in the 2017 "Attention Is All You Need" paper. Transformers use self-attention to process all tokens in parallel, enabling training on internet-scale data and powering every major LLM in use today.

Large Language Model (LLM)

AI models trained on vast amounts of text data that can understand and generate human-like text. Examples include GPT-4, Claude, and Llama. LLMs power modern chatbots, content generation, and code assistance tools.

Foundation Model

A large AI model trained on broad data at scale that can be adapted to a wide range of downstream tasks through fine-tuning or prompting. GPT-4, Claude, and Gemini are examples; under the EU AI Act, providers of GPAI models face specific transparency obligations.

Knowing the Terms Is Step One. Applying Them Is Step Two.

Book a 30-minute call to discuss how these AI concepts translate to your specific industry and business challenges.