Architecture

Transformer

Definition

The dominant neural network architecture for language, vision, and multimodal AI, introduced in the 2017 "Attention Is All You Need" paper. Transformers use self-attention to process all tokens in parallel, enabling training on internet-scale data and powering every major LLM in use today.

Related Terms

Large Language Model (LLM)

AI models trained on vast amounts of text data that can understand and generate human-like text. Examples include GPT-4, Claude, and Llama. LLMs power modern chatbots, content generation, and code assistance tools.

Attention Mechanism

A neural network component that allows a model to dynamically focus on the most relevant parts of its input when producing each output element. Self-attention is the core innovation of the Transformer architecture and is responsible for LLMs' ability to handle long, complex contexts.

Deep Learning

A subset of machine learning based on artificial neural networks with multiple layers. Deep learning can learn complex patterns from large amounts of data and is particularly effective for image recognition, speech processing, and natural language understanding.

Knowing the Terms Is Step One. Applying Them Is Step Two.

Book a Physical AI Fit Call to discuss how these AI concepts translate to your specific industry and business challenges.