Architecture

Residual Connection

Définition

A shortcut connection that adds a layer's input directly to its output, allowing gradients to flow more easily during training and enabling very deep networks. Residual connections are essential in transformers and were pioneered in the ResNet image model family.

Termes Connexes

Layer Normalization

A technique that normalises activations across the feature dimension within a single layer, stabilising training of deep networks. Layer norm is standard in transformer architectures, enabling effective training at billion-parameter scale.

Deep Learning

A subset of machine learning based on artificial neural networks with multiple layers. Deep learning can learn complex patterns from large amounts of data and is particularly effective for image recognition, speech processing, and natural language understanding.

Transformer

The dominant neural network architecture for language, vision, and multimodal AI, introduced in the 2017 "Attention Is All You Need" paper. Transformers use self-attention to process all tokens in parallel, enabling training on internet-scale data and powering every major LLM in use today.

Besoin d'Aide pour Comprendre l'IA?

Réservez un appel de cadrage Physical AI pour discuter de l'application de ces concepts IA à votre secteur et vos défis métier.