架构

Layer Normalization

定义

A technique that normalises activations across the feature dimension within a single layer, stabilising training of deep networks. Layer norm is standard in transformer architectures, enabling effective training at billion-parameter scale.

相关术语

Transformer

The dominant neural network architecture for language, vision, and multimodal AI, introduced in the 2017 "Attention Is All You Need" paper. Transformers use self-attention to process all tokens in parallel, enabling training on internet-scale data and powering every major LLM in use today.

Batch Normalization

A technique that normalises activations across the batch dimension during training, accelerating convergence and reducing sensitivity to hyperparameters. Though largely superseded by layer norm in transformers, batch norm remains prevalent in convolutional vision architectures.

Residual Connection

A shortcut connection that adds a layer's input directly to its output, allowing gradients to flow more easily during training and enabling very deep networks. Residual connections are essential in transformers and were pioneered in the ResNet image model family.

了解术语只是第一步，将其落地应用才是第二步。

预约一次 Physical AI 适配性沟通，探讨这些 AI 概念如何转化到您所在的具体行业与业务挑战中。