Techniques

LoRA (Low-Rank Adaptation)

Definition

A parameter-efficient fine-tuning method that injects trainable low-rank matrices into transformer layers instead of updating all model weights. LoRA enables cost-effective domain adaptation of large models on modest hardware, making enterprise fine-tuning accessible without full GPU clusters.

Related Terms

PEFT (Parameter-Efficient Fine-Tuning)

A family of techniques—including LoRA, prefix tuning, and adapters—that adapt large pre-trained models to new tasks by training only a small fraction of parameters. PEFT dramatically reduces compute and storage costs compared to full fine-tuning.

Fine-tuning

The process of taking a pre-trained AI model and further training it on a specific dataset to adapt it for a particular task or domain. Fine-tuning can deliver significantly better performance than prompting alone for specialised enterprise workflows.

Transformer

The dominant neural network architecture for language, vision, and multimodal AI, introduced in the 2017 "Attention Is All You Need" paper. Transformers use self-attention to process all tokens in parallel, enabling training on internet-scale data and powering every major LLM in use today.

Knowing the Terms Is Step One. Applying Them Is Step Two.

Book a 30-minute call to discuss how these AI concepts translate to your specific industry and business challenges.