Back to section
Event · now

Learning from experience

Lightweight adapters

Training small separate modules — a compromise between frozen weights and full retraining.

Parameter-efficient tuning updates a small part of the system and is therefore cheaper than full retraining. It supports controlled specialisation, but does not by itself solve continual learning: sequential updates still have to be tested for loss of earlier skills.

Sources
Microsoft Research · LoRAOpen primary source