Paper summary
Paper summary: Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
Drag-and-Drop LLMs train a generator that maps a few unlabelled task prompts straight to LoRA weights, specialising an LLM for a new task in seconds without any gradient-based fine-tuning on it.
- Instead of training a LoRA per task, a hypernetwork generates the LoRA weights from example prompts.
- The paper reports up to 12,000× lower overhead than full fine-tuning.