#pruning

• 27 Mar 2024 · Paper summary

  Paper summary: The Unreasonable Ineffectiveness of the Deeper Layers

  Removing up to about half of the layers of some open LLMs, chosen by layer similarity and followed by a little QLoRA "healing", barely hurts question-answering performance.