Paper summary
Paper summary: LLM360: Towards Fully Transparent Open-Source LLMs
LLM360 argues for releasing everything behind an LLM (data, code, intermediate checkpoints, analyses) and starts with two fully open 7B models, Amber and CrystalCoder.
- "Open weights" isn't the same as open training: LLM360 releases the data, the code and the intermediate checkpoints.
- Intermediate checkpoints let others study how capabilities appear during training.