Paper summary
Paper summary: OLMo: Accelerating the Science of Language Models
OLMo is a competitive language model released with its training data, training code and evaluation code, not just weights, so others can study how such models are built.
- "Truly open" here means weights plus the pre-training data plus the training and evaluation code.
- The authors list limitations for each stage: data, training, adaptation and evaluation.