Paper summary
Paper summary: Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds
Lumine is an open recipe for a generalist game-playing agent built on a vision-language model: pixels in at 5 Hz, keyboard and mouse out at 30 Hz, completing a five-hour storyline and transferring zero-shot to other games.
- The agent plays like a person: it sees raw pixels and presses keys, and only stops to reason when needed.
- Without fine-tuning, it completed long missions in two other games it had never been trained on.