Sakana

This experiment shows an AI agent autonomously improving a small GPT’s training recipe. Using AutoResearch (Karpathy et al.) – which iteratively edits training code, runs experiments, and keeps only changes that lower validation bits-per-byte (BPB) – the agent ran 123 … Read more