I trained a small transformer in 1.5hrs and it beats many LLMs

A small transformer was trained from scratch in 1.5 hours on a 5090, achieving performance comparable to TRM/HRM and outperforming many LLMs. The training utilized ARC-2, a dataset containing 773 ARC-1 puzzles and 347 new ones. To prevent data leakage, the 773 repeated ARC-1 puzzles were carefully filtered out, ensuring a fair evaluation. The author acknowledges that real-life problem sets rarely present all problems simultaneously, similar to an exam where humans typically tackle one problem at a time.