mini-AGI - dynamically grown (530M params currently and growing) continual learning model trained from scratch on 8GB VRAM laptop from batch-1 stream of data.

A developer is training a "mini-AGI" continual learning model from scratch on an 8GB VRAM laptop. The model, currently at 530M parameters and growing, is processing a 7.8B character corpus. It reads continuous interleaved passages, each 32K characters long, as a single stream. The weights are not yet available, with an estimated couple of weeks until they are fully processed.

This report details a unique approach to AGI development, training a 530M parameter model from scratch on an 8GB VRAM laptop, unlike typical large-scale, high-resource training.