We swapped AdamW's optimizer states for a Fast Fourier Transform (FFT) to cut VRAM in half. Anyone else trying non-quantization methods?

A developer community post discusses a method to reduce VRAM usage by replacing AdamW's optimizer states with a Fast Fourier Transform (FFT). This technique reportedly halved VRAM consumption, addressing Out-Of-Memory (OOM) errors encountered when fine-tuning 8B and 70B models on consumer GPUs. The author is seeking others who are exploring frequency-domain methods or other non-quantization approaches for VRAM reduction.