Sub-1-Bit LLM Compression via Latent Factorization

The LittleBit Project introduces a method for sub-1-bit LLM compression using latent factorization. This project, licensed under CC BY-NC 4.0, utilizes a CUDA-enabled Python script for training, as demonstrated by a command line example. Key parameters include model_id meta-llama/Llama-2-7b-hf, dataset c4_wiki, num_train_epochs 5.0, per_device_train_batch_size 4, lr 4e-05, quant_func SmoothSign, quant_mod LittleBitLinear, and eff_bit 1.0.