ESP32S3 cluster running 1.58-bit (BitNet) Language model

A distributed pipeline inference engine has been developed, running a 1.58-bit (BitNet) Language model on multiple ESP32S3 microcontrollers. The system utilizes a master node for prompt processing, BPE Tokenizer, and Token Embedding (INT4), distributing layers 0 to 23 across compute nodes (1 to 6). Each compute node handles 4x Transformer Blocks with 1.58-bit Attention and MLP, using FP16 scaled to FP32 for RMSNorm and PSRAM for KV Cache. The master node then performs final RMS Norm and LM Head for greedy sampling.

This project is the first to demonstrate a 1.58-bit BitNet language model running on a cluster of ESP32S3 microcontrollers, unlike previous implementations on more powerful hardware.