Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

Spnda is a tool designed to detect hallucinations in local language models ranging from 1.5B to 120B parameters without consuming significant VRAM. It works by sampling multiple responses from a local model using ollama.generate and then running a zero-cost entropy check on the CPU with compute_spanda. This process, which takes approximately 1.5 microseconds, calculates a risk score where 0 indicates high confidence and 1 indicates high uncertainty, helping to identify potential model hallucinations efficiently.