Deepity: A C++ library showing Predictive Coding Networks can match Backprop (97.73% on MNIST in 60s) [P]
Heat trend
Collecting trend data
The percentage is based on available heat signal, not comment count or independent people.
Deepity is a new C++ machine learning library designed to test alternative credit assignment algorithms, specifically Predictive Coding Networks (PCNs). The developer implemented recent research, "Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment," and used algorithmic caching to improve performance. This allowed Deepity to achieve 97.73% accuracy on MNIST in 60 seconds, demonstrating that PCNs can match backpropagation's performance on a CPU, despite naive PCN implementations being slow.
I've spent the last month building a local C++ machine learning library called Deepity to test alternative credit assignment algorithms; specifically Predictive Coding Networks (PCNs). While PCNs are fascinating for biological plausibility and continual learning, naive implementations are painfully slow.
By implementing recent research ( Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment ) and utilizing algorithmic caching to bypass redundant forward projections during the inference settling phase, I managed to close the performance gap with backpropagation on my CPU when training on MNIST (50 epochs).
- PyTorch Backprop (Feedforward): 98.27% test accuracy in ~70s.
- Deepity DKPPCN: 97.73% test accuracy in 59.5s.
Next up is porting these kernels to CUDA to scale up the architecture and testing its capabilities in continual learning scenarios where standard backprop struggles.
If you are interested in local learning, alternative credit assignment, or HPC for ML, I'd love your feedback!
- GitHub: https://github.com/ra4ster/deepity
- Project Site: https://ra4ster.github.io/Deepity