Deepity: A C++ library showing Predictive Coding Networks can match Backprop (97.73% on MNIST in 60s) [P]
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Deepity 是一个用 C++ 构建的机器学习库，旨在测试替代的信用分配算法，特别是预测编码网络（PCNs）。该开发者通过实现最新的研究成果“通过直接 Kolen-Pollack 反馈对齐加速 PCNs”，并利用算法缓存来绕过推理稳定阶段的冗余前向投影，显著提升了性能。这使得 Deepity 在 MNIST 数据集上实现了 97.73% 的准确率，并在 60 秒内完成训练，证明了 PCNs 在 CPU 上可以与反向传播的性能相媲美，尽管朴素的 PCN 实现通常速度较慢。
I've spent the last month building a local C++ machine learning library called Deepity to test alternative credit assignment algorithms; specifically Predictive Coding Networks (PCNs). While PCNs are fascinating for biological plausibility and continual learning, naive implementations are painfully slow.
By implementing recent research ( Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment ) and utilizing algorithmic caching to bypass redundant forward projections during the inference settling phase, I managed to close the performance gap with backpropagation on my CPU when training on MNIST (50 epochs).
- PyTorch Backprop (Feedforward): 98.27% test accuracy in ~70s.
- Deepity DKPPCN: 97.73% test accuracy in 59.5s.
Next up is porting these kernels to CUDA to scale up the architecture and testing its capabilities in continual learning scenarios where standard backprop struggles.
If you are interested in local learning, alternative credit assignment, or HPC for ML, I'd love your feedback!
- GitHub: https://github.com/ra4ster/deepity
- Project Site: https://ra4ster.github.io/Deepity