Update: Strix Halo + R9700 with llama-halo-hybrid - now beats DGX Spark

An experiment with Strix Halo and R9700, utilizing a proof-of-concept called llama-halo-hybrid, has shown significant performance improvements, now surpassing DGX Spark. This setup allows for dense model parts, KV, and some layers to be placed on the GPU, with the APU handling the rest. The extra GPU can be added via a PCIe extender, Occulink, or a Thunderbolt dock. The project is open-source and MIT-licensed, with detailed notes and code available on GitHub. Cost analysis shows Strix offers a lower price per GB compared to Gorgon.