Skip to content
RCreddit.com·
Not on the current live radar

MoE SSD streaming on a 64 GB Mac mini: GPU still waits 27% of decode on experts. Ideas?

AI summary

A developer is seeking ideas to optimize MoE SSD streaming on a 64 GB Mac mini, where the GPU experiences a 27% decode wait on experts. They are already implementing lookahead guess fetching for subsequent layers' experts with 72% accuracy. The developer is considering expanding this lookahead strategy and using a separate staging buffer for guessed experts to prevent eviction of frequently used experts. They provided a GitHub link for further context.

Time & source

Times shown in UTC

Display time zone: UTC

Local time zone unavailable; showing UTC.

IngestedOffset at this time: UTC+0Oct 5, 2026, 18:00 UTC

Ingested
Oct 5, 2026, 18:00
Source type
Dev community

Full text isn't available here.

Read at source →
Source·reddit.com