Complete local model asset generation pipeline
This covers a coding tool or code-capability update — useful for developers assessing workflow changes and reusable value.
A developer shared their new feature set for local model asset generation, stemming from research into LLMs for an isometric RPG.…
So I figured I'd update the community given I just shipped a nice little feature set and feel like sharing it finally :)
In the past few weeks, I've been test-coding an isometric RPG game/engine in Three.js, as part of my research into how LLMs work at scale in higher quality projects written from scratch (spoiler: they don't, even the SOTA ones). For that, I needed a complete team of virtual creators ;) and working through the Python pipelines for all those models is insanely frustrating (bonus points for doing that on a Strix Halo box), so I decided to port that to GGML .
Fortunately, for AceStep I didn't have to do anything since u/webdelic made an AceStep.cpp already ( https://www.reddit.com/r/LocalLLaMA/comments/1ry1dy1/acestepcpp_portable_c17_implementation_of_acestep ), so all I had to do was to add some CIs for building artifacts on my fork. But I did port three other things:
https://github.com/pwilkin/openmoss https://github.com/pwilkin/thinksound.cpp https://github.com/pwilkin/trellis.cpp All of those are standalone tools you can use for asset generation, but there's more! Thanks to the great folks at Lemonade who reached out to me for a little cooperation, the entirety of those features (summarized here: https://github.com/lemonade-sdk/lemonade/issues/2529 ) are now going to be available in the newest build of Lemonade. This includes nice stuff such as cascading model calls (Trellis.2 is an image-to-3D model, but you can cascade your favorite text-to-image model that uses the stablediffusion.cpp engine in Lemonade to run a full text-to-3D cycle).
How does it work? Well, here's a sample screenshot from my game - all of this has been generated using either procedurals in Blender or with the models described here. In other words: all free tools on permissive open-source licenses. Hope others have as much fun with this as I do :)
EDIT: Oh yeah, forgot to mention. All the engines ship with CUDA + Vulkan + ROCm support, so most hardware covered (I don't have a Mac unfortunately, so no Mac, happy to accept PRs).