GUI harness [video]

A research project is developing a GUI harness, aiming for a Minecraft-like experience where users or LLMs can build complex worlds using simple tools. The system allows users to interact with it, and an LLM generates code that outputs vector components, similar to .SVG. This code utilizes only seven basic vector graphics functions (Rectangle, Circle, Line, Image, Text, List, Div) to construct all GUI elements, enabling full hackability and the creation of novel components. Every pixel displayed is vector graphics generated by the LLM's code.