Extensible Software in the age of LLMs

Modern web software often serves the majority of users with static features, leaving a long-tail of unmet needs. While traditional trace waterfall diagrams are useful for request/response systems, modern applications with stateful or dynamic elements, like non-deterministic agents or durable workflow engines, require more flexible visualization. Trace spans offer a good foundation, but extensibility for custom visualizations is crucial. This extensibility, however, becomes challenging for software handling sensitive data like customer records or financial transactions.