when a run is wrong but nothing actually failed, where do you start? [D] [R]

Developers are discussing debugging strategies for situations where a workflow completes successfully without errors, exceptions, or timeouts, yet produces an incorrect final result. They are seeking advice on initial steps, such as working backward from the output, comparing with previous good runs, inspecting state transitions, checking retrieval/tool behavior, or examining model inputs. The community is interested in practical, production-level approaches and any tools built to assist with this challenging debugging scenario.

Unlike typical debugging where errors or failures are evident, this discussion focuses on the more challenging scenario of 'silent failures' where systems report success despite incorrect outputs.