Thinking that we’ll get safety by CoT traces is wishful thinking. Safety lives in the harness, not the chain of thought

The discourse surrounding Astra's launch highlights concerns about AI safety, particularly regarding the use of recurrent depth and Chain of Thought (CoT) traces. While CoT can be useful for monitoring and post-incident analysis, it is not a faithful transcript of a model's behavior and can be manipulated by LLM providers. Researchers emphasize that readable reasoning is merely evidence, whereas the "harness" provides actual control, urging against confusing the two for ensuring safety.

This report uniquely argues that Chain of Thought (CoT) traces are not reliable for AI safety, unlike common assumptions, because they can be manipulated and do not fully reflect a model's internal processes.