A lot of "prompting" problems are really context retrieval problems

A carefully written system prompt doesn't help much if the model is looking at the wrong document section. In document-heavy workflows, we often waste time tweaking instructions like "answer based only on the provided text," when the real issue is that the provided text is just a messy, giant chunk someone manually copy-pasted. If the actual answer is buried inside an unorganized local PDF, spec, or markdown note, the battle is already lost before the prompt even runs.

I ended up re-architecting my pipeline by wrapping Linkly AI over MCP to separate retrieval from prompting entirely.

Instead of forcing the prompt to compensate for bad context, the tool exposes the directories to the LLM as granular primitives. The agent searches the directory map, inspects the document outline, and reads the precise snippet needed on the fly. This keeps local files local, but more importantly, it drastically simplifies the prompt engineering side. The instructions can be shorter and hyper-strict: treat this retrieved section as the absolute source of truth, cite it, and do not hallucinate beyond it.

The retrieval layer works flawlessly now, but I'm running into a design dilemma on the system prompt side. For those building multi-step agents: how strict do you make your system instructions to keep the model from getting lazy? I'm trying to balance getting the agent to aggressively use the read tool when an outline looks ambiguous, without letting its Chain-of-Thought planning run so long that the turn latency becomes a bottleneck. Any elegant prompt constraints you guys swear by for tool-calling loops like this?