Arjun Nair

Answered How much does temperature actually matter for code generation?

Format rules go in the system prompt, and it caches, which matters more as the prompt grows. The "closer to the end is followed better" effect is real but it is about recency across the whole assembled context, not about which field the text sits in. If you have a long user message, a one-line reminder at the end of it is cheap and closes most of the gap without duplicating 400 tokens.

Asked a question Is RAG still worth it now that context windows are enormous?

We have about 40MB of documentation. That does not fit in any context window, so retrieval is unavoidable at that size. But for the many teams with 200 pages total — is retrieval still the right call, or does stuffing the lot in and paying for the tokens win on both quality and complexity?